asterisk users - Sep 2005 - IAX2 Problems causing server to hang

Setup is as follows:

Server 1
Single TE110p configured for E1
40 SIP softphone clients
Voicemail to email on 3 of the extensions

Server 2
Single TE110p configured for E1
40 IAX2 softphone clients
Voicemail to email on 2 of the extensions

Server 2 has random service interruptions with a log file full of
"channel.c: Dropping voice to exceptionally long queue on IAX2...".

Solution ranges from ssh to server and restarting * to having to log on the
console and reboot the server because there is no network access.

Everything points to IAX2.  Short of changing all clients to SIP, does
anyone have any ideas?

Alex