asterisk users - Apr 2007 - Problems with queue announcements under high call volumes

List members,

Our inbound call center Asterisk server got slammed today.  At around 
4:30 there were 500-550 calls and roughly 675 active channels (all 
SIP).  I'd estimate that 150-200 of the calls were connected to agents 
and being recorded, but the majority were in queue.

Overall, the box performed very well.  The CPU ranged from about 10%-20% 
idle (with scary load averages of 60-80), and only 500 MB of RAM were 
used (minus buffers and cache).  On test calls, the quality was good 
once you were connected to an agent.  Music on hold (using the native 
format) and the call recordings both sounded good, and our agents 
reported no problems with overall call quality.

However, we did experience problems with queue announcements, audio file 
playbacks, and our IVR.   My hunch is that these problems are closely 
related so I'm treating them as one issue, but please correct me if I'm 
mistaken.  Overall, the audio quality in these scenarios was 
understandable, but with intermittent drops that were accompanied by a 
tinny sound.  Unfortunately, in our busiest queue (which had 150-250 
people in it) the drops made the playback completely inaudible.  
Asterisk's log file reflected these problems with a large number of the 
following message:

  Apr 16 14:30:01 WARNING[19451] file.c: Failed to write frame

Given that we are stuck with the hardware and the call volume we have, 
can anyone suggest any methods for addressing this problem?  Note that 
all of our audio files are in the native format for our codec and that 
we perform no transcoding on the Asterisk server.  An outright fix, a 
stop gap solution, or just some brainstorming on how to increase the 
number of audio files that Asterisk can play back concurrently would be 
greatly appreciated.

Thank you,

Matthew Roth
InterMedia Marketing Solutions
Software Engineer and Systems Developer

On Mon, 2007-04-16 at 18:18 -0400, Matthew J. Roth wrote:
[snip]
>   Apr 16 14:30:01 WARNING[19451] file.c: Failed to write frame
[snip]

Just a wild guess because I don't really have an idea what is causing
this but are your ulimit settings high enough?

Regards,
Patrick