Dovid Bender
2020-Jun-28 17:25 UTC
[asterisk-users] Exceptionally long queue length queuing
Hi, We have a box up and we are starting to see a lot of "Exceptionally long queue length queuing" in the logs. From all the research so far it seems like this leads to their systems crashing and being unreachable. In our case the box remains up and takes calls. We are running Asterisk 16.6.1. We are using MusicOnHold to play online music streams via ffmpeg. Any idea on how to troubleshoot this further to see why this is happening? TIA. Regards, Dovid -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20200628/089c7215/attachment.html>
Joshua C. Colp
2020-Jun-29 10:45 UTC
[asterisk-users] Exceptionally long queue length queuing
On Sun, Jun 28, 2020 at 2:26 PM Dovid Bender <dovid at telecurve.com> wrote:> Hi, > > We have a box up and we are starting to see a lot of "Exceptionally long > queue length queuing" in the logs. From all the research so far it seems > like this leads to their systems crashing and being unreachable. In our > case the box remains up and takes calls. We are running Asterisk 16.6.1. We > are using MusicOnHold to play online music streams via ffmpeg. Any idea on > how to troubleshoot this further to see why this is happening? >The message occurs when either a channel is deadlocked or the processing time to handle what a channel is doing is substantial, and things build up. Getting a backtrace[1] at the time it occurs will show what the system is actually doing and can provide insight into what is causing it. [1] https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace#GettingaBacktrace-Runningast_coredumperfordeadlocks,taskprocessorbackups,etc . -- Joshua C. Colp Asterisk Technical Lead Sangoma Technologies Check us out at www.sangoma.com and www.asterisk.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20200629/785f378d/attachment.html>
Dovid Bender
2020-Jul-03 06:31 UTC
[asterisk-users] Exceptionally long queue length queuing
On Mon, Jun 29, 2020 at 6:46 AM Joshua C. Colp <jcolp at sangoma.com> wrote:> On Sun, Jun 28, 2020 at 2:26 PM Dovid Bender <dovid at telecurve.com> wrote: > >> Hi, >> >> We have a box up and we are starting to see a lot of "Exceptionally long >> queue length queuing" in the logs. From all the research so far it seems >> like this leads to their systems crashing and being unreachable. In our >> case the box remains up and takes calls. We are running Asterisk 16.6.1. We >> are using MusicOnHold to play online music streams via ffmpeg. Any idea on >> how to troubleshoot this further to see why this is happening? >> > > The message occurs when either a channel is deadlocked or the processing > time to handle what a channel is doing is substantial, and things build up. > Getting a backtrace[1] at the time it occurs will show what the system is > actually doing and can provide insight into what is causing it. > > [1] > https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace#GettingaBacktrace-Runningast_coredumperfordeadlocks,taskprocessorbackups,etc > . > > -- >I ended up re-writing my code to use less Local channels and most of the errors went away. I also noticed that my load average and CPU usage is way down. I will open a ticket since it seems that it may be a bug that others are experiencing.>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20200703/c4d838a6/attachment.html>