Dan Cropp
2021-Sep-23 16:58 UTC
[asterisk-users] Any thoughts on Asterisk 16.17.0 outputting FRACK refcount related messages
We have an extremely busy/large customer. They run fine most of the time, but periodically asterisk will output FRACK refcount related messages. It doesn't seem to be related to the volume, because it's not breaking during their peak times. When this happens, the system becomes unstable and they have to restart to get things resolved. To give an idea of the instability, we have seen INVITE/Trying responses in SIP messaging logs. We tell Asterisk to answer via AMI, but Asterisk never sends the OK (even 24 seconds later it hasn't sent). Eventually the other send CANCEL of the call. We've now captured 4 different days where something like the following occurs. 1) Is there a good way to tell if this may be fixed in Asterisk 16.20.0 (short of upgrading)? 2) Would this be something I should submit as an asterisk issue? Unfortunately, site is so busy capturing the debug will be very difficult (if not impossible) due to amount of data. [09/23 14:43:45.095] ERROR[34763][C-00001d7f] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[34763][C-00001d7f] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[29830] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[29832][C-00001920] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[29830] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[29832][C-00001920] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[32973] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[32973] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[3248] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[3248] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] WARNING[2798][C-00000093] channel.c: Exceptionally long queue length queuing to CBAnn/IS__8b9c6719-ca29-4c1b-ac87-75e8c6fe7074-00000062;1 [09/23 14:43:45.095] ERROR[12979] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[12979] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] WARNING[21123][C-00001157] channel.c: Exceptionally long queue length queuing to CBAnn/IS__a652155f-b1fb-4c31-83e5-09ffa2107979-000010de;1 [09/23 14:43:45.096] ERROR[29830] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.097] WARNING[36475][C-00000172] channel.c: Exceptionally long voice queue length queuing to CBAnn/IS__64bc075a-1ba4-4ad8-ba48-f0aea6ca6bab-00001ed3;1 [09/23 14:43:45.098] ERROR[12979] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.098] ERROR[34763][C-00001d7f] : Got 19 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(+0xa9517) [0x5637ebf2e517] # 3: /usr/sbin/asterisk(ast_unreal_write_stream+0x15a) [0x5637ebf6ce1a] # 4: /usr/sbin/asterisk(ast_write_stream+0xd2e) [0x5637ebf4551e] # 5: /usr/sbin/asterisk(bridge_channel_internal_join+0xe6d) [0x5637ebf2238d] # 6: /usr/sbin/asterisk(ast_bridge_join+0x3d5) [0x5637ebf05ce5] # 7: /usr/lib/asterisk/modules/app_confbridge.so(+0xff35) [0x7fc05c258f35] # 8: /usr/sbin/asterisk(pbx_exec+0xd2) [0x5637ebfdcf92] # 9: /usr/lib/asterisk/modules/res_agi.so(+0xbea5) [0x7fc059269ea5] #10: /usr/lib/asterisk/modules/res_agi.so(+0xd29b) [0x7fc05926b29b] #11: /usr/lib/asterisk/modules/res_agi.so(+0xf7b0) [0x7fc05926d7b0] #12: /usr/sbin/asterisk(pbx_exec+0xd2) [0x5637ebfdcf92] #13: /usr/sbin/asterisk(+0x149062) [0x5637ebfce062] #14: /usr/sbin/asterisk(+0x14d143) [0x5637ebfd2143] #15: /usr/sbin/asterisk(+0x14e9e9) [0x5637ebfd39e9] #15: /usr/sbin/asterisk(+0x14e9e9) [0x5637ebfd39e9] #16: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] #17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] #18: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.098] ERROR[29832][C-00001920] : Got 10 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(+0xa9517) [0x5637ebf2e517] # 3: /usr/sbin/asterisk(ast_unreal_write+0x15a) [0x5637ebf6ca0a] # 4: /usr/sbin/asterisk(ast_write_stream+0xd2e) [0x5637ebf4551e] # 5: /usr/sbin/asterisk(bridge_channel_internal_join+0xe6d) [0x5637ebf2238d] # 6: /usr/sbin/asterisk(+0x806f3) [0x5637ebf056f3] # 7: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 8: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 9: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.099] ERROR[3248] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.099] ERROR[32973] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.207] ERROR[36630] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.207] ERROR[36630] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.207] ERROR[34172][C-00001cfb] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.207] ERROR[34172][C-00001cfb] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.207] ERROR[28738] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.207] ERROR[28738] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.207] ERROR[36630] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.208] WARNING[2798][C-00000093] channel.c: Exceptionally long queue length queuing to CBAnn/IS__8b9c6719-ca29-4c1b-ac87-75e8c6fe7074-00000062;1 [09/23 14:43:45.209] ERROR[34172][C-00001cfb] : Got 14 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_readframe+0x45) [0x5637ebf8c675] # 3: /usr/lib/asterisk/modules/res_musiconhold.so(+0x4ca0) [0x7fc153b07ca0] # 4: /usr/sbin/asterisk(+0xb5d6c) [0x5637ebf3ad6c] # 5: /usr/sbin/asterisk(+0xc7a47) [0x5637ebf4ca47] # 6: /usr/lib/asterisk/modules/res_agi.so(+0xeb9c) [0x7fc05926cb9c] # 7: /usr/sbin/asterisk(pbx_exec+0xd2) [0x5637ebfdcf92] # 8: /usr/sbin/asterisk(+0x149062) [0x5637ebfce062] # 9: /usr/sbin/asterisk(+0x14d143) [0x5637ebfd2143] #10: /usr/sbin/asterisk(+0x14e82b) [0x5637ebfd382b] #11: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] #12: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] #13: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.210] ERROR[28738] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] Dan This email is intended only for the use of the party to which it is addressed and may contain information that is privileged, confidential, or protected by law. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this email or its contents is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20210923/640a8e89/attachment.html>
Joshua C. Colp
2021-Sep-23 17:02 UTC
[asterisk-users] Any thoughts on Asterisk 16.17.0 outputting FRACK refcount related messages
On Thu, Sep 23, 2021 at 1:59 PM Dan Cropp <dan at amtelco.com> wrote:> We have an extremely busy/large customer. They run fine most of the time, > but periodically asterisk will output FRACK refcount related messages. It > doesn’t seem to be related to the volume, because it’s not breaking during > their peak times. > > > > When this happens, the system becomes unstable and they have to restart to > get things resolved. > > To give an idea of the instability, we have seen INVITE/Trying responses > in SIP messaging logs. > > We tell Asterisk to answer via AMI, but Asterisk never sends the OK (even > 24 seconds later it hasn’t sent). > > Eventually the other send CANCEL of the call. > > > > > > We’ve now captured 4 different days where something like the following > occurs. > > 1) Is there a good way to tell if this may be fixed in Asterisk 16.20.0 > (short of upgrading)? >You could examine the changes between the two and see if any issues seem relevant.> 2) Would this be something I should submit as an asterisk issue? > Unfortunately, site is so busy capturing the debug will be very difficult > (if not impossible) due to amount of data. >Without a backtrace, or debug logs it'd be really hard to look at. The message just means that an object was referenced a lot. Why that is - who knows. It could be that the system can't keep up for some reason, or there's an off-nominal path, or something is deadlocked, or something else. -- Joshua C. Colp Asterisk Technical Lead Sangoma Technologies Check us out at www.sangoma.com and www.asterisk.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20210923/55027448/attachment.html>
Telium Technical Support
2021-Nov-17 20:46 UTC
[asterisk-users] Any thoughts on Asterisk 16.17.0 outputting FRACK refcount related messages
I don't *think* it would purely volume related. We have 16.17 deployments with very large loads running without issue, and we also run 16.17 against load simulators without issue. In each case you have to traceback to find the cause of the problem. For example, a bad SBC which does not fully adhere to the SIP protocol could be confusing PJSIP, causing timeouts, etc. Our engineers spent a few days tracing such a problem before we shipped a high volume system to a customer earlier this year. (I have NOT traced through your logs below, so I'm not saying that is your problem) There are some helpful posts here to help you trace, in the archives of this list. (In fact I see one responding to a question you asked 2 months ago). Did that yield nothing? (The response was dead on in terms of how to diagnose). If you share the result of that diagnosis you might get some helpful answers. Dave From: asterisk-users [mailto:asterisk-users-bounces at lists.digium.com] On Behalf Of Dan Cropp Sent: Thursday, September 23, 2021 12:59 PM To: 'asterisk-users at lists.digium.com' <asterisk-users at lists.digium.com> Subject: [asterisk-users] Any thoughts on Asterisk 16.17.0 outputting FRACK refcount related messages We have an extremely busy/large customer. They run fine most of the time, but periodically asterisk will output FRACK refcount related messages. It doesn't seem to be related to the volume, because it's not breaking during their peak times. When this happens, the system becomes unstable and they have to restart to get things resolved. To give an idea of the instability, we have seen INVITE/Trying responses in SIP messaging logs. We tell Asterisk to answer via AMI, but Asterisk never sends the OK (even 24 seconds later it hasn't sent). Eventually the other send CANCEL of the call. We've now captured 4 different days where something like the following occurs. 1) Is there a good way to tell if this may be fixed in Asterisk 16.20.0 (short of upgrading)? 2) Would this be something I should submit as an asterisk issue? Unfortunately, site is so busy capturing the debug will be very difficult (if not impossible) due to amount of data. [09/23 14:43:45.095] ERROR[34763][C-00001d7f] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[34763][C-00001d7f] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[29830] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[29832][C-00001920] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[29830] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[29832][C-00001920] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[32973] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[32973] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] ERROR[3248] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[3248] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] WARNING[2798][C-00000093] channel.c: Exceptionally long queue length queuing to CBAnn/IS__8b9c6719-ca29-4c1b-ac87-75e8c6fe7074-00000062;1 [09/23 14:43:45.095] ERROR[12979] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.095] ERROR[12979] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.095] WARNING[21123][C-00001157] channel.c: Exceptionally long queue length queuing to CBAnn/IS__a652155f-b1fb-4c31-83e5-09ffa2107979-000010de;1 [09/23 14:43:45.096] ERROR[29830] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.097] WARNING[36475][C-00000172] channel.c: Exceptionally long voice queue length queuing to CBAnn/IS__64bc075a-1ba4-4ad8-ba48-f0aea6ca6bab-00001ed3;1 [09/23 14:43:45.098] ERROR[12979] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.098] ERROR[34763][C-00001d7f] : Got 19 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(+0xa9517) [0x5637ebf2e517] # 3: /usr/sbin/asterisk(ast_unreal_write_stream+0x15a) [0x5637ebf6ce1a] # 4: /usr/sbin/asterisk(ast_write_stream+0xd2e) [0x5637ebf4551e] # 5: /usr/sbin/asterisk(bridge_channel_internal_join+0xe6d) [0x5637ebf2238d] # 6: /usr/sbin/asterisk(ast_bridge_join+0x3d5) [0x5637ebf05ce5] # 7: /usr/lib/asterisk/modules/app_confbridge.so(+0xff35) [0x7fc05c258f35] # 8: /usr/sbin/asterisk(pbx_exec+0xd2) [0x5637ebfdcf92] # 9: /usr/lib/asterisk/modules/res_agi.so(+0xbea5) [0x7fc059269ea5] #10: /usr/lib/asterisk/modules/res_agi.so(+0xd29b) [0x7fc05926b29b] #11: /usr/lib/asterisk/modules/res_agi.so(+0xf7b0) [0x7fc05926d7b0] #12: /usr/sbin/asterisk(pbx_exec+0xd2) [0x5637ebfdcf92] #13: /usr/sbin/asterisk(+0x149062) [0x5637ebfce062] #14: /usr/sbin/asterisk(+0x14d143) [0x5637ebfd2143] #15: /usr/sbin/asterisk(+0x14e9e9) [0x5637ebfd39e9] #15: /usr/sbin/asterisk(+0x14e9e9) [0x5637ebfd39e9] #16: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] #17: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] #18: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.098] ERROR[29832][C-00001920] : Got 10 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(+0xa9517) [0x5637ebf2e517] # 3: /usr/sbin/asterisk(ast_unreal_write+0x15a) [0x5637ebf6ca0a] # 4: /usr/sbin/asterisk(ast_write_stream+0xd2e) [0x5637ebf4551e] # 5: /usr/sbin/asterisk(bridge_channel_internal_join+0xe6d) [0x5637ebf2238d] # 6: /usr/sbin/asterisk(+0x806f3) [0x5637ebf056f3] # 7: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 8: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 9: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.099] ERROR[3248] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.099] ERROR[32973] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.207] ERROR[36630] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.207] ERROR[36630] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.207] ERROR[34172][C-00001cfb] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.207] ERROR[34172][C-00001cfb] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.207] ERROR[28738] frame.c: Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 [09/23 14:43:45.207] ERROR[28738] frame.c: FRACK!, Failed assertion Excessive refcount 100000 reached on ao2 object 0x5637ed3742b8 (0) [09/23 14:43:45.207] ERROR[36630] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.208] WARNING[2798][C-00000093] channel.c: Exceptionally long queue length queuing to CBAnn/IS__8b9c6719-ca29-4c1b-ac87-75e8c6fe7074-00000062;1 [09/23 14:43:45.209] ERROR[34172][C-00001cfb] : Got 14 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_readframe+0x45) [0x5637ebf8c675] # 3: /usr/lib/asterisk/modules/res_musiconhold.so(+0x4ca0) [0x7fc153b07ca0] # 4: /usr/sbin/asterisk(+0xb5d6c) [0x5637ebf3ad6c] # 5: /usr/sbin/asterisk(+0xc7a47) [0x5637ebf4ca47] # 6: /usr/lib/asterisk/modules/res_agi.so(+0xeb9c) [0x7fc05926cb9c] # 7: /usr/sbin/asterisk(pbx_exec+0xd2) [0x5637ebfdcf92] # 8: /usr/sbin/asterisk(+0x149062) [0x5637ebfce062] # 9: /usr/sbin/asterisk(+0x14d143) [0x5637ebfd2143] #10: /usr/sbin/asterisk(+0x14e82b) [0x5637ebfd382b] #11: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] #12: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] #13: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] [09/23 14:43:45.210] ERROR[28738] : Got 8 backtrace records # 0: /usr/sbin/asterisk(__ao2_ref+0x209) [0x5637ebef1519] # 1: /usr/sbin/asterisk(ast_frdup+0x1e2) [0x5637ebf96612] # 2: /usr/sbin/asterisk(ast_bridge_channel_queue_frame+0x61) [0x5637ebf1cfe1] # 3: /usr/lib/asterisk/modules/bridge_softmix.so(+0x40af) [0x7fc15362f0af] # 4: /usr/lib/asterisk/modules/bridge_softmix.so(+0x560a) [0x7fc15363060a] # 5: /usr/sbin/asterisk(+0x1db41f) [0x5637ec06041f] # 6: /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7fc1e9ebf6db] # 7: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fc1e93f971f] Dan This email is intended only for the use of the party to which it is addressed and may contain information that is privileged, confidential, or protected by law. If you are not the intended recipient you are hereby notified that any dissemination, copying or distribution of this email or its contents is strictly prohibited. If you have received this message in error, please notify us immediately by replying to the message and deleting it from your computer. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.digium.com/pipermail/asterisk-users/attachments/20211117/d1ea70a7/attachment.html>