Hello asterisk-users,
We've been having intermittent issues with chan_sip - it stops responding
to cli requests, trying to reload chan_sip from cli doesn't seem to have
any effect, initiated calls carry on for a short period, but no new SIP
requests are processed ('sip show channels' hangs forever, server stops
responding to SIP OPTIONS, or any other SIP messages). We have updated the
build from 1.8.23.1 to the latest asterisk 1.8 (1.8.32.3), however the
problem still persists. We have gathered debugging information from 'core
show locks' and from gdb, attached to this message (with phone numbers and
extension and context names obscured). We are running realtime under CentOS
6.6, built from source and packaged using rpmbuild, with the following
menuselect options (debugging version):
menuselect/menuselect --disable BUILD_NATIVE --enable DEBUG_THREADS
--enable DONT_OPTIMIZE --disable CORE-SOUNDS-EN-GSM --disable-category
MENUSELECT_EXTRA_SOUNDS --disable MOH-OPSOUND-WAV --enable-category
MENUSELECT_ADDONS --disable format_mp3 --disable cdr_tds --disable cel_tds
--disable cdr_pgsql --disable cel_pgsql --disable res_config_pgsql
menuselect.makeopts
under kernel 2.6.32-504.el6.x86_64, and linked against the following
library versions:
/usr/lib64/libssl.so.10: symbolic link to `libssl.so.1.0.1e'
/usr/lib64/libcrypto.so.10: symbolic link to `libcrypto.so.1.0.1e'
/lib64/libc.so.6: symbolic link to `libc-2.12.so'
/usr/lib64/libxml2.so.2: symbolic link to `libxml2.so.2.7.6'
/lib64/libz.so.1: symbolic link to `libz.so.1.2.3'
/lib64/libm.so.6: symbolic link to `libm-2.12.so'
/lib64/libdl.so.2: symbolic link to `libdl-2.12.so'
/lib64/libpthread.so.0: symbolic link to `libpthread-2.12.so'
/lib64/libtinfo.so.5: symbolic link to `libtinfo.so.5.7'
/lib64/libresolv.so.2: symbolic link to `libresolv-2.12.so'
/lib64/libgssapi_krb5.so.2: symbolic link to `libgssapi_krb5.so.2.2'
/lib64/libkrb5.so.3: symbolic link to `libkrb5.so.3.3'
/lib64/libcom_err.so.2: symbolic link to `libcom_err.so.2.1'
/lib64/libk5crypto.so.3: symbolic link to `libk5crypto.so.3.1'
/lib64/libkrb5support.so.0: symbolic link to `libkrb5support.so.0.1'
/lib64/libkeyutils.so.1: symbolic link to `libkeyutils.so.1.3'
We'd appreciate any possible assistance, as we're having problems
working
out what exactly triggers the deadlock and we have not been able to find
the correct sequence of steps to reproduce the issue yet, other than
waiting for it to lock up at an arbitrary time with the debugging code in
place. It does seem to happen at least once a day, however.
What is the best way of getting the core show locks output for people to
see as it appears to be too big to mail?
Ish
--
Ishfaq Malik
Department: VOIP Support
Company: Packnet Limited
t: +44 (0)845 004 4994
f: +44 (0)161 660 9825
e: ish at pack-net.co.uk
w: http://www.pack-net.co.uk
Registered Address: PACKNET LIMITED, Duplex 2, Ducie House
37 Ducie Street
Manchester, M1 2JW
COMPANY REG NO. 04920552
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.digium.com/pipermail/asterisk-users/attachments/20150429/4318194c/attachment.html>
On Wed, Apr 29, 2015 at 8:42 AM, Ishfaq Malik <ish at pack-net.co.uk> wrote:> Hello asterisk-users, > > We've been having intermittent issues with chan_sip - it stops responding to > cli requests, trying to reload chan_sip from cli doesn't seem to have any > effect, initiated calls carry on for a short period, but no new SIP requests > are processed ('sip show channels' hangs forever, server stops responding to > SIP OPTIONS, or any other SIP messages). We have updated the build from > 1.8.23.1 to the latest asterisk 1.8 (1.8.32.3), however the problem still > persists. We have gathered debugging information from 'core show locks' and > from gdb, attached to this message (with phone numbers and extension and > context names obscured). We are running realtime under CentOS 6.6, built > from source and packaged using rpmbuild, with the following menuselect > options (debugging version): > menuselect/menuselect --disable BUILD_NATIVE --enable DEBUG_THREADS --enable > DONT_OPTIMIZE --disable CORE-SOUNDS-EN-GSM --disable-category > MENUSELECT_EXTRA_SOUNDS --disable MOH-OPSOUND-WAV --enable-category > MENUSELECT_ADDONS --disable format_mp3 --disable cdr_tds --disable cel_tds > --disable cdr_pgsql --disable cel_pgsql --disable res_config_pgsql > menuselect.makeopts > > under kernel 2.6.32-504.el6.x86_64, and linked against the following library > versions: > > /usr/lib64/libssl.so.10: symbolic link to `libssl.so.1.0.1e' > /usr/lib64/libcrypto.so.10: symbolic link to `libcrypto.so.1.0.1e' > /lib64/libc.so.6: symbolic link to `libc-2.12.so' > /usr/lib64/libxml2.so.2: symbolic link to `libxml2.so.2.7.6' > /lib64/libz.so.1: symbolic link to `libz.so.1.2.3' > /lib64/libm.so.6: symbolic link to `libm-2.12.so' > /lib64/libdl.so.2: symbolic link to `libdl-2.12.so' > /lib64/libpthread.so.0: symbolic link to `libpthread-2.12.so' > /lib64/libtinfo.so.5: symbolic link to `libtinfo.so.5.7' > /lib64/libresolv.so.2: symbolic link to `libresolv-2.12.so' > /lib64/libgssapi_krb5.so.2: symbolic link to `libgssapi_krb5.so.2.2' > /lib64/libkrb5.so.3: symbolic link to `libkrb5.so.3.3' > /lib64/libcom_err.so.2: symbolic link to `libcom_err.so.2.1' > /lib64/libk5crypto.so.3: symbolic link to `libk5crypto.so.3.1' > /lib64/libkrb5support.so.0: symbolic link to `libkrb5support.so.0.1' > /lib64/libkeyutils.so.1: symbolic link to `libkeyutils.so.1.3' > > > We'd appreciate any possible assistance, as we're having problems working > out what exactly triggers the deadlock and we have not been able to find the > correct sequence of steps to reproduce the issue yet, other than waiting for > it to lock up at an arbitrary time with the debugging code in place. It does > seem to happen at least once a day, however. > > What is the best way of getting the core show locks output for people to see > as it appears to be too big to mail? >Please go ahead and make an issue on the issue tracker. Make sure you get both the output of 'core show locks', as well as a GDB backtrace. Instructions for both can be found here: https://wiki.asterisk.org/wiki/display/AST/Getting+a+Backtrace -- Matthew Jordan Digium, Inc. | Director of Technology 445 Jan Davis Drive NW - Huntsville, AL 35806 - USA Check us out at: http://digium.com & http://asterisk.org
Ishfaq Malik wrote:> Hello asterisk-users, > > We've been having intermittent issues with chan_sip - it stops > responding to cli requests, trying to reload chan_sip from cli doesn't > seem to have any effect, initiated calls carry on for a short period, > but no new SIP requests are processed ('sip show channels' hangs > forever, server stops responding to SIP OPTIONS, or any other SIP > messages). We have updated the build from 1.8.23.1 to the latest > asterisk 1.8 (1.8.32.3), however the problem still persists. We have > gathered debugging information from 'core show locks' and from gdb, > attached to this message (with phone numbers and extension and context > names obscured). We are running realtime under CentOS 6.6, built from > source and packaged using rpmbuild, with the following menuselect > options (debugging version): > menuselect/menuselect --disable BUILD_NATIVE --enable DEBUG_THREADS > --enable DONT_OPTIMIZE --disable CORE-SOUNDS-EN-GSM --disable-category > MENUSELECT_EXTRA_SOUNDS --disable MOH-OPSOUND-WAV --enable-category > MENUSELECT_ADDONS --disable format_mp3 --disable cdr_tds --disable > cel_tds --disable cdr_pgsql --disable cel_pgsql --disable > res_config_pgsql menuselect.makeoptsAfter looking at the issue this appears to be a duplicate of an existing one[1] and a known issue. [1] https://issues.asterisk.org/jira/browse/ASTERISK-21228 -- Joshua Colp Digium, Inc. | Senior Software Developer 445 Jan Davis Drive NW - Huntsville, AL 35806 - US Check us out at: www.digium.com & www.asterisk.org