I recently upgraded asterisk13 from 13.17.0_1 to 13.17.1, and also
upgraded freebsd from 11.0 to 11.1. since then asterisk starts crashing
every few minutes.
I ran /usr/local/share/asterisk/scripts/ast_coredumper /tmp/asterisk.core
and got the following result in asterisk.core-full.txt
Thread 6 (LWP 101423):
#0? 0x000000080335558a in _poll () from /lib/libc.so.7
No symbol table info available.
#1? 0x000000080303e706 in ?? () from /lib/libthr.so.3
No symbol table info available.
#2? 0x00000000004506b2 in ?? ()
No symbol table info available.
#3? 0x00000000005921ea in ?? ()
No symbol table info available.
#4? 0x000000080303bbc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#5? 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdfe0a000
Thread 5 (LWP 101336):
#0? 0x0000000803049c7c in ?? () from /lib/libthr.so.3
No symbol table info available.
#1? 0x0000000803046325 in ?? () from /lib/libthr.so.3
No symbol table info available.
#2? 0x0000000000589d18 in ?? ()
No symbol table info available.
#3? 0x00000000004552f1 in ?? ()
No symbol table info available.
#4? 0x00000000004552f1 in ?? ()
No symbol table info available.
#5? 0x0000000000589c4b in ?? ()
No symbol table info available.
#6? 0x000000000058318e in ast_taskprocessor_execute ()
No symbol table info available.
#7? 0x00000000005832de in ?? ()
No symbol table info available.
#8? 0x00000000005921ea in ?? ()
No symbol table info available.
#9? 0x000000080303bbc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#10 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdff04000
Thread 4 (LWP 101240):
#0? 0x00000008032d88b8 in _umtx_op () from /lib/libc.so.7
No symbol table info available.
#1? 0x00000008032c275d in sem_clockwait_np () from /lib/libc.so.7
No symbol table info available.
#2? 0x00000000005832c8 in ?? ()
No symbol table info available.
#3? 0x00000000005921ea in ?? ()
No symbol table info available.
#4? 0x000000080303bbc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#5? 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdff81000
Thread 3 (LWP 101183):
#0? 0x0000000803049c7c in ?? () from /lib/libthr.so.3
No symbol table info available.
#1? 0x000000080303dba0 in ?? () from /lib/libthr.so.3
No symbol table info available.
#2? 0x00000008030479f8 in ?? () from /lib/libthr.so.3
No symbol table info available.
#3? 0x00000000004c9756 in ?? ()
No symbol table info available.
#4? 0x00000000005921ea in ?? ()
No symbol table info available.
#5? 0x000000080303bbc5 in ?? () from /lib/libthr.so.3
No symbol table info available.
#6? 0x0000000000000000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdfffe000
Thread 2 (LWP 100441):
#0? 0x000000080335558a in _poll () from /lib/libc.so.7
No symbol table info available.
#1? 0x000000080303e706 in ?? () from /lib/libthr.so.3
No symbol table info available.
#2? 0x0000000000452980 in ?? ()
No symbol table info available.
#3? 0x0000000000436483 in ?? ()
No symbol table info available.
#4? 0x0000000000437b1f in ?? ()
No symbol table info available.
#5? 0x0000000800876000 in ?? ()
No symbol table info available.
#6? 0x0000000000000000 in ?? ()
No symbol table info available.
Thread 1 (LWP 100493):
#0? x86_64_freebsd_fallback_frame_state (context=0x7fffdfd0fe20,
context=0x7fffdfd0fe20, fs=0x7fffdfd0fb70) at ./md-unwind-support.h:60
??????? sf = <optimized out>
??????? new_cfa = <optimized out>
#1? uw_frame_state_for (context=context at entry=0x7fffdfd0fe20,
fs=fs at entry=0x7fffdfd0fb70) at
/wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind-dw2.c:1249
??????? fde = 0x0
??????? cie = <optimized out>
??????? aug = <optimized out>
??????? insn = <optimized out>
??????? end = <optimized out>
#2? 0x0000000802e2cffb in _Unwind_ForcedUnwind_Phase2
(exc=exc at entry=0x80701dc30, context=context at entry=0x7fffdfd0fe20) at
/wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:155
??????? fs = {regs = {reg = {{loc = {reg = 0, offset = 0, exp = 0x0},
how = REG_UNSAVED} <repeats 18 times>}, prev = 0x0, cfa_offset = 0,
cfa_reg = 0, cfa_exp = 0x0, cfa_how = CFA_UNSET}, pc = 0x0, personality
= 0x0, data_align = 0, code_align = 0, retaddr_column = 0, fde_encoding
= 0 '000', lsda_encoding = 0 '000', saw_z = 0 '000',
signal_frame = 0
'000', eh_ptr = 0x0}
??????? action = <optimized out>
??????? stop = 0x8030497b0
??????? stop_argument = 0x0
??????? code = <optimized out>
??????? stop_code = <optimized out>
#3? 0x0000000802e2d334 in _Unwind_ForcedUnwind (exc=0x80701dc30,
stop=0x8030497b0, stop_argument=<optimized out>) at
/wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:207
??????? this_context = {reg = {0x7fffdfd0ff18, 0x7fffdfd0ff20, 0x0,
0x7fffdfd0ff28, 0x0, 0x0, 0x7fffdfd0ff50, 0x0, 0x0, 0x0, 0x0, 0x0,
0x7fffdfd0ff30, 0x7fffdfd0ff38, 0x7fffdfd0ff40, 0x7fffdfd0ff48,
0x7fffdfd0ff58, 0x0}, cfa = 0x7fffdfd0ff60, ra = 0x803049613, lsda =
0x0, bases = {tbase = 0x0, dbase = 0x0, func = 0x802e2d2d0
<_Unwind_ForcedUnwind>}, flags = 4611686018427387904, version = 0,
args_size = 0, by_value = '000' <repeats 17 times>}
??????? cur_context = {reg = {0x7fffdfd0ff18, 0x7fffdfd0ff20, 0x0,
0x7fffdfd0ffd8, 0x0, 0x0, 0x7fffdfd0fff0, 0x0, 0x0, 0x0, 0x0, 0x0,
0x7fffdfd0ff30, 0x7fffdfd0ff38, 0x7fffdfd0ffe0, 0x7fffdfd0ffe8,
0x7fffdfd0fff8, 0x0}, cfa = 0x7fffdfd10000, ra = 0x7fffdfc94000, lsda =
0x0, bases = {tbase = 0x0, dbase = 0x0, func = 0x80303ba80}, flags =
4611686018427387904, version = 0, args_size = 0, by_value = '000'
<repeats 17 times>}
??????? code = <optimized out>
#4? 0x0000000803049613 in ?? () from /lib/libthr.so.3
No symbol table info available.
#5? 0x000000080304942b in pthread_exit () from /lib/libthr.so.3
No symbol table info available.
#6? 0x000000080303bbcd in ?? () from /lib/libthr.so.3
No symbol table info available.
#7? 0x00007fffdfc94000 in ?? ()
No symbol table info available.
Backtrace stopped: Cannot access memory at address 0x7fffdfd10000
I tried to run without loading any asterisk modules, and it still crashes.
I also tried copying all the dependent libs from another machine running
freebsd 11.0, put them in a separate directory and use chrpath to force
asterisk use those libs, and also got the same error.
Any ideas?
Thanks
--
Tao
Hans Petter Selasky
2017-Sep-18 07:40 UTC
[Asterisk-bsd] Asterisk13 coredump on freebsd 11.1
On 09/18/17 09:39, Tao Zhou wrote:> I recently upgraded asterisk13 from 13.17.0_1 to 13.17.1, and also > upgraded freebsd from 11.0 to 11.1. since then asterisk starts crashing > every few minutes. > > I ran /usr/local/share/asterisk/scripts/ast_coredumper /tmp/asterisk.core > > and got the following result in asterisk.core-full.txt > > > Thread 6 (LWP 101423): > #0 0x000000080335558a in _poll () from /lib/libc.so.7 > No symbol table info available. > #1 0x000000080303e706 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #2 0x00000000004506b2 in ?? () > No symbol table info available. > #3 0x00000000005921ea in ?? () > No symbol table info available. > #4 0x000000080303bbc5 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #5 0x0000000000000000 in ?? () > No symbol table info available. > Backtrace stopped: Cannot access memory at address 0x7fffdfe0a000 > > Thread 5 (LWP 101336): > #0 0x0000000803049c7c in ?? () from /lib/libthr.so.3 > No symbol table info available. > #1 0x0000000803046325 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #2 0x0000000000589d18 in ?? () > No symbol table info available. > #3 0x00000000004552f1 in ?? () > No symbol table info available. > #4 0x00000000004552f1 in ?? () > No symbol table info available. > #5 0x0000000000589c4b in ?? () > No symbol table info available. > #6 0x000000000058318e in ast_taskprocessor_execute () > No symbol table info available. > #7 0x00000000005832de in ?? () > No symbol table info available. > #8 0x00000000005921ea in ?? () > No symbol table info available. > #9 0x000000080303bbc5 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #10 0x0000000000000000 in ?? () > No symbol table info available. > Backtrace stopped: Cannot access memory at address 0x7fffdff04000 > > Thread 4 (LWP 101240): > #0 0x00000008032d88b8 in _umtx_op () from /lib/libc.so.7 > No symbol table info available. > #1 0x00000008032c275d in sem_clockwait_np () from /lib/libc.so.7 > No symbol table info available. > #2 0x00000000005832c8 in ?? () > No symbol table info available. > #3 0x00000000005921ea in ?? () > No symbol table info available. > #4 0x000000080303bbc5 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #5 0x0000000000000000 in ?? () > No symbol table info available. > Backtrace stopped: Cannot access memory at address 0x7fffdff81000 > > Thread 3 (LWP 101183): > #0 0x0000000803049c7c in ?? () from /lib/libthr.so.3 > No symbol table info available. > #1 0x000000080303dba0 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #2 0x00000008030479f8 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #3 0x00000000004c9756 in ?? () > No symbol table info available. > #4 0x00000000005921ea in ?? () > No symbol table info available. > #5 0x000000080303bbc5 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #6 0x0000000000000000 in ?? () > No symbol table info available. > Backtrace stopped: Cannot access memory at address 0x7fffdfffe000 > > Thread 2 (LWP 100441): > #0 0x000000080335558a in _poll () from /lib/libc.so.7 > No symbol table info available. > #1 0x000000080303e706 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #2 0x0000000000452980 in ?? () > No symbol table info available. > #3 0x0000000000436483 in ?? () > No symbol table info available. > #4 0x0000000000437b1f in ?? () > No symbol table info available. > #5 0x0000000800876000 in ?? () > No symbol table info available. > #6 0x0000000000000000 in ?? () > No symbol table info available. > > Thread 1 (LWP 100493): > #0 x86_64_freebsd_fallback_frame_state (context=0x7fffdfd0fe20, > context=0x7fffdfd0fe20, fs=0x7fffdfd0fb70) at ./md-unwind-support.h:60 > sf = <optimized out> > new_cfa = <optimized out> > #1 uw_frame_state_for (context=context at entry=0x7fffdfd0fe20, > fs=fs at entry=0x7fffdfd0fb70) at > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind-dw2.c:1249 > fde = 0x0 > cie = <optimized out> > aug = <optimized out> > insn = <optimized out> > end = <optimized out> > #2 0x0000000802e2cffb in _Unwind_ForcedUnwind_Phase2 > (exc=exc at entry=0x80701dc30, context=context at entry=0x7fffdfd0fe20) at > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:155 > fs = {regs = {reg = {{loc = {reg = 0, offset = 0, exp = 0x0}, > how = REG_UNSAVED} <repeats 18 times>}, prev = 0x0, cfa_offset = 0, > cfa_reg = 0, cfa_exp = 0x0, cfa_how = CFA_UNSET}, pc = 0x0, personality > = 0x0, data_align = 0, code_align = 0, retaddr_column = 0, fde_encoding > = 0 '000', lsda_encoding = 0 '000', saw_z = 0 '000', signal_frame = 0 > '000', eh_ptr = 0x0} > action = <optimized out> > stop = 0x8030497b0 > stop_argument = 0x0 > code = <optimized out> > stop_code = <optimized out> > #3 0x0000000802e2d334 in _Unwind_ForcedUnwind (exc=0x80701dc30, > stop=0x8030497b0, stop_argument=<optimized out>) at > /wrkdirs/usr/ports/lang/gcc6/work/gcc-6.4.0/libgcc/unwind.inc:207 > this_context = {reg = {0x7fffdfd0ff18, 0x7fffdfd0ff20, 0x0, > 0x7fffdfd0ff28, 0x0, 0x0, 0x7fffdfd0ff50, 0x0, 0x0, 0x0, 0x0, 0x0, > 0x7fffdfd0ff30, 0x7fffdfd0ff38, 0x7fffdfd0ff40, 0x7fffdfd0ff48, > 0x7fffdfd0ff58, 0x0}, cfa = 0x7fffdfd0ff60, ra = 0x803049613, lsda = > 0x0, bases = {tbase = 0x0, dbase = 0x0, func = 0x802e2d2d0 > <_Unwind_ForcedUnwind>}, flags = 4611686018427387904, version = 0, > args_size = 0, by_value = '000' <repeats 17 times>} > cur_context = {reg = {0x7fffdfd0ff18, 0x7fffdfd0ff20, 0x0, > 0x7fffdfd0ffd8, 0x0, 0x0, 0x7fffdfd0fff0, 0x0, 0x0, 0x0, 0x0, 0x0, > 0x7fffdfd0ff30, 0x7fffdfd0ff38, 0x7fffdfd0ffe0, 0x7fffdfd0ffe8, > 0x7fffdfd0fff8, 0x0}, cfa = 0x7fffdfd10000, ra = 0x7fffdfc94000, lsda = > 0x0, bases = {tbase = 0x0, dbase = 0x0, func = 0x80303ba80}, flags = > 4611686018427387904, version = 0, args_size = 0, by_value = '000' > <repeats 17 times>} > code = <optimized out> > #4 0x0000000803049613 in ?? () from /lib/libthr.so.3 > No symbol table info available. > #5 0x000000080304942b in pthread_exit () from /lib/libthr.so.3 > No symbol table info available. > #6 0x000000080303bbcd in ?? () from /lib/libthr.so.3 > No symbol table info available. > #7 0x00007fffdfc94000 in ?? () > No symbol table info available. > Backtrace stopped: Cannot access memory at address 0x7fffdfd10000 > > > I tried to run without loading any asterisk modules, and it still crashes. > > I also tried copying all the dependent libs from another machine running > freebsd 11.0, put them in a separate directory and use chrpath to force > asterisk use those libs, and also got the same error. > > > Any ideas? >Hi, There is a known issue with the latest version of Asterisk 13.xxx crashing. I don't know the root cause. Try downgrading the Asterisk version. You probably should compile all code with debug flags enabled if you want to find the root cause of this. --HPS
On 09/18/2017 09:39, Tao Zhou wrote:> I recently upgraded asterisk13 from 13.17.0_1 to 13.17.1, and also > upgraded freebsd from 11.0 to 11.1. since then asterisk starts crashing > every few minutes. >Hi, Are you using a locally build asterisk port or the package provided by the FreeBSD cluster? In both cases, have you tried reinstalling (or rebuilding in case you're using ports) all the ports/packages asterisk depends on?> I ran /usr/local/share/asterisk/scripts/ast_coredumper /tmp/asterisk.core > > and got the following result in asterisk.core-full.txt > > > Thread 6 (LWP 101423):[...]> Backtrace stopped: Cannot access memory at address 0x7fffdfd10000 >Unluckily the core dump to be of any use should be created using a debug build of asterisk, you can try set "WITH_DEBUG" with the port, also check that no optimization is passed to the compiler when building the debug version.> > I tried to run without loading any asterisk modules, and it still crashes. > > I also tried copying all the dependent libs from another machine running > freebsd 11.0, put them in a separate directory and use chrpath to force > asterisk use those libs, and also got the same error.Better rebuild things from scratch on the same machine.> > > Any ideas?Have you tried using Asterisk 13.17.0_1 on FreeBSD 11.1? and 13.17.1 on 11.0? Just to rule out one variable. -- Guido Falsi <mad at madpilot.net>