Mark Johnston
2017-Jan-13 19:37 UTC
stable/11 debugging kernel unable to produce crashdump
On Sat, Jan 14, 2017 at 02:21:23AM +0700, Eugene Grosbein wrote:> Hi! > > I'm struggling to debug a panic in 11.0-STABLE/i386 that successfully produces crashdump > but I want more information. So I've rebuilt my custom kernel to include > options INVARIANTS, WITNESS and DEADLKRES. Now any panic results in quick unclean reboot > without crashdump generation. Serial console shows: > > Script started on Sat Jan 14 02:03:16 2017 > Command: cu -l cuau0 -s 115200 > Connected > > root at gw:~ # sysctl debug.kdb.panic=1 > debug.kdb.panic:panic: kdb_sysctl_panic > KDB: stack backtrace: > db_trace_self_wrapper(e8bc2ae8,0,e8bc2ae8,e8bc2a48,c06b4af4,...) at 0xc04c457b = db_trace_self_wrapper+0x2b/frame 0xe8bc2a20 > vpanic(c0a4916e,e8bc2a54,e8bc2a54,e8bc2a5c,c06e881f,...) at 0xc06b4a7f = vpanic+0x6f/frame 0xe8bc2a34 > panic(c0a4916e,1,c0ae7e48,e8bc2a88,c06bfdd3,...) at 0xc06b4af4 = panic+0x14/frame 0xe8bc2a48 > kdb_sysctl_panic(c0ae7e48,0,0,0,e8bc2ae8) at 0xc06e881f = kdb_sysctl_panic+0x4f/frame 0xe8bc2a5c > sysctl_root_handler_locked(0,0,e8bc2ae8,e8bc2aa8) at 0xc06bfdd3 = sysctl_root_handler_locked+0x83/frame 0xe8bc2a88 > sysctl_root(0,e8bc2ae8) at 0xc06bf744 = sysctl_root+0x144/frame 0xe8bc2ad8 > userland_sysctl(c759f9c0,e8bc2b60,3,0,0,0,bfbfdc5c,4,e8bc2bc0,0) at 0xc06bfb9d = userland_sysctl+0x12d/frame 0xe8bc2b30 > sys___sysctl(c759f9c0,e8bc2c00) at 0xc06bfa32 = sys___sysctl+0x52/frame 0xe8bc2bd0 > syscall(e8bc2ce8) at 0xc0980801 = syscall+0x2a1/frame 0xe8bc2cdc > Xint0x80_syscall() at 0xc096e45e = Xint0x80_syscall+0x2e/frame 0xe8bc2cdc > --- syscall (202, FreeBSD ELF32, sys___sysctl), eip = 0x2818541b, esp = 0xbfbfdbc8, ebp = 0xbfbfdbf0 --- > Uptime: 4m36s > panic: malloc: called with spinlock or critical section held > Uptime: 4m36s > panic: _mtx_lock_sleep: recursed on non-recursive mutex CAM device lock @ /home/src/sys/cam/ata/ata_da.c:3382I suspect that this is because we only stop the scheduler upon a panic if SMP is configured. Can you retest with the patch below applied? Index: sys/kern/kern_shutdown.c ==================================================================--- sys/kern/kern_shutdown.c (revision 312082) +++ sys/kern/kern_shutdown.c (working copy) @@ -713,6 +713,7 @@ CPU_CLR(PCPU_GET(cpuid), &other_cpus); stop_cpus_hard(other_cpus); } +#endif /* * Ensure that the scheduler is stopped while panicking, even if panic @@ -719,7 +720,6 @@ * has been entered from kdb. */ td->td_stopsched = 1; -#endif bootopt = RB_AUTOBOOT; newpanic = 0;
Eugene Grosbein
2017-Jan-14 11:40 UTC
stable/11 debugging kernel unable to produce crashdump
> I suspect that this is because we only stop the scheduler upon a panic > if SMP is configured. Can you retest with the patch below applied? > > Index: sys/kern/kern_shutdown.c > ==================================================================> --- sys/kern/kern_shutdown.c (revision 312082) > +++ sys/kern/kern_shutdown.c (working copy) > @@ -713,6 +713,7 @@ > CPU_CLR(PCPU_GET(cpuid), &other_cpus); > stop_cpus_hard(other_cpus); > } > +#endif > > /* > * Ensure that the scheduler is stopped while panicking, even if panic > @@ -719,7 +720,6 @@ > * has been entered from kdb. > */ > td->td_stopsched = 1; > -#endif > > bootopt = RB_AUTOBOOT; > newpanic = 0; > >Indeed, my router is uniprocessor system and your patch really solves the problem. Now kernel generates crashdump just fine in case of panic. Please commit the fix, thanks! Eugene Grosbein