Hello, FreeBSD-stable. Last week I have a lot of kernel panics like:> [root@router1 /usr/obj/usr/src/sys/TINCOKERNEL2]# kgdb kernel.debug /var/crash/vmcore.20 > [GDB will not be able to debug user-mode threads: > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd". > > Unread portion of the kernel message buffer: > kernel trap 12 with interrupts disabled > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x9 > fault code = supervisor read, page not present > instruction pointer = 0x20:0xc079f1af > stack pointer = 0x28:0xe5697c80 > frame pointer = 0x28:0xe5697cbc > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = resume, IOPL = 0 > current process = 14 (swi4: clock) > trap number = 12 > panic: page fault > cpuid = 0 > Uptime: 51m49s > Physical memory: 2032 MB > Dumping 177 MB: 162 146 130 114 98 82 66 50 34 18 2 > > #0 doadump () at pcpu.h:195 > 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); > (kgdb) bt > #0 doadump () at pcpu.h:195 > #1 0xc078d1b7 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc078d479 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:563 > #3 0xc0a0eaac in trap_fatal (frame=0xe5697c40, eva=9) at /usr/src/sys/i386/i386/trap.c:899 > #4 0xc0a0f42f in trap (frame=0xe5697c40) at /usr/src/sys/i386/i386/trap.c:280 > #5 0xc09f565b in calltrap () at > /usr/src/sys/i386/i386/exception.s:139 > #6 0xc079f1af in softclock (dummy=0x0) at > /usr/src/sys/kern/kern_timeout.c:202 > #7 0xc076f31b in ithread_loop (arg=0xc5101250) at > /usr/src/sys/kern/kern_intr.c:1036 > #8 0xc076c119 in fork_exit (callout=0xc076f170 <ithread_loop>, arg=0xc5101250, frame=0xe5697d38) > at /usr/src/sys/kern/kern_fork.c:781 > #9 0xc09f56d0 in fork_trampoline () at > /usr/src/sys/i386/i386/exception.s:205 > (kgdb)What I should do? -- ? ?????????, L1NYX01D mailto:l1nyx01d@googlemail.com
On Mon, 2009-01-12 at 21:27 +0300, L1NYX01D@GOOGLEMAIL.COM wrote:> Hello, FreeBSD-stable. > > Last week I have a lot of kernel panics like:Firstly: are these new panics on a system that has been reliable until now, or has something changed on the system recently (hardware change, different software in use, increased load, etc)? Is there any way you can reliably reproduce this? And is the panic message and backtrace always the same?> > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0x9 > > fault code = supervisor read, page not present > > instruction pointer = 0x20:0xc079f1af > > stack pointer = 0x28:0xe5697c80 > > frame pointer = 0x28:0xe5697cbc > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, def32 1, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 14 (swi4: clock) > > trap number = 12 > > panic: page fault > > cpuid = 0 > > Uptime: 51m49s > > Physical memory: 2032 MB > > Dumping 177 MB: 162 146 130 114 98 82 66 50 34 18 2 > > > > #0 doadump () at pcpu.h:195 > > 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); > > (kgdb) bt > > #0 doadump () at pcpu.h:195 > > #1 0xc078d1b7 in boot (howto=260) at > > /usr/src/sys/kern/kern_shutdown.c:409 > > #2 0xc078d479 in panic (fmt=Variable "fmt" is not available. > > ) at /usr/src/sys/kern/kern_shutdown.c:563 > > #3 0xc0a0eaac in trap_fatal (frame=0xe5697c40, eva=9) at /usr/src/sys/i386/i386/trap.c:899 > > #4 0xc0a0f42f in trap (frame=0xe5697c40) at /usr/src/sys/i386/i386/trap.c:280 > > #5 0xc09f565b in calltrap () at > > /usr/src/sys/i386/i386/exception.s:139 > > #6 0xc079f1af in softclock (dummy=0x0) at > > /usr/src/sys/kern/kern_timeout.c:202 > > #7 0xc076f31b in ithread_loop (arg=0xc5101250) at > > /usr/src/sys/kern/kern_intr.c:1036 > > #8 0xc076c119 in fork_exit (callout=0xc076f170 <ithread_loop>, arg=0xc5101250, frame=0xe5697d38) > > at /usr/src/sys/kern/kern_fork.c:781 > > #9 0xc09f56d0 in fork_trampoline () at > > /usr/src/sys/i386/i386/exception.s:205 > > (kgdb) > > What I should do?In kgdb, enter "frame 6" then "list". The output should show where in the code this occurred. You can then print any variables that look interesting by: p c p c->ctime Gavin
L1NYX01D@GOOGLEMAIL.COM wrote:> Hello, FreeBSD-stable. > > Last week I have a lot of kernel panics like: > >> [root@router1 /usr/obj/usr/src/sys/TINCOKERNEL2]# kgdb kernel.debug /var/crash/vmcore.20 >> [GDB will not be able to debug user-mode threads: >> /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you are >> welcome to change it and/or distribute copies of it under certain conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for details. >> This GDB was configured as "i386-marcel-freebsd". >> >> Unread portion of the kernel message buffer: >> kernel trap 12 with interrupts disabled >> >> >> Fatal trap 12: page fault while in kernel mode >> cpuid = 0; apic id = 00 >> fault virtual address = 0x9 >> fault code = supervisor read, page not present >> instruction pointer = 0x20:0xc079f1af >> stack pointer = 0x28:0xe5697c80 >> frame pointer = 0x28:0xe5697cbc >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, def32 1, gran 1 >> processor eflags = resume, IOPL = 0 >> current process = 14 (swi4: clock) >> trap number = 12 >> panic: page fault >> cpuid = 0 >> Uptime: 51m49s >> Physical memory: 2032 MB >> Dumping 177 MB: 162 146 130 114 98 82 66 50 34 18 2 >> >> #0 doadump () at pcpu.h:195 >> 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); >> (kgdb) bt >> #0 doadump () at pcpu.h:195 >> #1 0xc078d1b7 in boot (howto=260) at >> /usr/src/sys/kern/kern_shutdown.c:409 >> #2 0xc078d479 in panic (fmt=Variable "fmt" is not available. >> ) at /usr/src/sys/kern/kern_shutdown.c:563 >> #3 0xc0a0eaac in trap_fatal (frame=0xe5697c40, eva=9) at /usr/src/sys/i386/i386/trap.c:899 >> #4 0xc0a0f42f in trap (frame=0xe5697c40) at /usr/src/sys/i386/i386/trap.c:280 >> #5 0xc09f565b in calltrap () at >> /usr/src/sys/i386/i386/exception.s:139 >> #6 0xc079f1af in softclock (dummy=0x0) at >> /usr/src/sys/kern/kern_timeout.c:202 >> #7 0xc076f31b in ithread_loop (arg=0xc5101250) at >> /usr/src/sys/kern/kern_intr.c:1036 >> #8 0xc076c119 in fork_exit (callout=0xc076f170 <ithread_loop>, arg=0xc5101250, frame=0xe5697d38) >> at /usr/src/sys/kern/kern_fork.c:781 >> #9 0xc09f56d0 in fork_trampoline () at >> /usr/src/sys/i386/i386/exception.s:205 >> (kgdb)Looks likely to be a hardware fault, to me. Kris