Tom Judge
2007-Apr-23 12:25 UTC
6.2-STABLE (i386) Repeating crash (supervisor read, page not present)
Hi, Recently I have noticed that one of our Dell PE1950's has been crashing a lot with the following reason "supervisor read, page not present". The system runs 6.2 Release under i386. I have attached 2 back traces, and I still have both cores if any more information is required. Any light that can be shed on this problem would be greatly appreciated. Tom ========== uname -a FreeBSD narthex.mintel.co.uk 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Mon Apr 2 20:13:11 BST 2007 root@bob.mintel.co.uk:/usr/obj/usr/src/sys/PE1950 i386 ## Core 1 root@narthex '13:14:47' '/home/london/tj' > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.1 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x100005c fault code = supervisor read, page not present instruction pointer = 0x20:0xc05df61f stack pointer = 0x28:0xe4f63c30 frame pointer = 0x28:0xe4f63c90 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi1: net) trap number = 12 panic: page fault cpuid = 0 Uptime: 1h25m33s Dumping 2047 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919 1903 1887 <7>arp_rtrequest: bad gateway 172.31.1.1 (!AF_LINK) <7>arp_rtrequest: bad gateway 172.31.0.1 (!AF_LINK) 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc05622ba in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc05625e1 in panic (fmt=0xc06e2578 "%s") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc06b4580 in trap_fatal (frame=0xe4f63bf0, eva=16777308) at /usr/src/sys/i386/i386/trap.c:837 #4 0xc06b42bf in trap_pfault (frame=0xe4f63bf0, usermode=0, eva=16777308) at /usr/src/sys/i386/i386/trap.c:745 #5 0xc06b3f19 in trap (frame {tf_fs = -1067581432, tf_es = -965803992, tf_ds = -964624344, tf_edi = -957112288, tf_esi = -965676032, tf_ebp = -453624688, tf_isp = -453624804, tf_ebx = 16777216, tf_edx = -968955648, tf_ecx = 4, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1067583969, tf_cs = 32, tf_eflags = 66118, tf_esp = 3, tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:435 #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc05df61f in in_arpinput (m=0xc68ba200) at /usr/src/sys/netinet/if_ether.c:636 #8 0xc05df4ea in arpintr (m=0xc68ba200) at /usr/src/sys/netinet/if_ether.c:551 #9 0xc05d861b in netisr_processqueue (ni=0xc076b078) at /usr/src/sys/net/netisr.c:236 #10 0xc05d881a in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:349 #11 0xc054cc49 in ithread_execute_handlers (p=0xc63ed860, ie=0xc643bb80) at /usr/src/sys/kern/kern_intr.c:682 #12 0xc054cd59 in ithread_loop (arg=0xc63bb870) at /usr/src/sys/kern/kern_intr.c:765 #13 0xc054b9fd in fork_exit (callout=0xc054cd04 <ithread_loop>, arg=0xc63bb870, frame=0xe4f63d38) at /usr/src/sys/kern/kern_fork.c:821 #14 0xc06a09bc in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:208 (kgdb) exit Undefined command: "exit". Try "help". (kgdb) quit ## Core 2 root@narthex '13:15:32' '/home/london/tj' > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.0 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0xb1 fault code = supervisor read, page not present instruction pointer = 0x20:0xc047ed16 stack pointer = 0x28:0xecd9c9e0 frame pointer = 0x28:0xecd9c9e0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 22409 (pfctl) trap number = 12 panic: page fault cpuid = 1 Uptime: 3d1h41m58s Dumping 2047 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc05622ba in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc05625e1 in panic (fmt=0xc06e2578 "%s") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc06b4580 in trap_fatal (frame=0xecd9c9a0, eva=177) at /usr/src/sys/i386/i386/trap.c:837 #4 0xc06b42bf in trap_pfault (frame=0xecd9c9a0, usermode=0, eva=177) at /usr/src/sys/i386/i386/trap.c:745 #5 0xc06b3f19 in trap (frame {tf_fs = 8, tf_es = -321322968, tf_ds = -1067909080, tf_edi = -965816192, tf_esi = -257, tf_ebp = -321271328, tf_isp = -321271348, tf_ebx = -957803852, tf_edx = 1, tf_ecx = -957803852, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1069028074, tf_cs = 32, tf_eflags = 66050, tf_esp = -321271304, tf_ss = -1069021108}) at /usr/src/sys/i386/i386/trap.c:435 #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc047ed16 in pfi_ifhead_RB_NEXT (elm=0x1) at /usr/src/sys/contrib/pf/net/pf_if.c:120 #8 0xc048084c in pfi_clear_flags (name=0xc66ed080 "", flags=-257) at /usr/src/sys/contrib/pf/net/pf_if.c:1004 #9 0xc0485629 in pfioctl (dev=0xc66d6100, cmd=3223602266, addr=0xc66ed080 "", flags=3, td=0x0) at /usr/src/sys/contrib/pf/net/pf_ioctl.c:3191 #10 0xc050fac3 in devfs_ioctl_f (fp=0xc6c72120, com=3223602266, data=0xc66ed080, cred=0xc6cab500, td=0xc6bcc900) at /usr/src/sys/fs/devfs/devfs_vnops.c:479 #11 0xc0586249 in ioctl (td=0xc6bcc900, uap=0xecd9cd04) at file.h:264 #12 0xc06b48c7 in syscall (frame {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = -1077944364, tf_esi = 2, tf_ebp = -1077944344, tf_isp = -321270428, tf_ebx = 0, tf_edx = 0, tf_ecx = 0, tf_eax = 54, tf_trapno = 12, tf_err = 2, tf_eip = 672674543, tf_cs = 51, tf_eflags = 582, tf_esp = -1077944420, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:983 #13 0xc06a09af in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) supervisor read, page not presentQuit (kgdb)
Brian A. Seklecki
2007-Apr-23 15:42 UTC
6.2-STABLE (i386) Repeating crash (supervisor read, page not present)
Are you running i386 on this platform to keep your network platform homogeneous. We run 6.2p2 on amd64 on the same hardware w/o issue. That's an interesting ARP issue; are you using bce(4)? We prefer to stack the machine with PCIe4x em(4) cards. ~BAS On Mon, 2007-04-23 at 13:24 +0100, Tom Judge wrote:> I have attached 2 back traces, and I still have both cores if any > more-- Brian A. Seklecki <bseklecki@collaborativefusion.com> Collaborative Fusion, Inc. IMPORTANT: This message contains confidential information and is intended only for the individual named. If the reader of this message is not an intended recipient (or the individual responsible for the delivery of this message to an intended recipient), please be advised that any re-use, dissemination, distribution or copying of this message is prohibited. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system.
Kris Kennaway
2007-Apr-23 15:57 UTC
6.2-STABLE (i386) Repeating crash (supervisor read, page not present)
On Mon, Apr 23, 2007 at 01:24:52PM +0100, Tom Judge wrote:> Hi, > > Recently I have noticed that one of our Dell PE1950's has been crashing > a lot with the following reason "supervisor read, page not present". > > The system runs 6.2 Release under i386. > > I have attached 2 back traces, and I still have both cores if any more > information is required. Any light that can be shed on this problem > would be greatly appreciated. > > Tom > > ==========> > uname -a > FreeBSD narthex.mintel.co.uk 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Mon > Apr 2 20:13:11 BST 2007 > root@bob.mintel.co.uk:/usr/obj/usr/src/sys/PE1950 i386 > > > ## Core 1 > > root@narthex '13:14:47' '/home/london/tj' > > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.1 > [GDB will not be able to debug user-mode threads: > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd". > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x100005c > fault code = supervisor read, page not present > instruction pointer = 0x20:0xc05df61f > stack pointer = 0x28:0xe4f63c30 > frame pointer = 0x28:0xe4f63c90 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 12 (swi1: net) > trap number = 12 > panic: page fault > cpuid = 0 > Uptime: 1h25m33s > Dumping 2047 MB (2 chunks) > chunk 0: 1MB (159 pages) ... ok > chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919 > 1903 1887 > <7>arp_rtrequest: bad gateway 172.31.1.1 (!AF_LINK) > <7>arp_rtrequest: bad gateway 172.31.0.1 (!AF_LINK)You might be hitting a bug in an obscure code path because of the above errors. I'm CC'ing someone who might be able to help. Kris> 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663 > 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439 > 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215 > 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975 > 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687 > 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399 > 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111 > 95 79 63 47 31 15 > > #0 doadump () at pcpu.h:165 > 165 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc05622ba in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc05625e1 in panic (fmt=0xc06e2578 "%s") at > /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc06b4580 in trap_fatal (frame=0xe4f63bf0, eva=16777308) at > /usr/src/sys/i386/i386/trap.c:837 > #4 0xc06b42bf in trap_pfault (frame=0xe4f63bf0, usermode=0, > eva=16777308) at /usr/src/sys/i386/i386/trap.c:745 > #5 0xc06b3f19 in trap (frame> {tf_fs = -1067581432, tf_es = -965803992, tf_ds = -964624344, > tf_edi = -957112288, tf_esi = -965676032, tf_ebp = -453624688, tf_isp = > -453624804, tf_ebx = 16777216, tf_edx = -968955648, tf_ecx = 4, tf_eax = > 0, tf_trapno = 12, tf_err = 0, tf_eip = -1067583969, tf_cs = 32, > tf_eflags = 66118, tf_esp = 3, tf_ss = 0}) at > /usr/src/sys/i386/i386/trap.c:435 > #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > #7 0xc05df61f in in_arpinput (m=0xc68ba200) at > /usr/src/sys/netinet/if_ether.c:636 > #8 0xc05df4ea in arpintr (m=0xc68ba200) at > /usr/src/sys/netinet/if_ether.c:551 > #9 0xc05d861b in netisr_processqueue (ni=0xc076b078) at > /usr/src/sys/net/netisr.c:236 > #10 0xc05d881a in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:349 > #11 0xc054cc49 in ithread_execute_handlers (p=0xc63ed860, ie=0xc643bb80) > at /usr/src/sys/kern/kern_intr.c:682 > #12 0xc054cd59 in ithread_loop (arg=0xc63bb870) at > /usr/src/sys/kern/kern_intr.c:765 > #13 0xc054b9fd in fork_exit (callout=0xc054cd04 <ithread_loop>, > arg=0xc63bb870, frame=0xe4f63d38) at /usr/src/sys/kern/kern_fork.c:821 > #14 0xc06a09bc in fork_trampoline () at > /usr/src/sys/i386/i386/exception.s:208 > (kgdb) exit > Undefined command: "exit". Try "help". > (kgdb) quit > > > ## Core 2 > root@narthex '13:15:32' '/home/london/tj' > > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.0 > [GDB will not be able to debug user-mode threads: > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd". > > Unread portion of the kernel message buffer: > > > Fatal trap 12: page fault while in kernel mode > cpuid = 1; apic id = 01 > fault virtual address = 0xb1 > fault code = supervisor read, page not present > instruction pointer = 0x20:0xc047ed16 > stack pointer = 0x28:0xecd9c9e0 > frame pointer = 0x28:0xecd9c9e0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 22409 (pfctl) > trap number = 12 > panic: page fault > cpuid = 1 > Uptime: 3d1h41m58s > Dumping 2047 MB (2 chunks) > chunk 0: 1MB (159 pages) ... ok > chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919 > 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 > 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 > 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 > 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 > 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 > 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 > 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 > 143 127 111 95 79 63 47 31 15 > > #0 doadump () at pcpu.h:165 > 165 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc05622ba in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 > #2 0xc05625e1 in panic (fmt=0xc06e2578 "%s") at > /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc06b4580 in trap_fatal (frame=0xecd9c9a0, eva=177) at > /usr/src/sys/i386/i386/trap.c:837 > #4 0xc06b42bf in trap_pfault (frame=0xecd9c9a0, usermode=0, eva=177) at > /usr/src/sys/i386/i386/trap.c:745 > #5 0xc06b3f19 in trap (frame> {tf_fs = 8, tf_es = -321322968, tf_ds = -1067909080, tf_edi = > -965816192, tf_esi = -257, tf_ebp = -321271328, tf_isp = -321271348, > tf_ebx = -957803852, tf_edx = 1, tf_ecx = -957803852, tf_eax = 0, > tf_trapno = 12, tf_err = 0, tf_eip = -1069028074, tf_cs = 32, tf_eflags > = 66050, tf_esp = -321271304, tf_ss = -1069021108}) at > /usr/src/sys/i386/i386/trap.c:435 > #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > #7 0xc047ed16 in pfi_ifhead_RB_NEXT (elm=0x1) at > /usr/src/sys/contrib/pf/net/pf_if.c:120 > #8 0xc048084c in pfi_clear_flags (name=0xc66ed080 "", flags=-257) at > /usr/src/sys/contrib/pf/net/pf_if.c:1004 > #9 0xc0485629 in pfioctl (dev=0xc66d6100, cmd=3223602266, > addr=0xc66ed080 "", flags=3, td=0x0) at > /usr/src/sys/contrib/pf/net/pf_ioctl.c:3191 > #10 0xc050fac3 in devfs_ioctl_f (fp=0xc6c72120, com=3223602266, > data=0xc66ed080, cred=0xc6cab500, td=0xc6bcc900) at > /usr/src/sys/fs/devfs/devfs_vnops.c:479 > #11 0xc0586249 in ioctl (td=0xc6bcc900, uap=0xecd9cd04) at file.h:264 > #12 0xc06b48c7 in syscall (frame> {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = -1077944364, tf_esi > = 2, tf_ebp = -1077944344, tf_isp = -321270428, tf_ebx = 0, tf_edx = 0, > tf_ecx = 0, tf_eax = 54, tf_trapno = 12, tf_err = 2, tf_eip = 672674543, > tf_cs = 51, tf_eflags = 582, tf_esp = -1077944420, tf_ss = 59}) > at /usr/src/sys/i386/i386/trap.c:983 > #13 0xc06a09af in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:200 > #14 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) supervisor read, page not presentQuit > (kgdb) > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20070423/da44f09f/attachment.pgp