Martin Sugioarto
2009-May-14 12:06 UTC
kernel trap 12 with interrupts disabled [bge0 on 7.2R]
Hi,
I've received a panic today on RELEASE 7.2 with bge(4). We have got
an apache 2.2 running that mounts an NFS share from a file server.
We have put some load on it, because we
have downloaded big files (700MB) for installation on two
workstations, about 15 of files were downloaded at the same time.
After about 20 minutes we received a panic output 2 times. I wrote it
down on paper. I could not access the debugger, because the output of
the panic stopped almost at the end. I've got only an USB keyboard that
would not help in this situation. It wasn't even plugged in.
Btw, promiscuous mode is enabled, because ipcad is running to count
traffic. I've got this problem the second time now.
The panic looks like this:
kernel trap 12 with interrupts disabled
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 0
fault virtual address = 0x80000000000
fault code = supervisor write data, page not present
instruction pointer = 0x8:0xffffffff80186249
stack pointer = 0x10:0xffffffff8065f200
frame pointer = 0x10:0x36ee7f
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = resume, IOPL = 0
current process = 26 (irq256: bge0)
trap number = 12
p[*CURSOR STOPPED HERE*]
dmesg:
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights
reserved. FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.2-RELEASE #0: Wed May 6 10:18:03 CEST 2009
root@inky:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(R) CPU X3350 @ 2.66GHz (2666.63-MHz
K8-class CPU) Origin = "GenuineIntel" Id = 0x10677 Stepping = 7
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x8e3fd<SSE3,RSVD2,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,<b19>>
AMD Features=0x20100800<SYSCALL,NX,LM>
AMD Features2=0x1<LAHF>
Cores per package: 4
usable memory = 8576458752 (8179 MB)
avail memory = 8290664448 (7906 MB)
ACPI APIC Table: <022108 APIC2247>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP): APIC ID: 3
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <022108 RSDT2247> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
acpi0: reservation of 100000, eff00000 (3) failed
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci5: <ACPI PCI bus> on pcib1
3ware device driver for 9000 series storage controllers, version:
3.70.05.001 twa0: <3ware 9000 series Storage Controller> port
0xe800-0xe8ff mem 0xfc000000-0xfdffffff,0xfebff000-0xfebfffff irq 16 at
device 0.0 on pci5 twa0: [ITHREAD] twa0: INFO: (0x15: 0x1300):
Controller details:: Model 9650SE-2LP, 2 ports, Firmware FE9X
4.06.00.004, BIOS BE9X 4.05.00.015 pcib2: <ACPI PCI-PCI bridge> irq 16
at device 28.0 on pci0 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI
PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib3
bge0: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x4201> mem 0xfe9f0000-0xfe9fffff irq 16 at device 0.0 on pci3 miibus0:
0x4201> <MII bus> on bge0
brgphy0: <BCM5750 10/100/1000baseTX PHY> PHY 1 on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto bge0: Ethernet address: xx:xx:xx:xx:xx:xx
bge0: [ITHREAD]
pcib4: <ACPI PCI-PCI bridge> irq 17 at device 28.5 on pci0
pci4: <ACPI PCI bus> on pcib4
bge1: <Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev.
0x4201> mem 0xfeaf0000-0xfeafffff irq 17 at device 0.0 on pci4 miibus1:
0x4201> <MII bus> on bge1
brgphy1: <BCM5750 10/100/1000baseTX PHY> PHY 1 on miibus1
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto bge1: Ethernet address: yy:yy:yy:yy:yy:yy
bge1: [ITHREAD]
uhci0: <UHCI (generic) USB controller> port 0xc080-0xc09f irq 23 at
device 29.0 on pci0 uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <UHCI (generic) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <UHCI (generic) USB controller> port 0xc000-0xc01f irq 19 at
device 29.1 on pci0 uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <UHCI (generic) USB controller> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem
0xfe7ff800-0xfe7ffbff irq 23 at device 29.7 on pci0 ehci0:
[GIANT-LOCKED] ehci0: [ITHREAD]
usb2: EHCI version 1.0
usb2: companion controllers, 2 ports each: usb0 usb1
usb2: <Intel 82801GB/R (ICH7) USB 2.0 controller> on ehci0
usb2: USB revision 2.0
uhub2: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb2
uhub2: 4 ports with 4 removable, self powered
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib5
vgapci0: <VGA-compatible display> port 0xdc00-0xdc7f mem
0xf8000000-0xfbffffff,0xfe8c0000-0xfe8fffff at device 4.0 on pci1
isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on
isab0
atapci0: <Intel ICH7 UDMA100 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 31.1 on
pci0 ata0: <ATA channel 0> on atapci0 ata0: [ITHREAD]
atapci1: <Intel ICH7 SATA300 controller> port
0xcc00-0xcc07,0xc880-0xc883,0xc800-0xc807,0xc480-0xc483,0xc400-0xc40f
mem 0xfe7ffc00-0xfe7fffff irq 19 at device 31.2 on pci0 atapci1:
[ITHREAD] ata2: <ATA channel 0> on atapci1 ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
acpi_button0: <Power Button> on acpi0
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: configured irq 4 not in bitmap of probed irqs 0
sio0: port may not be enabled
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0 sio0: type 16550A
sio0: [FILTER]
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
sio1: [FILTER]
cpu0: <ACPI CPU> on acpi0
ACPI Warning (tbutils-0243): Incorrect checksum in table [OEMB] - 77,
should be 74 [20070320] est0: <Enhanced SpeedStep Frequency Control> on
cpu0 p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
est2: <Enhanced SpeedStep Frequency Control> on cpu2
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
est3: <Enhanced SpeedStep Frequency Control> on cpu3
p4tcc3: <CPU Frequency Thermal Control> on cpu3
orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xc9fff on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: cannot reserve I/O port range
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on
isa0 ukbd0: <vendor 0x046a product 0x0023, class 0/0, rev 2.00/0.32,
addr 2> on uhub0 kbd2 at ukbd0
uhid0: <vendor 0x046a product 0x0023, class 0/0, rev 2.00/0.32, addr 2>
on uhub0 Timecounters tick every 1.000 msec
acd0: DVDR <ASUS DRW-1612BL/1.06> at ata0-slave UDMA66
da0 at twa0 bus 0 target 0 lun 0
da0: <AMCC 9650SE-2LP DISK 4.06> Fixed Direct Access SCSI-5 device
da0: 100.000MB/s transfers
da0: 476827MB (976541696 512 byte sectors: 255H 63S/T 60786C)
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
GEOM_LABEL: Label for provider da0p2 is ufsid/4933dfd79a3e27cc.
GEOM_LABEL: Label for provider da0p4 is ufsid/4933dfe53ca04410.
GEOM_LABEL: Label for provider da0p5 is ufsid/4933dfedbb4398a4.
GEOM_JOURNAL: Journal 326427402: da0p6 contains data.
GEOM_JOURNAL: Journal 326427402: da0p6 contains journal.
GEOM_JOURNAL: Journal da0p6 clean.
GEOM_LABEL: Label for provider da0p6.journal is ufsid/4933e04607a73efa.
Trying to mount root from ufs:/dev/da0p2
GEOM_LABEL: Label ufsid/4933dfd79a3e27cc removed.
GEOM_LABEL: Label for provider da0p2 is ufsid/4933dfd79a3e27cc.
GEOM_LABEL: Label ufsid/4933dfe53ca04410 removed.
GEOM_LABEL: Label for provider da0p4 is ufsid/4933dfe53ca04410.
GEOM_LABEL: Label ufsid/4933dfedbb4398a4 removed.
GEOM_LABEL: Label for provider da0p5 is ufsid/4933dfedbb4398a4.
GEOM_LABEL: Label ufsid/4933dfd79a3e27cc removed.
GEOM_LABEL: Label ufsid/4933dfe53ca04410 removed.
GEOM_LABEL: Label ufsid/4933dfedbb4398a4 removed.
GEOM_LABEL: Label ufsid/4933e04607a73efa removed.
bge0: promiscuous mode enabled
--
Martin
__________________________________________________________________________
Verschicken Sie SMS direkt vom Postfach aus - in alle deutschen und viele
ausl?ndische Netze zum gleichen Preis!
https://produkte.web.de/webde_sms/sms
On Thursday 14 May 2009 7:47:23 am Martin Sugioarto wrote:> Hi, > > I've received a panic today on RELEASE 7.2 with bge(4). We have got > an apache 2.2 running that mounts an NFS share from a file server. > We have put some load on it, because we > have downloaded big files (700MB) for installation on two > workstations, about 15 of files were downloaded at the same time. > > After about 20 minutes we received a panic output 2 times. I wrote it > down on paper. I could not access the debugger, because the output of > the panic stopped almost at the end. I've got only an USB keyboard that > would not help in this situation. It wasn't even plugged in. > > Btw, promiscuous mode is enabled, because ipcad is running to count > traffic. I've got this problem the second time now. > > > The panic looks like this: > > kernel trap 12 with interrupts disabled > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 0 > fault virtual address = 0x80000000000Given that that is a single bit set, it could possibly be due to bad RAM. Does your kernel have debug symbols? If so, running 'l *0xffffffff80186249' (from the 'instruction pointer' line in the fault message) would be helpful.> fault code = supervisor write data, page not present > instruction pointer = 0x8:0xffffffff80186249 > stack pointer = 0x10:0xffffffff8065f200 > frame pointer = 0x10:0x36ee7f > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = resume, IOPL = 0 > current process = 26 (irq256: bge0) > trap number = 12 > p[*CURSOR STOPPED HERE*]-- John Baldwin
Am Thu, 14 May 2009 09:16:40 -0400 schrieb John Baldwin <jhb@freebsd.org>:> On Thursday 14 May 2009 7:47:23 am Martin Sugioarto wrote: > [...] > > kernel trap 12 with interrupts disabled > > > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 0 > > fault virtual address = 0x80000000000 > > Given that that is a single bit set, it could possibly be due to bad > RAM.This is the second panic output that appeared on the screen. I could not read the first lines of the first panic. The last ones looked similar (same trap/process etc).> Does your kernel have debug symbols?This is GENERIC kernel configuration. The kernel was totally frozen. I could not type anything. I just noticed, I've got a vmcore.0 of the crash. I can see some other panic output when loading the kernel in kgdb: Unread portion of the kernel message buffer: Fatal trap 9: general protection fault while in kernel mode cpuid = 2; apic id = 02 instruction pointer = 0x8:0xffffffff805bbc66 stack pointer = 0x10:0xffffffff51e2e410 frame pointer = 0x10:0xffffffff51e2e4c0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 1311 (nfsiod 0) trap number = 9 panic: general protection fault cpuid = 2 Uptime: 1h5m39s Physical memory: 8179 MB Dumping 479 MB: 464 448 432 416 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16 Reading symbols from /boot/kernel/geom_journal.ko...Reading symbols from /boot/kernel/geom_journal.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_journal.ko Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /boot/kernel/nullfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/nullfs.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /boot/kernel/pflog.ko.symbols...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel/pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko #0 doadump () at pcpu.h:195 195 __asm __volatile("movq %%gs:0,%0" : "=r" (td)); Here the backtrace: #0 doadump () at pcpu.h:195 #1 0x0000000000000004 in ?? () #2 0xffffffff8050df19 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #3 0xffffffff8050e322 in panic (fmt=0x104 <Address 0x104 out of bounds>) at /usr/src/sys/kern/kern_shutdown.c:574 #4 0xffffffff807d2193 in trap_fatal (frame=0xffffff0006abb000, eva=Variable "eva" is not available. ) at /usr/src/sys/amd64/amd64/trap.c:757 #5 0xffffffff807d2ce5 in trap (frame=0xffffffff51e2e360) at /usr/src/sys/amd64/amd64/trap.c:558 #6 0xffffffff807b700e in calltrap () at /usr/src/sys/amd64/amd64/exception.S:209 #7 0xffffffff805bbc66 in rt_maskedcopy (src=0xffffffff51e2e6c8, dst=0xffffff00525ebd80, netmask=0xef3fdf377db53afa) at /usr/src/sys/net/route.c:1362 #8 0xffffffff805bc4e5 in rtrequest1_fib (req=11, info=0xffffffff51e2e4c0, ret_nrt=0xffffffff51e2e5e8, fibnum=0) at /usr/src/sys/net/route.c:1036 #9 0xffffffff805bd09d in rtrequest_fib (req=11, dst=0xffffffff51e2e6c8, gateway=0x0, netmask=0x0, flags=0, ret_nrt=0xffffffff51e2e5e8, fibnum=0) at /usr/src/sys/net/route.c:738 #10 0xffffffff805bd531 in rtalloc1_fib (dst=0xffffffff51e2e6c8, report=1, ignflags=18446744073709551615, fibnum=0) at /usr/src/sys/net/route.c:315 #11 0xffffffff805be749 in rtalloc_ign_fib (ro=0xffffffff51e2e6c0, ignore=0, fibnum=0) at /usr/src/sys/net/route.c:252 #12 0xffffffff805f4cad in ip_output (m=0xffffff0006b04b00, opt=0x0, ro=0xffffffff51e2e6c0, flags=0, imo=0x0, inp=0xffffff0006c41120) at /usr/src/sys/netinet/ip_output.c:230 #13 0xffffffff806582fa in tcp_output (tp=0xffffff0006c65b60) at /usr/src/sys/netinet/tcp_output.c:1128 #14 0xffffffff80663774 in tcp_usr_send (so=0xffffff0006aa85a0, flags=0, m=0xffffff00526f3c00, nam=Variable "nam" is not available. ) at tcp_offload.h:269 #15 0xffffffff8056addb in sosend_generic (so=0xffffff0006aa85a0, addr=0x0, uio=0x0, top=0xffffff00526f3c00, control=0x0, flags=Variable "flags" is not available. ) at /usr/src/sys/kern/uipc_socket.c:1246 #16 0xffffffff8069f73f in nfs_send (so=0xffffff0006aa85a0, nam=Variable "nam" is not available. ) at /usr/src/sys/nfsclient/nfs_socket.c:664 #17 0xffffffff806a2ab9 in nfs_request (vp=0xffffff0052bd9bd0, mrest=Variable "mrest" is not available. ) at /usr/src/sys/nfsclient/nfs_socket.c:1217 #18 0xffffffff806aadfa in nfs_readrpc (vp=0xffffff0052bd9bd0, uiop=0xffffffff51e2eb30, cred=0xffffff0052899d00) at /usr/src/sys/nfsclient/nfs_vnops.c:1119 #19 0xffffffff8069a1c9 in nfs_doio (vp=0xffffff0052bd9bd0, bp=0xffffffff26332020, cr=0xffffff0052899d00, td=Variable "td" is not available. ) at /usr/src/sys/nfsclient/nfs_bio.c:1571 #20 0xffffffff806a5e48 in nfssvc_iod (instance=Variable "instance" is not available. ) at /usr/src/sys/nfsclient/nfs_nfsiod.c:280 #21 0xffffffff804ea913 in fork_exit (callout=0xffffffff806a5c00 <nfssvc_iod>, arg=0xffffffff80b4c880, frame=0xffffffff51e2ec80) at /usr/src/sys/kern/kern_fork.c:810 #22 0xffffffff807b73ce in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:455 #23 0x0000000000000000 in ?? () #24 0x0000000000000000 in ?? () #25 0x0000000000000001 in ?? () #26 0x0000000000000000 in ?? () #27 0x0000000000000000 in ?? () #28 0x0000000000000000 in ?? () [...]> If so, running 'l > *0xffffffff80186249' (from the 'instruction pointer' line in the > fault message) would be helpful.This seems to point to crap... cam subsystem. 0xffffffff80186249 is in cam_periph_alloc (/usr/src/sys/cam/cam_periph.c:153) I'll try to give you the lines from the panic above... This seems to make more sense. (kgdb) l *0xffffffff805bbc66 0xffffffff805bbc66 is in rt_maskedcopy (/usr/src/sys/net/route.c:1366). 1361 rt_maskedcopy(struct sockaddr *src, struct sockaddr *dst, struct sockaddr *netmask) 1362 { 1363 register u_char *cp1 = (u_char *)src; 1364 register u_char *cp2 = (u_char *)dst; 1365 register u_char *cp3 = (u_char *)netmask; 1366 u_char *cplim = cp2 + *cp3; 1367 u_char *cplim2 = cp2 + *cp1; 1368 1369 *cp2++ = *cp1++; *cp2++ = *cp1++; /* copies sa_len & sa_family */ 1370 cp3 += 2; I don't know what I can do to help you more. Message me, if you need more details. I've disabled promiscuous mode now (disabled ipcad). First I/O tests showed no panics. But the server has run for 4 days without problems last time, so I'm going to let it run a bit longer. -- Martin
Chris Timmons
2009-May-14 22:32 UTC
kernel trap 12 with interrupts disabled [bge0 on 7.2R]
> Can you get a stack trace? Your panic is quite different then the original > one.Let me know if there is any other information which would be helpful. I rebooted the 7.0 kernel from July, and the machine has been happily chugging along running Nessus under load for almost 6 hours. 3:30PM up 5:42, 1 user, load averages: 33.67, 33.80, 35.14 Tomorrow I can see if the panic is easily reproduced. -c (kgdb) bt #0 doadump () at pcpu.h:196 #1 0xc07e2ee7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e31b9 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae49ec in trap_fatal (frame=0xee156a94, eva=28) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4c70 in trap_pfault (frame=0xee156a94, usermode=0, eva=28) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae561c in trap (frame=0xee156a94) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac9d2b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc07a4dac in devvn_refthread (vp=0x0, devp=0xee156b0c) at /usr/src/sys/kern/kern_conf.c:209 #8 0xc076cf64 in devfs_fp_check (fp=0xc78fadf4, devp=0xee156b0c, dswp=0xee156b08) at /usr/src/sys/fs/devfs/devfs_vnops.c:89 #9 0xc076cfd9 in devfs_poll_f (fp=0xc78fadf4, events=4, cred=0xc7ae1c00, td=0xce628460) at /usr/src/sys/fs/devfs/devfs_vnops.c:966 #10 0xc081cce1 in poll (td=0xce628460, uap=0xee156cfc) at file.h:280 #11 0xc0ae4fc5 in syscall (frame=0xee156d38) at /usr/src/sys/i386/i386/trap.c:1090 #12 0xc0ac9d90 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255 #13 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) quit
Kostik Belousov
2009-May-15 08:25 UTC
kernel trap 12 with interrupts disabled [bge0 on 7.2R]
On Thu, May 14, 2009 at 03:32:34PM -0700, Chris Timmons wrote:> > > >Can you get a stack trace? Your panic is quite different then the original > >one. > > Let me know if there is any other information which would be helpful. I > rebooted the 7.0 kernel from July, and the machine has been happily > chugging along running Nessus under load for almost 6 hours. > > 3:30PM up 5:42, 1 user, load averages: 33.67, 33.80, 35.14 > > Tomorrow I can see if the panic is easily reproduced. > > -c > > > (kgdb) bt > #0 doadump () at pcpu.h:196 > #1 0xc07e2ee7 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:418 > #2 0xc07e31b9 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xc0ae49ec in trap_fatal (frame=0xee156a94, eva=28) at > /usr/src/sys/i386/i386/trap.c:939 > #4 0xc0ae4c70 in trap_pfault (frame=0xee156a94, usermode=0, eva=28) at > /usr/src/sys/i386/i386/trap.c:852 > #5 0xc0ae561c in trap (frame=0xee156a94) at > /usr/src/sys/i386/i386/trap.c:530 > #6 0xc0ac9d2b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 > #7 0xc07a4dac in devvn_refthread (vp=0x0, devp=0xee156b0c) at > /usr/src/sys/kern/kern_conf.c:209 > #8 0xc076cf64 in devfs_fp_check (fp=0xc78fadf4, devp=0xee156b0c, > dswp=0xee156b08) at /usr/src/sys/fs/devfs/devfs_vnops.c:89Please, show the output of p *(struct file *)0xc78fadf4> #9 0xc076cfd9 in devfs_poll_f (fp=0xc78fadf4, events=4, cred=0xc7ae1c00, > td=0xce628460) at /usr/src/sys/fs/devfs/devfs_vnops.c:966 > #10 0xc081cce1 in poll (td=0xce628460, uap=0xee156cfc) at file.h:280 > #11 0xc0ae4fc5 in syscall (frame=0xee156d38) at > /usr/src/sys/i386/i386/trap.c:1090 > #12 0xc0ac9d90 in Xint0x80_syscall () at > /usr/src/sys/i386/i386/exception.s:255 > #13 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) quit > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20090515/6cc6703e/attachment.pgp
Chris Timmons
2009-May-15 12:32 UTC
kernel trap 12 with interrupts disabled [bge0 on 7.2R]
#8 0xc076cf64 in devfs_fp_check (fp=0xc78fadf4, devp=0xee156b0c,
dswp=0xee156b08) at /usr/src/sys/fs/devfs/devfs_vnops.c:89
89 *dswp = devvn_refthread(fp->f_vnode, devp);
(kgdb) p *(struct file *)0xc78fadf4
$1 = {f_list = {le_next = 0xc78ab5f0, le_prev = 0xc789e5f0}, f_type = 1,
f_data = 0xce5f9b00, f_flag = 3, f_mtxp = 0xc74540a0, f_ops = 0xc0c48e80,
f_cred = 0xc7ae1c00, f_count = 2, f_vnode = 0xc90f4000, f_offset = 0,
f_vnread_flags = 0, f_gcflag = 0, f_msgcount = 0, f_seqcount = 1,
f_nextoff = 0, f_label = 0x0, f_cdevpriv = 0x0}
On Fri, 15 May 2009, Kostik Belousov wrote:
>> #8 0xc076cf64 in devfs_fp_check (fp=0xc78fadf4, devp=0xee156b0c,
>> dswp=0xee156b08) at /usr/src/sys/fs/devfs/devfs_vnops.c:89
> Please, show the output of
> p *(struct file *)0xc78fadf4
Kostik Belousov
2009-May-15 13:06 UTC
kernel trap 12 with interrupts disabled [bge0 on 7.2R]
On Fri, May 15, 2009 at 05:32:49AM -0700, Chris Timmons wrote:> > #8 0xc076cf64 in devfs_fp_check (fp=0xc78fadf4, devp=0xee156b0c, > dswp=0xee156b08) at /usr/src/sys/fs/devfs/devfs_vnops.c:89 > 89 *dswp = devvn_refthread(fp->f_vnode, devp); > > (kgdb) p *(struct file *)0xc78fadf4 > $1 = {f_list = {le_next = 0xc78ab5f0, le_prev = 0xc789e5f0}, f_type = 1, > f_data = 0xce5f9b00, f_flag = 3, f_mtxp = 0xc74540a0, f_ops = 0xc0c48e80, > f_cred = 0xc7ae1c00, f_count = 2, f_vnode = 0xc90f4000, f_offset = 0, > f_vnread_flags = 0, f_gcflag = 0, f_msgcount = 0, f_seqcount = 1, > f_nextoff = 0, f_label = 0x0, f_cdevpriv = 0x0} > > > > On Fri, 15 May 2009, Kostik Belousov wrote: > > >>#8 0xc076cf64 in devfs_fp_check (fp=0xc78fadf4, devp=0xee156b0c, > >>dswp=0xee156b08) at /usr/src/sys/fs/devfs/devfs_vnops.c:89 > >Please, show the output of > >p *(struct file *)0xc78fadf4The file structure in the dump is fully initialized. It seems that the issue is with devfs replacing file ops vector with devfs-specific one in devfs_open() before the struct file is fully initialized in vn_open. Please, try the patch below (against 7) and report results. Index: fs/devfs/devfs_vnops.c ==================================================================--- fs/devfs/devfs_vnops.c (revision 192089) +++ fs/devfs/devfs_vnops.c (working copy) @@ -890,6 +890,7 @@ if (fp != NULL) { FILE_LOCK(fp); fp->f_data = dev; + fp->f_vnode = vp; FILE_UNLOCK(fp); } fpop = td->td_fpop; -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 195 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20090515/ebeb4743/attachment.pgp