Dear colleagues,
I have a NFS server running that suddenly crashed on me after over 4
months uptime.
It's running 5.3-STABLE as of Fri Feb 11 14:23:23 CET
SMP system, Intel SE7501HG2 with two 3.02 GHz Xeons, HTT disabled and
4G RAM.
SRCU42X (amr) is used for storage, Intel Gigabit (em) used for network.
Scheduler and Preemtion options not tweaked (as they were in GENERIC
at the time)
I tried to "call doadump", but it just hung and did nothing :-(
uname -a:
FreeBSD localhost 5.3-STABLE FreeBSD 5.3-STABLE #1: Fri Feb 11
14:23:23 CET 2005 root@localhost:/usr/obj/usr/src/sys/PTSMP i386
db> trace
Tracing pid 86 tid 100089 td 0xc54a1e10
propagate_priority(c54a1e10,c08f3750,c54c6960,c0901ae0,c54a1e10) at
propagate_priority+0x7f
turnstile_wait(c56f6a80,c0901ae0,c54c6960) at turnstile_wait+0x2e1
_mtx_lock_sleep(c0901ae0,c54a1e10,0,0,0) at _mtx_lock_sleep+0xf9
nfsrv_rcv(c5b32b64,c5b5b580,1) at nfsrv_rcv+0x4e
sowakeup(c5b32b64,c5b32bb4) at sowakeup+0xad
udp_append(c5a8c870,c5c96810,c5a99200,1c,ee83dc60) at udp_append+0x158
udp_input(c5a99200,14,8e009fc3,0,0) at udp_input+0x5fb
ip_input(c5a99200) at ip_input+0x4f1
netisr_processqueue(c08f5718) at netisr_processqueue+0xa3
swi_net(0) at swi_net+0xf2
ithread_loop(c5455e00,ee83dd48) at ithread_loop+0x159
fork_exit(c0605048,c5455e00,ee83dd48) at fork_exit+0x75
fork_trampoline() at fork_trampoline+0x8
--- trap 0x1, eip = 0, esp = 0xee83dd7c, ebp = 0 ---
kernel config is default except for the following:
makeoptions DEBUG=-g
options VM_KMEM_SIZE_MAX=(400*1024*1024)
options KDB # Enable kernel debugger
support.
options KDB_UNATTENDED
options BREAK_TO_DEBUGGER
options DDB # Support DDB.
options GDB # Support remote GDB.
Although I have makeoptions -g, I'm not sure if I have the unstripped
binary available anymore. (unless it's copied from /usr/obj to /boot/
kernel).
-------------- next part --------------
FreeBSD localhost 5.3-STABLE FreeBSD 5.3-STABLE #1: Fri Feb 11 14:23:23 CET 2005
root@localhost:/usr/obj/usr/src/sys/PTSMP i386
sarepta# dmesg
Copyright (c) 1992-2005 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 5.3-STABLE #1: Fri Feb 11 14:23:23 CET 2005
root@localhost:/usr/obj/usr/src/sys/PTSMP
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Xeon(TM) CPU 3.06GHz (3056.82-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0xf29 Stepping = 9
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Hyperthreading: 2 logical CPUs
real memory = 4026466304 (3839 MB)
avail memory = 3941961728 (3759 MB)
ACPI APIC Table: <INTEL S7501HG0>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 6
ACPI-0698: *** Warning: Type override - [DEB_] had invalid type (Integer)
for Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [MLIB] had invalid type (Integer)
for Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [DATA] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [SIO_] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [LEDP] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [GPEN] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [GPST] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [GP1N] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [WUES] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [WUSE] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [SBID] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [SWCE] had invalid type (String) for
Scope operator, changed to (Scope)
ACPI-0698: *** Warning: Type override - [SMIR] had invalid type (String) for
Scope operator, changed to (Scope)
ioapic0 <Version 2.0> irqs 0-23 on motherboard
ioapic1 <Version 2.0> irqs 24-47 on motherboard
ioapic2 <Version 2.0> irqs 48-71 on motherboard
lapic0: Forcing LINT1 to edge trigger
npx0: <math processor> on motherboard
npx0: INT 16 interface
acpi0: <INTEL S7501HG0> on motherboard
acpi0: Power Button (fixed)
acpi_ec0: <Embedded Controller: GPE 0x8> port 0xca7,0xca6 on acpi0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_button0: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci0: <unknown> at device 0.1 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> at device 3.0 on pci0
pci2: <ACPI PCI bus> on pcib1
pci2: <base peripheral, interrupt controller> at device 28.0 (no driver
attached)
pcib2: <ACPI PCI-PCI bridge> at device 29.0 on pci2
pci4: <ACPI PCI bus> on pcib2
pcib3: <PCI-PCI bridge> at device 2.0 on pci4
pci5: <PCI bus> on pcib3
amr0: <LSILogic MegaRAID> mem 0xfe980000-0xfe9fffff,0xfc3f0000-0xfc3fffff
irq 52 at device 0.0 on pci5
amr0: <LSILogic Intel(R) RAID Controller SRCU42X> Firmware 411M, BIOS
H404, 128MB RAM
em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
0x3040-0x307f mem 0xfea60000-0xfea7ffff irq 58 at device 5.0 on pci4
em0: Ethernet address: 00:07:e9:31:99:6c
em0: Speed:N/A Duplex:N/A
em1: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
0x3000-0x303f mem 0xfea80000-0xfea9ffff irq 59 at device 5.1 on pci4
em1: Ethernet address: 00:07:e9:31:99:6d
em1: Speed:N/A Duplex:N/A
pci2: <base peripheral, interrupt controller> at device 30.0 (no driver
attached)
pcib4: <ACPI PCI-PCI bridge> at device 31.0 on pci2
pci3: <ACPI PCI bus> on pcib4
ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port
0x2c00-0x2cff,0x2800-0x28ff mem 0xfe7e0000-0xfe7e1fff irq 30 at device 4.0 on
pci3
aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port
0x2400-0x24ff,0x2000-0x20ff mem 0xfe7f0000-0xfe7f1fff irq 31 at device 4.1 on
pci3
aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
pci0: <unknown> at device 3.1 (no driver attached)
uhci0: <Intel 82801CA/CAM (ICH3) USB controller USB-A> port 0x4040-0x405f
irq 16 at device 29.0 on pci0
usb0: <Intel 82801CA/CAM (ICH3) USB controller USB-A> on uhci0
usb0: USB revision 1.0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <Intel 82801CA/CAM (ICH3) USB controller USB-B> port 0x4020-0x403f
irq 19 at device 29.1 on pci0
usb1: <Intel 82801CA/CAM (ICH3) USB controller USB-B> on uhci1
usb1: USB revision 1.0
uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhci2: <Intel 82801CA/CAM (ICH3) USB controller USB-C> port 0x4000-0x401f
irq 18 at device 29.2 on pci0
usb2: <Intel 82801CA/CAM (ICH3) USB controller USB-C> on uhci2
usb2: USB revision 1.0
uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci1: <ACPI PCI bus> on pcib5
fwohci0: <Texas Instruments TSB43AB23> mem
0xfe4b0000-0xfe4b3fff,0xfe4e0000-0xfe4e07ff irq 18 at device 8.0 on pci1
fwohci0: OHCI version 1.10 (ROM=1)
fwohci0: No. of Isochronous channels is 4.
fwohci0: EUI64 00:01:08:00:20:04:44:48
fwohci0: Phy 1394a available S400, 3 ports.
fwohci0: Link S400, max_rec 2048 bytes.
firewire0: <IEEE1394(FireWire) bus> on fwohci0
fwe0: <Ethernet over FireWire> on firewire0
if_fwe0: Fake Ethernet address: 02:01:08:04:44:48
fwe0: Ethernet address: 02:01:08:04:44:48
fwe0: if_start running deferred for Giant
sbp0: <SBP-2/SCSI over FireWire> on firewire0
fwohci0: Initiate bus reset
fwohci0: node_id=0xc800ffc2, gen=1, CYCLEMASTER mode
firewire0: 3 nodes, maxhop <= 2, cable IRM = 2 (me)
firewire0: bus manager 2 (me)
pci1: <display, VGA> at device 12.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH3 UDMA100 controller> port
0x3a0-0x3af,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
ata0: channel #0 on atapci0
ata1: channel #1 on atapci0
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
atkbdc0: <Keyboard controller (i8042)> port 0x64,0x60 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
fdc0: <floppy drive controller (FDE)> port 0x3f7,0x3f4-0x3f5,0x3f2-0x3f3
irq 6 drq 2 on acpi0
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0
sio0: type 16550A, console
sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
sio1: type 16550A
ppc0: <ECP parallel printer port> port 0x778-0x77f,0x378-0x37f irq 7 drq 3
on acpi0
ppc0: Generic chipset (ECP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
orm0: <ISA Option ROMs> at iomem
0xe4000-0xe7fff,0xc9800-0xcafff,0xc8000-0xc97ff,0xc0000-0xc7fff on isa0
pmtimer0 on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 10.000 msec
Waiting 15 seconds for SCSI devices to settle
amrd0: <LSILogic MegaRAID logical drive> on amr0
amrd0: 70102MB (143568896 sectors) RAID 1 (optimal)
amrd1: <LSILogic MegaRAID logical drive> on amr0
amrd1: 350510MB (717844480 sectors) RAID 5 (optimal)
firewire0: New S400 device ID:0010b92100109711
firewire0: New S400 device ID:0010b92100208717
ses0 at amr0 bus 0 target 6 lun 0
ses0: <ESG-SHV SCA HSBP M15 0.11> Fixed Processor SCSI-2 device
ses0: SAF-TE Compliant Device
ses1 at amr0 bus 1 target 5 lun 0
ses1: <ESG-SHV SCA HSBP M15 0.11> Fixed Processor SCSI-2 device
ses1: SAF-TE Compliant Device
da0 at sbp0 bus 0 target 0 lun 0
da0: <Maxtor OneTouch II 0310> Fixed Direct Access SCSI-4 device
da0: 50.000MB/s transfers
da0: 286188MB (586114704 512 byte sectors: 255H 63S/T 36483C)
da1 at sbp0 bus 0 target 1 lun 0
da1: <Maxtor OneTouch II 0310> Fixed Direct Access SCSI-4 device
da1: 50.000MB/s transfers
da1: 286188MB (586114704 512 byte sectors: 255H 63S/T 36483C)
lapic6: Forcing LINT1 to edge trigger
SMP: AP CPU #1 Launched!
Mounting root from ufs:/dev/amrd0s2a
WARNING: / was not properly dismounted
WARNING: /export/1 was not properly dismounted
/export/1: mount pending error: blocks 44 files 104
/export/1: superblock summary recomputed
WARNING: /tmp was not properly dismounted
WARNING: /usr was not properly dismounted
WARNING: /var was not properly dismounted
ipfw2 initialized, divert disabled, rule-based forwarding disabled, default to
deny, logging disabled
em0: Link is up 1000 Mbps Full Duplex
-------------- next part --------------
Frode Nordahl
frode@nordahl.net