Hi !
Sometimes (once a week approximately) I have a problem with the same
symptoms described here on SMP FreeBSD 6.2-STABLE with dual AMD Opteron(tm)
Processor 850:
http://www.freebsd.org/cgi/query-pr.cgi?pr=104406&cat
Sometimes (apparently when CPU load suddenly goes up) all processes that
interacts with disk gets stuck in "ufs" state, but in my case
SIGSTOP/SIGCONT seemingly does not help.
uname -a output:
FreeBSD serv2.vsi.ru 6.2-STABLE FreeBSD 6.2-STABLE #2: Sat Mar 3 01:59:08
MSK 2007 oleg@serv2.vsi.ru:/usr/obj/usr/src/sys/serv2 i386
dmesg.boot:
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-STABLE #2: Sat Mar 3 01:59:08 MSK 2007
oleg@serv2.vsi.ru:/usr/obj/usr/src/sys/serv2
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Opteron(tm) Processor 850 (2389.26-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0x20f51 Stepping = 1
Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
Features2=0x1<SSE3>
AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow+,3DNow>
AMD Features2=0x1<LAHF>
real memory = 8589934592 (8192 MB)
avail memory = 8350457856 (7963 MB)
ACPI APIC Table: <PTLTD APIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 24-27 on motherboard
ioapic2 <Version 1.1> irqs 28-31 on motherboard
ioapic3 <Version 1.1> irqs 32-35 on motherboard
ioapic4 <Version 1.1> irqs 36-39 on motherboard
ioapic5 <Version 1.1> irqs 40-43 on motherboard
ioapic6 <Version 1.1> irqs 44-47 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD XSDT> on motherboard
acpi0: Power Button (fixed)
unknown: I/O range not supported
unknown: I/O range not supported
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0xf008-0xf00b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff,0xf000-0xf07f,0xf080-0xf0ff
iomem 0xd8000-0xdbfff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
ohci0: <OHCI (generic) USB controller> mem 0xfc900000-0xfc900fff irq 19 at
device 0.0 on pci1
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfc901000-0xfc901fff irq 19 at
device 0.1 on pci1
ohci1: [GIANT-LOCKED]
usb1: OHCI version 1.0, legacy support
usb1: SMM does not respond, resetting
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
pci1: <display, VGA> at device 5.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <AMD 8111 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1000-0x100f at device 7.1 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
pci0: <bridge> at device 7.3 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci2: <ACPI PCI bus> on pcib2
bge0: <Broadcom BCM5704 A3, ASIC rev. 0x2003> mem
0xfe010000-0xfe01ffff,0xfe000000-0xfe00ffff irq 25 at device 2.0 on pci2
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
bge0: Ethernet address: 00:09:3d:13:fd:00
bge1: <Broadcom BCM5704 A3, ASIC rev. 0x2003> mem
0xfe030000-0xfe03ffff,0xfe020000-0xfe02ffff irq 26 at device 2.1 on pci2
miibus1: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus1
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
bge1: Ethernet address: 00:09:3d:13:fd:01
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x2000-0x20ff mem
0xfe050000-0xfe05ffff,0xfe040000-0xfe04ffff irq 27 at device 4.0 on pci2
mpt0: [GIANT-LOCKED]
mpt0: MPI Version=1.2.15.0
mpt0: Capabilities: ( RAID-1E RAID-1 SAFTE )
mpt0: 0 Active Volumes (1 Max)
mpt0: 0 Hidden Drive Members (6 Max)
pci0: <base peripheral, interrupt controller> at device 10.1 (no driver
attached)
pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci0: <base peripheral, interrupt controller> at device 11.1 (no driver
attached)
pcib4: <ACPI Host-PCI bridge> iomem
0xfe301000-0xfe301fff,0xfe303000-0xfe303fff,0xfe305000-0xfe305fff,0xfe307000-0xfe307fff
on acpi0
pci32: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> mem 0xfe300000-0xfe300fff irq 32 at device
1.0
on pci32
pci33: <ACPI PCI bus> on pcib5
pci32: <base peripheral, interrupt controller> at device 1.1 (no driver
attached)
pcib6: <ACPI PCI-PCI bridge> mem 0xfe302000-0xfe302fff irq 36 at device
2.0
on pci32
pci34: <ACPI PCI bus> on pcib6
pci32: <base peripheral, interrupt controller> at device 2.1 (no driver
attached)
pcib7: <ACPI PCI-PCI bridge> mem 0xfe304000-0xfe304fff irq 40 at device
3.0
on pci32
pci35: <ACPI PCI bus> on pcib7
pci32: <base peripheral, interrupt controller> at device 3.1 (no driver
attached)
pcib8: <ACPI PCI-PCI bridge> mem 0xfe306000-0xfe306fff irq 44 at device
4.0
on pci32
pci36: <ACPI PCI bus> on pcib8
pci32: <base peripheral, interrupt controller> at device 4.1 (no driver
attached)
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0
sio0: type 16550A
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
acpi0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem
0xc0000-0xc7fff,0xc8000-0xc97ff,0xc9800-0xcafff,0xcb000-0xcefff on isa0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
IP Filter: v4.1.13 initialized. Default = block all, Logging = enabled
Waiting 5 seconds for SCSI devices to settle
acd0: DMA limited to UDMA33, controller found non-ATA66 cable
acd0: DVDROM <MATSHITADVD-ROM SR-8178/PZ21> at ata1-master UDMA33
ses0 at mpt0 bus 0 target 6 lun 0
ses0: <SDR GEM318P 1> Fixed Processor SCSI-2 device
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device
SMP: AP CPU #1 Launched!
da0 at mpt0 bus 0 target 0 lun 0
da0: <SEAGATE ST373207LC 0003> Fixed Direct Access SCSI-3 device
da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing
Enabled
da0: 70007MB (143374744 512 byte sectors: 255H 63S/T 8924C)
da1 at mpt0 bus 0 target 2 lun 0
da1: <SEAGATE ST336807LC 0C01> Fixed Direct Access SCSI-3 device
da1: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing
Enabled
da1: 35003MB (71687372 512 byte sectors: 255H 63S/T 4462C)
Trying to mount root from ufs:/dev/da0s1a
Accounting enabled
Recently I posted followup to this PR with description of the problem. Any
ideas on how to debug this ?
--
Oleg Derevenetz <oleg@vsi.ru> OOD3-RIPE
Phone: +7 4732 539880
Fax: +7 4732 531415 http://www.vsi.ru
CenterTelecom Voronezh ISP http://isp.vsi.ru
Hi !
Sometimes (once a week approximately) I have a problem with the same
symptoms described here on SMP FreeBSD 6.2-STABLE with dual AMD Opteron(tm)
Processor 850:
http://www.freebsd.org/cgi/query-pr.cgi?pr=104406&cat
Sometimes (apparently when CPU load suddenly goes up) all processes that
interacts with disk gets stuck in "ufs" state, but in my case
SIGSTOP/SIGCONT seemingly does not help.
uname -a output:
FreeBSD serv2.vsi.ru 6.2-STABLE FreeBSD 6.2-STABLE #2: Sat Mar 3 01:59:08
MSK 2007 oleg@serv2.vsi.ru:/usr/obj/usr/src/sys/serv2 i386
dmesg.boot:
Copyright (c) 1992-2007 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 6.2-STABLE #2: Sat Mar 3 01:59:08 MSK 2007
oleg@serv2.vsi.ru:/usr/obj/usr/src/sys/serv2
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: AMD Opteron(tm) Processor 850 (2389.26-MHz 686-class CPU)
Origin = "AuthenticAMD" Id = 0x20f51 Stepping = 1
Features=0x78bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2>
Features2=0x1<SSE3>
AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow+,3DNow>
AMD Features2=0x1<LAHF>
real memory = 8589934592 (8192 MB)
avail memory = 8350457856 (7963 MB)
ACPI APIC Table: <PTLTD APIC >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP): APIC ID: 1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 24-27 on motherboard
ioapic2 <Version 1.1> irqs 28-31 on motherboard
ioapic3 <Version 1.1> irqs 32-35 on motherboard
ioapic4 <Version 1.1> irqs 36-39 on motherboard
ioapic5 <Version 1.1> irqs 40-43 on motherboard
ioapic6 <Version 1.1> irqs 44-47 on motherboard
kbd1 at kbdmux0
acpi0: <PTLTD XSDT> on motherboard
acpi0: Power Button (fixed)
unknown: I/O range not supported
unknown: I/O range not supported
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0xf008-0xf00b on acpi0
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
acpi_button0: <Power Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff,0xf000-0xf07f,0xf080-0xf0ff
iomem 0xd8000-0xdbfff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> at device 6.0 on pci0
pci1: <ACPI PCI bus> on pcib1
ohci0: <OHCI (generic) USB controller> mem 0xfc900000-0xfc900fff irq 19 at
device 0.0 on pci1
ohci0: [GIANT-LOCKED]
usb0: OHCI version 1.0, legacy support
usb0: SMM does not respond, resetting
usb0: <OHCI (generic) USB controller> on ohci0
usb0: USB revision 1.0
uhub0: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1: <OHCI (generic) USB controller> mem 0xfc901000-0xfc901fff irq 19 at
device 0.1 on pci1
ohci1: [GIANT-LOCKED]
usb1: OHCI version 1.0, legacy support
usb1: SMM does not respond, resetting
usb1: <OHCI (generic) USB controller> on ohci1
usb1: USB revision 1.0
uhub1: AMD OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
pci1: <display, VGA> at device 5.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <AMD 8111 UDMA133 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1000-0x100f at device 7.1 on pci0
ata0: <ATA channel 0> on atapci0
ata1: <ATA channel 1> on atapci0
pci0: <bridge> at device 7.3 (no driver attached)
pcib2: <ACPI PCI-PCI bridge> at device 10.0 on pci0
pci2: <ACPI PCI bus> on pcib2
bge0: <Broadcom BCM5704 A3, ASIC rev. 0x2003> mem
0xfe010000-0xfe01ffff,0xfe000000-0xfe00ffff irq 25 at device 2.0 on pci2
miibus0: <MII bus> on bge0
brgphy0: <BCM5704 10/100/1000baseTX PHY> on miibus0
brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
bge0: Ethernet address: 00:09:3d:13:fd:00
bge1: <Broadcom BCM5704 A3, ASIC rev. 0x2003> mem
0xfe030000-0xfe03ffff,0xfe020000-0xfe02ffff irq 26 at device 2.1 on pci2
miibus1: <MII bus> on bge1
brgphy1: <BCM5704 10/100/1000baseTX PHY> on miibus1
brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX,
1000baseTX-FDX, auto
bge1: Ethernet address: 00:09:3d:13:fd:01
mpt0: <LSILogic 1030 Ultra4 Adapter> port 0x2000-0x20ff mem
0xfe050000-0xfe05ffff,0xfe040000-0xfe04ffff irq 27 at device 4.0 on pci2
mpt0: [GIANT-LOCKED]
mpt0: MPI Version=1.2.15.0
mpt0: Capabilities: ( RAID-1E RAID-1 SAFTE )
mpt0: 0 Active Volumes (1 Max)
mpt0: 0 Hidden Drive Members (6 Max)
pci0: <base peripheral, interrupt controller> at device 10.1 (no driver
attached)
pcib3: <ACPI PCI-PCI bridge> at device 11.0 on pci0
pci3: <ACPI PCI bus> on pcib3
pci0: <base peripheral, interrupt controller> at device 11.1 (no driver
attached)
pcib4: <ACPI Host-PCI bridge> iomem
0xfe301000-0xfe301fff,0xfe303000-0xfe303fff,0xfe305000-0xfe305fff,0xfe307000-0xfe307fff
on acpi0
pci32: <ACPI PCI bus> on pcib4
pcib5: <ACPI PCI-PCI bridge> mem 0xfe300000-0xfe300fff irq 32 at device
1.0
on pci32
pci33: <ACPI PCI bus> on pcib5
pci32: <base peripheral, interrupt controller> at device 1.1 (no driver
attached)
pcib6: <ACPI PCI-PCI bridge> mem 0xfe302000-0xfe302fff irq 36 at device
2.0
on pci32
pci34: <ACPI PCI bus> on pcib6
pci32: <base peripheral, interrupt controller> at device 2.1 (no driver
attached)
pcib7: <ACPI PCI-PCI bridge> mem 0xfe304000-0xfe304fff irq 40 at device
3.0
on pci32
pci35: <ACPI PCI bus> on pcib7
pci32: <base peripheral, interrupt controller> at device 3.1 (no driver
attached)
pcib8: <ACPI PCI-PCI bridge> mem 0xfe306000-0xfe306fff irq 44 at device
4.0
on pci32
pci36: <ACPI PCI bus> on pcib8
pci32: <base peripheral, interrupt controller> at device 4.1 (no driver
attached)
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0
sio0: type 16550A
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on
acpi0
fdc0: [FAST]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem
0xc0000-0xc7fff,0xc8000-0xc97ff,0xc9800-0xcafff,0xcb000-0xcefff on isa0
ppc0: parallel port not found.
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
IP Filter: v4.1.13 initialized. Default = block all, Logging = enabled
Waiting 5 seconds for SCSI devices to settle
acd0: DMA limited to UDMA33, controller found non-ATA66 cable
acd0: DVDROM <MATSHITADVD-ROM SR-8178/PZ21> at ata1-master UDMA33
ses0 at mpt0 bus 0 target 6 lun 0
ses0: <SDR GEM318P 1> Fixed Processor SCSI-2 device
ses0: 3.300MB/s transfers
ses0: SAF-TE Compliant Device
SMP: AP CPU #1 Launched!
da0 at mpt0 bus 0 target 0 lun 0
da0: <SEAGATE ST373207LC 0003> Fixed Direct Access SCSI-3 device
da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing
Enabled
da0: 70007MB (143374744 512 byte sectors: 255H 63S/T 8924C)
da1 at mpt0 bus 0 target 2 lun 0
da1: <SEAGATE ST336807LC 0C01> Fixed Direct Access SCSI-3 device
da1: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing
Enabled
da1: 35003MB (71687372 512 byte sectors: 255H 63S/T 4462C)
Trying to mount root from ufs:/dev/da0s1a
Accounting enabled
Recently I posted followup to this PR with description of the problem. Any
ideas on how to debug this ?
--
Oleg Derevenetz <oleg@vsi.ru> OOD3-RIPE
Phone: +7 4732 539880
Fax: +7 4732 531415 http://www.vsi.ru
CenterTelecom Voronezh ISP http://isp.vsi.ru
On Wed, Mar 07, 2007 at 05:22:38AM +0300, Oleg Derevenetz wrote:> Hi ! > > Sometimes (once a week approximately) I have a problem with the same > symptoms described here on SMP FreeBSD 6.2-STABLE with dual AMD Opteron(tm) > Processor 850: > > http://www.freebsd.org/cgi/query-pr.cgi?pr=104406&cat> > Sometimes (apparently when CPU load suddenly goes up) all processes that > interacts with disk gets stuck in "ufs" state, but in my case > SIGSTOP/SIGCONT seemingly does not help.See developer handbook, Deadlock Debugging chapter for instruction what information shall be gathered to debug the problem. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20070307/56e2d727/attachment.pgp
On Wed, Mar 07, 2007 at 05:22:38AM +0300, Oleg Derevenetz wrote:>> Sometimes (once a week approximately) I have a problem with the same >> symptoms described here on SMP FreeBSD 6.2-STABLE with dual AMD Opteron(tm) >> Processor 850: >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=104406&cat>> >> Sometimes (apparently when CPU load suddenly goes up) all processes that >> interacts with disk gets stuck in "ufs" state, but in my case >> SIGSTOP/SIGCONT seemingly does not help. > > See developer handbook, Deadlock Debugging chapter for instruction what > information shall be gathered to debug the problem.OK, I built kernel with debug options and will wait for stuck. By the way, when debug options turned on, I see this message on every boot when nullfs mounting in progress: acquiring duplicate lock of same type: "vnode interlock" 1st vnode interlock @ /usr/src/sys/kern/vfs_vnops.c:806 2nd vnode interlock @ /usr/src/sys/kern/vfs_subr.c:2040 KDB: stack backtrace: kdb_backtrace(3,cfc60300,c05926d0,c05926d0,c05542c4,...) at kdb_backtrace+0x29 witness_checkorder(cfd5c4dc,9,c051cf1e,7f8) at witness_checkorder+0x578 _mtx_lock_flags(cfd5c4dc,0,c051cf1e,7f8,cfb28b90,...) at _mtx_lock_flags+0x78 vrefcnt(cfd5c414) at vrefcnt+0x20 null_checkvp(cff5eae0,c050c4a6,215) at null_checkvp+0x56 null_lock(f02f1a68) at null_lock+0x66 VOP_LOCK_APV(c054d540,f02f1a68) at VOP_LOCK_APV+0x87 vn_lock(cff5eae0,1002,cfc60300,cff5eae0,cff5ed04,...) at vn_lock+0xac nullfs_root(cff76b90,2,f02f1ae0,cfc60300,0,8,0,c05cfca0,0,c051c79c,407) at nullfs_root+0x26 vfs_domount(cfc60300,cfe3d340,cfe3d130,d,cfe3d3f0,c05817e0,0,c051c79c,2bf) at vfs_domount+0x975 vfs_donmount(cfc60300,d,cfe73080,cfe73080,0,...) at vfs_donmount+0x3f9 nmount(cfc60300,f02f1d04) at nmount+0x8b syscall(3b,3b,3b,bf7fe5f5,bf7feea0,...) at syscall+0x25b Xint0x80_syscall() at Xint0x80_syscall+0x1f --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280bc0e7, esp = 0xbf7fe5bc, ebp = 0xbf7fee38 --- This host have nullfs filesystems. Is this can be related to deadlock ? -- Oleg Derevenetz <oleg@vsi.ru> OOD3-RIPE Phone: +7 4732 539880 Fax: +7 4732 531415 http://www.vsi.ru CenterTelecom Voronezh ISP http://isp.vsi.ru
On Fri, Mar 09, 2007 at 06:08:25PM +0300, Oleg Derevenetz wrote:> On Wed, Mar 07, 2007 at 05:22:38AM +0300, Oleg Derevenetz wrote: > > >>Sometimes (once a week approximately) I have a problem with the same > >>symptoms described here on SMP FreeBSD 6.2-STABLE with dual AMD > >>Opteron(tm) > >>Processor 850: > >> > >>http://www.freebsd.org/cgi/query-pr.cgi?pr=104406&cat> >> > >>Sometimes (apparently when CPU load suddenly goes up) all processes that > >>interacts with disk gets stuck in "ufs" state, but in my case > >>SIGSTOP/SIGCONT seemingly does not help. > > > >See developer handbook, Deadlock Debugging chapter for instruction what > >information shall be gathered to debug the problem. > > OK, I built kernel with debug options and will wait for stuck. By the way, > when debug options turned on, I see this message on every boot when nullfs > mounting in progress: > > acquiring duplicate lock of same type: "vnode interlock" > 1st vnode interlock @ /usr/src/sys/kern/vfs_vnops.c:806 > 2nd vnode interlock @ /usr/src/sys/kern/vfs_subr.c:2040 > KDB: stack backtrace: > kdb_backtrace(3,cfc60300,c05926d0,c05926d0,c05542c4,...) at > kdb_backtrace+0x29 > witness_checkorder(cfd5c4dc,9,c051cf1e,7f8) at witness_checkorder+0x578 > _mtx_lock_flags(cfd5c4dc,0,c051cf1e,7f8,cfb28b90,...) at > _mtx_lock_flags+0x78 > vrefcnt(cfd5c414) at vrefcnt+0x20 > null_checkvp(cff5eae0,c050c4a6,215) at null_checkvp+0x56 > null_lock(f02f1a68) at null_lock+0x66 > VOP_LOCK_APV(c054d540,f02f1a68) at VOP_LOCK_APV+0x87 > vn_lock(cff5eae0,1002,cfc60300,cff5eae0,cff5ed04,...) at vn_lock+0xac > nullfs_root(cff76b90,2,f02f1ae0,cfc60300,0,8,0,c05cfca0,0,c051c79c,407) at > nullfs_root+0x26 > vfs_domount(cfc60300,cfe3d340,cfe3d130,d,cfe3d3f0,c05817e0,0,c051c79c,2bf) > at vfs_domount+0x975 > vfs_donmount(cfc60300,d,cfe73080,cfe73080,0,...) at vfs_donmount+0x3f9 > nmount(cfc60300,f02f1d04) at nmount+0x8b > syscall(3b,3b,3b,bf7fe5f5,bf7feea0,...) at syscall+0x25b > Xint0x80_syscall() at Xint0x80_syscall+0x1f > --- syscall (378, FreeBSD ELF32, nmount), eip = 0x280bc0e7, esp = > 0xbf7fe5bc, ebp = 0xbf7fee38 --- > > This host have nullfs filesystems. Is this can be related to deadlock ?This is harmless, just ignore it. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20070309/60206289/attachment.pgp