Hi, After the update of em to 6.6.6 last, I experience watchdog timeouts on a server running 6-STABLE. I have two identical servers with Intel D915GAV boards. Both have Intel PRO/1000 PCI-Express network cards. Server balder: em0: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port 0xac00-0xac1f mem 0xff600000-0xff61ffff,0xff620000-0xff63ffff irq 16 at device 0.0 on pci5 em0: Ethernet address: 00:1b:21:00:48:c4 em0: [FAST] # vmstat -i interrupt total rate irq1: atkbd0 3 0 irq4: sio0 2 0 irq6: fdc0 12 0 irq14: ata0 68 0 irq16: em0 uhci3 219828879 450 irq19: uhci1++ 4287947 8 irq22: ahc0 232717293 476 irq23: uhci0 ehci0 1 0 cpu0: timer 976552804 2000 Total 1433387009 2935 # netstat -i Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll em0 1500 <Link#1> 00:1b:21:00:48:c4 209880531 773 206555522 84 0 em0 1500 10.255.253/24 balder 215210996 - 212337968 - - plip0 1500 <Link#2> 0 0 0 0 0 lo0 16384 <Link#3> 12040055 0 12055326 0 0 lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - - lo0 16384 localhost ::1 6 - 6 - - lo0 16384 your-net localhost 6249979 - 6249980 - - 00:00.0 Host bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL Memory Controller Hub (rev 04) 00:01.0 PCI bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL PCI Express Root Port (rev 04) 00:02.0 VGA compatible controller: Intel Corporation 82915G/GV/910GL Integrated Graphics Controller (rev 04) 00:1c.0 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 1 (rev 03) 00:1c.1 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 2 (rev 03) 00:1c.2 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 3 (rev 03) 00:1c.3 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 4 (rev 03) 00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #1 (rev 03) 00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #2 (rev 03) 00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #3 (rev 03) 00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #4 (rev 03) 00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB2 EHCI Controller (rev 03) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d3) 00:1f.0 ISA bridge: Intel Corporation 82801FB/FR (ICH6/ICH6R) LPC Interface Bridge (rev 03) 00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) IDE Controller (rev 03) 00:1f.2 IDE interface: Intel Corporation 82801FB/FW (ICH6/ICH6W) SATA Controller (rev 03) 00:1f.3 SMBus: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) SMBus Controller (rev 03) 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) 06:01.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U (rev 01) Server midgard: em0: <Intel(R) PRO/1000 Network Connection Version - 6.2.9> port 0xac00-0xac1f mem 0xff500000-0xff51ffff,0xff520000-0xff53ffff irq 16 at device 0.0 on pci5 em0: Ethernet address: 00:15:17:0e:05:f7 admglz@midgard> vmstat -i interrupt total rate irq1: atkbd0 11 0 irq4: sio0 2142746 0 irq6: fdc0 14 0 irq14: ata0 252 0 irq16: em0+ 666640101 164 irq19: atapci1+ 7932757 1 irq22: ahc0 87074425 21 cpu0: timer 3807810138 937 Total 4571600444 1125 admglz@midgard> netstat -i Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll em0 1500 <Link#1> 00:15:17:0e:05:f7 343771280 0 474609731 0 0 em0 1500 10.255.253/24 midgard 347467842 - 478700485 - - plip0 1500 <Link#2> 0 0 0 0 0 lo0 16384 <Link#3> 16821054 0 16947668 0 0 lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - - lo0 16384 localhost ::1 2610 - 2610 - - lo0 16384 your-net localhost 12616879 - 12616879 - - lo0 16384 10.255.253.12 appsrv1 0 - 0 - - lo0 16384 10.255.253.10 ca.glz.hidden-pow 0 - 0 - - lo0 16384 10.255.253.11 test 0 - 0 - - lo0 16384 10.255.253.13 secure 0 - 0 - - lo0 16384 10.255.253.18 rscds.hidden-powe 7 - 0 - - midgard# lspci 00:00.0 Host bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL Memory Controller Hub (rev 04) 00:01.0 PCI bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL PCI Express Root Port (rev 04) 00:02.0 VGA compatible controller: Intel Corporation 82915G/GV/910GL Integrated Graphics Controller (rev 04) 00:1c.0 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 1 (rev 03) 00:1c.1 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 2 (rev 03) 00:1c.2 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 3 (rev 03) 00:1c.3 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) PCI Express Port 4 (rev 03) 00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #1 (rev 03) 00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #2 (rev 03) 00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #3 (rev 03) 00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB UHCI #4 (rev 03) 00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) USB2 EHCI Controller (rev 03) 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d3) 00:1f.0 ISA bridge: Intel Corporation 82801FB/FR (ICH6/ICH6R) LPC Interface Bridge (rev 03) 00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) IDE Controller (rev 03) 00:1f.2 IDE interface: Intel Corporation 82801FB/FW (ICH6/ICH6W) SATA Controller (rev 03) 00:1f.3 SMBus: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) SMBus Controller (rev 03) 01:00.0 SCSI storage controller: Triones Technologies, Inc. Unknown device 2310 (rev 02) 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet Controller (Copper) (rev 06) 06:01.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U (rev 01) 06:02.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 46) When running netstat between servers balder and midgard, server balder get watchdog timeouts and resets the connection for a few seconds. Oct 19 13:12:47 balder kernel: em0: watchdog timeout -- resetting Oct 19 13:12:47 balder kernel: em0: link state changed to DOWN Oct 19 13:12:51 balder kernel: em0: link state changed to UP I have switched the cable between the two servers but get exactly the same problem. The switch is a Netgear GS108T with the latest firmware. The resp. dmesg.boot are attached. Please let me know if there is any other information I can supply to clear this. Best regards, G?ran L ................................................... the future isMobile Goran Lowkrantz <goran.lowkrantz@ismobile.com> System Architect, iaMobile AB Sandviksgatan 81, PO Box 58, S-971 03 Lule?, Sweden Mobile: +46(0)70-587 87 82 http://www.ismobile.com ............................................... -------------- next part -------------- A non-text attachment was scrubbed... Name: balder.dmesg.boot Type: application/octet-stream Size: 7785 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20071019/95383b6e/balder.dmesg.obj -------------- next part -------------- A non-text attachment was scrubbed... Name: midgard.dmesg.boot Type: application/octet-stream Size: 7992 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20071019/95383b6e/midgard.dmesg.obj
<goran.lowkrantz@ismobile.com> wrote:> Hi,snip> > When running netstat between servers balder and midgard, server balder > get watchdog timeouts and resets the connection for a few seconds. > Oct 19 13:12:47 balder kernel: em0: watchdog timeout -- resettings/netstat/netperf/ ................................................... the future isMobile Goran Lowkrantz <goran.lowkrantz@ismobile.com> System Architect, iaMobile AB Sandviksgatan 81, PO Box 58, S-971 03 Lule?, Sweden Mobile: +46(0)70-587 87 82 http://www.ismobile.com ...............................................
On 20/10/2007, at 1:06 AM, Goran Lowkrantz wrote:> Hi, > > After the update of em to 6.6.6 last, I experience watchdog timeouts > on a server running 6-STABLE. >"me too" on a Supermicro 5015MT+, although I notice my em0 is also sharing an interrupt with USB (uhci3)... not sure if that's the culprit. [pmurray@chance ~]$ dmesg Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-STABLE #0: Tue Oct 9 07:45:50 NZDT 2007 root@chance.open2view.net:/usr/obj/usr/src/sys/GENERIC ACPI APIC Table: <PTLTD APIC > Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU X3220 @ 2.40GHz (2394.01-MHz 686- class CPU) Origin = "GenuineIntel" Id = 0x6f7 Stepping = 7 Features = 0xbfebfbff < FPU ,VME ,DE ,PSE ,TSC ,MSR ,PAE ,MCE ,CX8 ,APIC ,SEP ,MTRR ,PGE ,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0xe3bd<SSE3,RSVD2,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM> AMD Features=0x20100000<NX,LM> AMD Features2=0x1<LAHF> Cores per package: 4 real memory = 2146304000 (2046 MB) avail memory = 2095353856 (1998 MB) ioapic0 <Version 2.0> irqs 0-23 on motherboard ioapic1 <Version 2.0> irqs 24-47 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: <PTLTD RSDT> on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: <ACPI CPU> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0 pci1: <ACPI PCI bus> on pcib1 pcib2: <ACPI PCI-PCI bridge> irq 17 at device 28.0 on pci0 pci9: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> at device 0.0 on pci9 pci10: <ACPI PCI bus> on pcib3 pcib4: <PCI-PCI bridge> at device 1.0 on pci10 pci11: <PCI bus> on pcib4 arcmsr0: <Areca SATA Host Adapter RAID Controller > mem 0xe0200000-0xe0200fff,0xe0800000-0xe0bfffff irq 26 at device 14.0 on pci11 ARECA RAID ADAPTER0: Driver Version 1.20.00.14 2007-2-05 ARECA RAID ADAPTER0: FIRMWARE VERSION V1.42 2006-10-13 pcib5: <ACPI PCI-PCI bridge> irq 17 at device 28.4 on pci0 pci13: <ACPI PCI bus> on pcib5 em0: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port 0x4000-0x401f mem 0xe0300000-0xe031ffff irq 16 at device 0.0 on pci13 em0: Ethernet address: 00:30:48:90:48:dc em0: [FAST] pcib6: <ACPI PCI-PCI bridge> irq 16 at device 28.5 on pci0 pci14: <ACPI PCI bus> on pcib6 em1: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port 0x5000-0x501f mem 0xe0400000-0xe041ffff irq 17 at device 0.0 on pci14 em1: Ethernet address: 00:30:48:90:48:dd em1: [FAST] uhci0: <UHCI (generic) USB controller> port 0x3000-0x301f irq 23 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: <UHCI (generic) USB controller> on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: <UHCI (generic) USB controller> port 0x3020-0x303f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: <UHCI (generic) USB controller> on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: <UHCI (generic) USB controller> port 0x3040-0x305f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] usb2: <UHCI (generic) USB controller> on uhci2 usb2: USB revision 1.0 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: <UHCI (generic) USB controller> port 0x3060-0x307f irq 16 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] usb3: <UHCI (generic) USB controller> on uhci3 usb3: USB revision 1.0 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem 0xe0000000-0xe00003ff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: <Intel 82801GB/R (ICH7) USB 2.0 controller> on ehci0 usb4: USB revision 2.0 uhub4: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered pcib7: <ACPI PCI-PCI bridge> at device 30.0 on pci0 pci15: <ACPI PCI bus> on pcib7 pci15: <display, VGA> at device 0.0 (no driver attached) isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel ICH7 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 ata0: <ATA channel 0> on atapci0 ata1: <ATA channel 1> on atapci0 pci0: <serial bus, SMBus> at device 31.3 (no driver attached) acpi_button0: <Power Button> on acpi0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FAST] ppc0: <ECP parallel printer port> port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: <Parallel port bus> on ppc0 plip0: <PLIP network interface> on ppbus0 lpt0: <Printer> on ppbus0 lpt0: Interrupt-driven port ppi0: <Parallel I/O> on ppbus0 pmtimer0 on isa0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xcafff,0xcb000-0xcbfff, 0xcc000-0xccfff,0xcd000-0xcdfff on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 2394010800 Hz quality 800 Timecounters tick every 1.000 msec Waiting 5 seconds for SCSI devices to settle acd0: CDROM <CD-224E-N/1.AA> at ata0-master UDMA33 pass2 at arcmsr0 bus 0 target 16 lun 0 pass2: <Areca RAID controller R001> Fixed Processor SCSI-0 device da0 at arcmsr0 bus 0 target 0 lun 0 da0: <Areca ARC-1110-VOL#00 R001> Fixed Direct Access SCSI-5 device da0: 166.666MB/s transfers (83.333MHz, offset 32, 16bit), Tagged Queueing Enabled da0: 28609MB (58592256 512 byte sectors: 255H 63S/T 3647C) da1 at arcmsr0 bus 0 target 0 lun 1 da1: <Areca ARC-1110-VOL#01 R001> Fixed Direct Access SCSI-5 device da1: 166.666MB/s transfers (83.333MHz, offset 32, 16bit), Tagged Queueing Enabled da1: 2117156MB (4335936000 512 byte sectors: 255H 63S/T 269899C) Trying to mount root from ufs:/dev/da0s1a WARNING: /backup was not properly dismounted em1: link state changed to UP em1: link state changed to DOWN em0: link state changed to UP ums0: Cyclades SUN/PC USB Terminator, rev 1.00/15.00, addr 2, iclass 3/1 ums0: 5 buttons and Z dir. ukbd0: Cyclades SUN/PC USB Terminator, rev 1.00/15.00, addr 2, iclass 3/1 kbd2 at ukbd0 ums0: at uhub0 port 1 (addr 2) disconnected ums0: detached ukbd0: at uhub0 port 1 (addr 2) disconnected ukbd0: detached gif0: promiscuous mode enabled gif0: promiscuous mode disabled gif0: promiscuous mode enabled gif0: promiscuous mode disabled gif0: promiscuous mode enabled gif0: promiscuous mode disabled em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP pid 38534 (locate.code), uid 65534 inumber 32898 on /tmp: filesystem full em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP em0: watchdog timeout -- resetting em0: link state changed to DOWN em0: link state changed to UP [pmurray@chance ~]$ vmstat -i interrupt total rate irq14: ata0 47 0 irq16: em0 uhci3 9168265 9 irq17: em1 107811254 114 irq18: uhci2 32252905 34 irq23: uhci0 ehci0 738 0 irq26: arcmsr0 33035746 34 cpu0: timer 1889578811 1999 Total 2071847766 2192 [pmurray@chance ~]$ pciconf -lv hostb0@pci0:0:0: class=0x060000 card=0x798015d9 chip=0x27788086 rev=0xc0 hdr=0x00 vendor = 'Intel Corporation' device = 'Server Memory Controller Hub' class = bridge subclass = HOST-PCI pcib1@pci0:1:0: class=0x060400 card=0x798015d9 chip=0x27798086 rev=0xc0 hdr=0x01 vendor = 'Intel Corporation' device = 'PCI Express Root Port' class = bridge subclass = PCI-PCI pcib2@pci0:28:0: class=0x060400 card=0x798015d9 chip=0x27d08086 rev=0x01 hdr=0x01 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) PCI Express Root Port' class = bridge subclass = PCI-PCI pcib5@pci0:28:4: class=0x060400 card=0x798015d9 chip=0x27e08086 rev=0x01 hdr=0x01 vendor = 'Intel Corporation' device = '82801GR/GH/GHM (ICH7 Family) PCI Express Root Port' class = bridge subclass = PCI-PCI pcib6@pci0:28:5: class=0x060400 card=0x798015d9 chip=0x27e28086 rev=0x01 hdr=0x01 vendor = 'Intel Corporation' device = '82801GR/GH/GHM (ICH7 Family) PCI Express Root Port' class = bridge subclass = PCI-PCI uhci0@pci0:29:0: class=0x0c0300 card=0x798015d9 chip=0x27c88086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci1@pci0:29:1: class=0x0c0300 card=0x798015d9 chip=0x27c98086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci2@pci0:29:2: class=0x0c0300 card=0x798015d9 chip=0x27ca8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) USB Universal Host Controller' class = serial bus subclass = USB uhci3@pci0:29:3: class=0x0c0300 card=0x798015d9 chip=0x27cb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) USB Universal Host Controller' class = serial bus subclass = USB ehci0@pci0:29:7: class=0x0c0320 card=0x798015d9 chip=0x27cc8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) USB 2.0 Enhanced Host Controller' class = serial bus subclass = USB pcib7@pci0:30:0: class=0x060401 card=0x798015d9 chip=0x244e8086 rev=0xe1 hdr=0x01 vendor = 'Intel Corporation' device = '82801BA/CA/DB/DBL/EB/ER/FB (ICH2/3/4/4/5/5/6), 6300ESB Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:31:0: class=0x060100 card=0x798015d9 chip=0x27b88086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801GB/GR (ICH7 Family) LPC Interface Controller' class = bridge subclass = PCI-ISA atapci0@pci0:31:1: class=0x01018a card=0x798015d9 chip=0x27df8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) Ultra ATA Storage Controller' class = mass storage subclass = ATA none0@pci0:31:3: class=0x0c0500 card=0x798015d9 chip=0x27da8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82801G (ICH7 Family) SMBus Controller' class = serial bus subclass = SMBus pcib3@pci9:0:0: class=0x060400 card=0x00000000 chip=0x032c8086 rev=0x09 hdr=0x01 vendor = 'Intel Corporation' device = '6702PXH PCI Express-to-PCI Express Bridge' class = bridge subclass = PCI-PCI ioapic0@pci9:0:1: class=0x080020 card=0x798015d9 chip=0x03268086 rev=0x09 hdr=0x00 vendor = 'Intel Corporation' device = 'PCI Bridge Hub I/OxAPIC Interrupt Controller A' class = base peripheral subclass = interrupt controller pcib4@pci10:1:0: class=0x060400 card=0x00000000 chip=0x03358086 rev=0x0a hdr=0x01 vendor = 'Intel Corporation' device = '80331 [Lindsay] I/O processor PCI-X bridge' class = bridge subclass = PCI-PCI arcmsr0@pci11:14:0: class=0x010400 card=0x111017d3 chip=0x111017d3 rev=0x00 hdr=0x00 vendor = 'Areca Technology Corporation' device = 'ARC-1110 4-Port PCI-X to SATA RAID Controller' class = mass storage subclass = RAID em0@pci13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = 'PRO/1000 PM' class = network subclass = ethernet em1@pci14:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet none1@pci15:0:0: class=0x030000 card=0x798015d9 chip=0x515e1002 rev=0x02 hdr=0x00 vendor = 'ATI Technologies Inc' class = display subclass = VGA [pmurray@chance ~]$
<goran.lowkrantz@ismobile.com> wrote:> Hi, > > After the update of em to 6.6.6 last, I experience watchdog timeouts on a > server running 6-STABLE. > > I have two identical servers with Intel D915GAV boards. Both have Intel > PRO/1000 PCI-Express network cards. > > Server balder: > em0: <Intel(R) PRO/1000 Network Connection Version - 6.6.6> port > 0xac00-0xac1f mem 0xff600000-0xff61ffff,0xff620000-0xff63ffff irq 16 at > device 0.0 on pci5 > em0: Ethernet address: 00:1b:21:00:48:c4 > em0: [FAST] > ># vmstat -i > interrupt total rate > irq1: atkbd0 3 0 > irq4: sio0 2 0 > irq6: fdc0 12 0 > irq14: ata0 68 0 > irq16: em0 uhci3 219828879 450 > irq19: uhci1++ 4287947 8 > irq22: ahc0 232717293 476 > irq23: uhci0 ehci0 1 0 > cpu0: timer 976552804 2000 > Total 1433387009 2935 > ># netstat -i > Name Mtu Network Address Ipkts Ierrs Opkts Oerrs > Coll > em0 1500 <Link#1> 00:1b:21:00:48:c4 209880531 773 206555522 > 84 0 > em0 1500 10.255.253/24 balder 215210996 - 212337968 > - - > plip0 1500 <Link#2> 0 0 0 0 > 0 > lo0 16384 <Link#3> 12040055 0 12055326 0 > 0 > lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - > - > lo0 16384 localhost ::1 6 - 6 - > - > lo0 16384 your-net localhost 6249979 - 6249980 - > - > > 00:00.0 Host bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL Memory > Controller Hub (rev 04) > 00:01.0 PCI bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL PCI Express > Root Port (rev 04) > 00:02.0 VGA compatible controller: Intel Corporation 82915G/GV/910GL > Integrated Graphics Controller (rev 04) > 00:1c.0 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 1 (rev 03) > 00:1c.1 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 2 (rev 03) > 00:1c.2 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 3 (rev 03) > 00:1c.3 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 4 (rev 03) > 00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #1 (rev 03) > 00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #2 (rev 03) > 00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #3 (rev 03) > 00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #4 (rev 03) > 00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB2 EHCI Controller (rev 03) > 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d3) > 00:1f.0 ISA bridge: Intel Corporation 82801FB/FR (ICH6/ICH6R) LPC > Interface Bridge (rev 03) > 00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) IDE Controller (rev 03) > 00:1f.2 IDE interface: Intel Corporation 82801FB/FW (ICH6/ICH6W) SATA > Controller (rev 03) > 00:1f.3 SMBus: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > SMBus Controller (rev 03) > 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet > Controller (Copper) (rev 06) > 06:01.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U (rev > 01) > > > Server midgard: > em0: <Intel(R) PRO/1000 Network Connection Version - 6.2.9> port > 0xac00-0xac1f mem 0xff500000-0xff51ffff,0xff520000-0xff53ffff irq 16 at > device 0.0 on pci5 > em0: Ethernet address: 00:15:17:0e:05:f7 > admglz@midgard> vmstat -i > interrupt total rate > irq1: atkbd0 11 0 > irq4: sio0 2142746 0 > irq6: fdc0 14 0 > irq14: ata0 252 0 > irq16: em0+ 666640101 164 > irq19: atapci1+ 7932757 1 > irq22: ahc0 87074425 21 > cpu0: timer 3807810138 937 > Total 4571600444 1125 > > admglz@midgard> netstat -i > Name Mtu Network Address Ipkts Ierrs Opkts Oerrs > Coll > em0 1500 <Link#1> 00:15:17:0e:05:f7 343771280 0 474609731 > 0 0 > em0 1500 10.255.253/24 midgard 347467842 - 478700485 > - - > plip0 1500 <Link#2> 0 0 0 0 > 0 > lo0 16384 <Link#3> 16821054 0 16947668 0 > 0 > lo0 16384 fe80:3::1 fe80:3::1 0 - 0 - > - > lo0 16384 localhost ::1 2610 - 2610 - > - > lo0 16384 your-net localhost 12616879 - 12616879 - > - > lo0 16384 10.255.253.12 appsrv1 0 - 0 - > - > lo0 16384 10.255.253.10 ca.glz.hidden-pow 0 - 0 - > - > lo0 16384 10.255.253.11 test 0 - 0 - > - > lo0 16384 10.255.253.13 secure 0 - 0 - > - > lo0 16384 10.255.253.18 rscds.hidden-powe 7 - 0 - > - > > midgard# lspci > 00:00.0 Host bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL Memory > Controller Hub (rev 04) > 00:01.0 PCI bridge: Intel Corporation 82915G/P/GV/GL/PL/910GL PCI Express > Root Port (rev 04) > 00:02.0 VGA compatible controller: Intel Corporation 82915G/GV/910GL > Integrated Graphics Controller (rev 04) > 00:1c.0 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 1 (rev 03) > 00:1c.1 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 2 (rev 03) > 00:1c.2 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 3 (rev 03) > 00:1c.3 PCI bridge: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > PCI Express Port 4 (rev 03) > 00:1d.0 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #1 (rev 03) > 00:1d.1 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #2 (rev 03) > 00:1d.2 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #3 (rev 03) > 00:1d.3 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB UHCI #4 (rev 03) > 00:1d.7 USB Controller: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) USB2 EHCI Controller (rev 03) > 00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d3) > 00:1f.0 ISA bridge: Intel Corporation 82801FB/FR (ICH6/ICH6R) LPC > Interface Bridge (rev 03) > 00:1f.1 IDE interface: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 > Family) IDE Controller (rev 03) > 00:1f.2 IDE interface: Intel Corporation 82801FB/FW (ICH6/ICH6W) SATA > Controller (rev 03) > 00:1f.3 SMBus: Intel Corporation 82801FB/FBM/FR/FW/FRW (ICH6 Family) > SMBus Controller (rev 03) > 01:00.0 SCSI storage controller: Triones Technologies, Inc. Unknown > device 2310 (rev 02) > 05:00.0 Ethernet controller: Intel Corporation 82572EI Gigabit Ethernet > Controller (Copper) (rev 06) > 06:01.0 SCSI storage controller: Adaptec AHA-2940U/UW/D / AIC-7881U (rev > 01) > 06:02.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host > Controller (rev 46) > > > When running netstat between servers balder and midgard, server balder > get watchdog timeouts and resets the connection for a few seconds. > Oct 19 13:12:47 balder kernel: em0: watchdog timeout -- resetting > Oct 19 13:12:47 balder kernel: em0: link state changed to DOWN > Oct 19 13:12:51 balder kernel: em0: link state changed to UP > > I have switched the cable between the two servers but get exactly the > same problem. The switch is a Netgear GS108T with the latest firmware. > > The resp. dmesg.boot are attached. > > Please let me know if there is any other information I can supply to > clear this. > > Best regards, > G?ran L >I have managed to get my performance back in two ways: - Switching to polling. - Build a kernel without USB. So it's the interrupt sharing between the network card and a USB hub that's the problem. /glz
Hi Jack, --On Thursday, November 01, 2007 13:36 -0700 Jack Vogel <jfvogel@gmail.com> wrote:> I should also note that this only applies to PCI-E NICs, 82571 and later. > > JackHave tested - enable MSI with the original 6.6.6 driver - the new driver files you sent to -stable with and without MSI enabled In all cases I can run all tests and programs that previous gave watchdog problems without any problems. Thanks! /glz ................................................... the future isMobile Goran Lowkrantz <goran.lowkrantz@ismobile.com> System Architect, isMobile AB Sandviksgatan 81, PO Box 58, S-971 03 Lule?, Sweden Mobile: +46(0)70-587 87 82 http://www.ismobile.com ...............................................