Michael L. Squires
2012-Jan-29 02:49 UTC
Regression in 8.2-STABLE bge code (from 7.4-STABLE)
There is a bug in the Tyan S4881/S4882 PCI-X bridges that was fixed with a
patch in 7.x (thank you very much). This patch is not present in the
8.2-STABLE code and the symptoms (watchdog timeouts) have recurred.
The watchdog timeouts do not appear to be present after I switched to an
Intel gigabit PCI-X card.
I did a brute-force patch of the 8.2-STABLE bge code using the patches for
7.4-STABLE; the resulting code compiled and, other than odd behavior at
startup, seems to be working normally.
This is using FreeBSD 8.2-STABLE amd64; I don't know what happens with
i386.
Given the age of the boards it may be easier if I just continue using the
Intel gigabit card but am happy to test anything that comes my way.
Thanks,
Mike Squires
mikes at siralan.org
Tyan S4882 (quad Opteron)
uname -a:
FreeBSD superxeon.familysquires.net 8.2-RELEASE-p5 FreeBSD 8.2-RELEASE-p5 #52:
at Jan 28 18:58:43 EST 2012 root@superxeon.familysquires.net:/usr/obj/usr/
c/sys/OPTERON8 amd64
bge0 dmesg (bge1 same; verbose; note bge cksum error):
Jan 28 20:59:28 superxeon kernel: em0: <Intel(R) PRO/1000 Legacy Network
Connect
ion 1.0.3> port 0x3000-0x303f mem 0xfe040000-0xfe05ffff irq 26 at device 2.0
on
pci2
Jan 28 20:59:28 superxeon kernel: em0: Reserved 0x20000 bytes for rid 0x10 type
3 at 0xfe040000
Jan 28 20:59:28 superxeon kernel: em0: Reserved 0x40 bytes for rid 0x20 type 4 a
t 0x3000
Jan 28 20:59:28 superxeon kernel: ioapic1: routing intpin 2 (PCI IRQ 26) to lapi
c 0 vector 55
Jan 28 20:59:28 superxeon kernel: em0: [FILTER]
Jan 28 20:59:28 superxeon kernel: em0: bpf attached
Jan 28 20:59:28 superxeon kernel: em0: Ethernet address: 00:0e:0c:36:54:64
Jan 28 20:59:28 superxeon kernel: pci0:2:9:0: bad VPD cksum, remain 72
Jan 28 20:59:28 superxeon kernel: bge0: <Broadcom Gigabit Ethernet
Controller, A
SIC rev. 0x002003> mem 0xfe010000-0xfe01ffff,0xfe000000-0xfe00ffff irq 24 at
dev
ice 9.0 on pci2
Jan 28 20:59:28 superxeon kernel: bge0: Reserved 0x10000 bytes for rid 0x10 type
3 at 0xfe010000
Jan 28 20:59:28 superxeon kernel: bge0: CHIP ID 0x00002003; ASIC REV 0x02; CHIP
REV 0x20; PCI
Jan 28 20:59:28 superxeon kernel: miibus0: <MII bus> on bge0
Jan 28 20:59:28 superxeon kernel: brgphy0: <BCM5704 10/100/1000baseTX PHY>
PHY 1
on miibus0
Jan 28 20:59:28 superxeon kernel: brgphy0: OUI 0x000818, model 0x0019, rev. 0
Jan 28 20:59:28 superxeon kernel: brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100
baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, au
to, auto-flow
Jan 28 20:59:28 superxeon kernel: bge0: bpf attached
Jan 28 20:59:28 superxeon kernel: bge0: Ethernet address: 00:e0:81:64:bc:7f
Jan 28 20:59:28 superxeon kernel: ioapic1: routing intpin 0 (PCI IRQ 24) to lapi
c 0 vector 56
Jan 28 20:59:28 superxeon kernel: bge0: [MPSAFE]
Jan 28 20:59:28 superxeon kernel: bge0: [ITHREAD]
Jan 28 20:59:28 superxeon kernel: pci0:2:9:1: bad VPD cksum, remain 72
ifconfig:
em0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=9b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
ether 00:0e:0c:36:54:64
media: Ethernet autoselect
status: no carrier
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTA
E>
ether 00:e0:81:64:bc:7f
inet 10.1.5.160 netmask 0xffffff00 broadcast 10.1.5.255
media: Ethernet autoselect (1000baseT
<full-duplex,flowcontrol,rxpause,
xpause>)
status: active
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTA
E>
ether 00:e0:81:64:bc:7e
media: Ethernet autoselect (none)
status: no carrier
plip0: flags=8810<POINTOPOINT,SIMPLEX,MULTICAST> metric 0 mtu 1500
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
options=3<RXCSUM,TXCSUM>
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
Changes to ./dev/bge/if_bgereg.h:
2813d2812
< #define BGE_FLAG_MBOX_REORDER 0x10000000
Changes to ./dev/bge/if_bge.c
369,370d368
< static int bge_mbox_reorder(struct bge_softc *);
<
628,629d625
< if ((sc->bge_flags & BGE_FLAG_MBOX_REORDER) != 0)
< CSR_READ_4(sc, off);
2486c2482
< if ((sc->bge_flags & BGE_FLAG_4G_BNDRY_BUG) != 0)
---> if ((sc->bge_flags & BGE_FLAG_4G_BNDRY_BUG) != 0) {
2487a2484,2493> /*
> * XXX
> * watchdog timeout issue was observed on BCM5704 which
> * lives behind PCI-X bridge(e.g AMD 8131 PCI-X bridge).
> * Limiting DMA address space to 32bits seems to address
> * it.
> */
> if (sc->bge_flags & BGE_FLAG_PCIX)
> lowaddr = BUS_SPACE_MAXADDR_32BIT;
> }
2641,2680d2646
< bge_mbox_reorder(struct bge_softc *sc)
< {
< /* Lists of PCI bridges that are known to reorder mailbox writes.
*/
< static const struct mbox_reorder {
< const uint16_t vendor;
< const uint16_t device;
< const char *desc;
< } const mbox_reorder_lists[] = {
< { 0x1022, 0x7450, "AMD-8131 PCI-X Bridge" },
< };
< devclass_t pcib;
< device_t dev;
< int i, count, unit;
<
< count = sizeof(mbox_reorder_lists) / sizeof(mbox_reorder_lists[0]);
< pcib = devclass_find("pcib");
< for (unit = 0; unit < devclass_get_maxunit(pcib); unit++) {
< dev = devclass_get_device(pcib, unit);
< if (dev == NULL)
< continue;
< #if 1
< printf("Found pcib%d : 0x%04x, 0x%04x\n", unit,
< pci_get_vendor(dev), pci_get_device(dev));
< #endif
< for (i = 0; i < count; i++) {
< if (pci_get_vendor(dev) =<
mbox_reorder_lists[i].vendor &&
< pci_get_device(dev) =<
mbox_reorder_lists[i].device) {
< device_printf(sc->bge_dev,
< "enabling MBOX workaround for
%s\n",
< mbox_reorder_lists[i].desc);
< return (1);
< }
< }
< }
< return (0);
< }
<
< static int
2954d2919
<
2956,2965d2920
< * Some PCI-X bridges are known to trigger write reordering to
< * the mailbox registers. Typical phenomena is watchdog timeouts
< * caused by out-of-order TX completions. Enable workaround for
< * PCI-X devices that live behind these bridges.
< */
< if (sc->bge_flags & BGE_FLAG_PCIX &&
bge_mbox_reorder(sc) != 0)
< sc->bge_flags |= BGE_FLAG_MBOX_REORDER;
<
< /*
<
On Sat, Jan 28, 2012 at 09:24:53PM -0500, Michael L. Squires wrote: Sorry for late reply. Had been busy due to relocation.> There is a bug in the Tyan S4881/S4882 PCI-X bridges that was fixed with a > patch in 7.x (thank you very much). This patch is not present in the > 8.2-STABLE code and the symptoms (watchdog timeouts) have recurred. >Hmm, I thought the mailbox reordering bug was avoided by limiting DMA address space to 32bits but it seems it was not right workaround for AMD 8131 PCI-X Bridge.> The watchdog timeouts do not appear to be present after I switched to an > Intel gigabit PCI-X card. > > I did a brute-force patch of the 8.2-STABLE bge code using the patches for > 7.4-STABLE; the resulting code compiled and, other than odd behavior at > startup, seems to be working normally. > > This is using FreeBSD 8.2-STABLE amd64; I don't know what happens with > i386. > > Given the age of the boards it may be easier if I just continue using the > Intel gigabit card but am happy to test anything that comes my way. >Try attached patch and let me know how it goes. I didn't enable 64bit DMA addressing though. I think the AMD-8131 PCI-X bridge needs both workarounds.> Thanks, > > Mike Squires > mikes at siralan.org-------------- next part -------------- A non-text attachment was scrubbed... Name: bge.mbox.reorder.diff Type: text/x-diff Size: 3417 bytes Desc: not available Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20120214/aafe9e03/bge.mbox.reorder.bin
Michael L. Squires
2012-Mar-12 02:58 UTC
Regression in 8.2-STABLE bge code (from 7.4-STABLE)
A patch which allows FreeBSD 8.3-STABLE to use the Broadcom GigE bge ports on the Tyan S4882 quad Opteron motherboard (and almost certainly on the S4881 motherboard, which had the same problem with 7.4-STABLE) has been developed by YongHyeon Pyun. The problem involves a bug in the PCI bridge which connects the Broadcom bge Ethernet ports. He will shortly be committing the patch to HEAD. Thank you! Mike Squires mikes at siralan.org UN*X at home since 1986