I found a solution to my em0 problem - I dropped in a PCI card with two
Intel controllers on it (em1 and em2 obviously) and those work fine.
So, the question is, what is the difference between the plug in card,
which is 82546EB, and the onboard 82566DM controller which makes the
latter stop working under circumstances where the former does not ?
Relevent bits of pciconv -lv:
em0@pci0:0:25:0: class=0x020000 card=0x281e103c chip=0x10bd8086 rev=0x02
hdr=0x00
vendor = 'Intel Corporation'
device = 'Intel 82566DM Gigabit Ethernet Adapter (82566DM)'
class = network
subclass = ethernet
em1@pci0:7:4:0: class=0x020000 card=0x11798086 chip=0x10798086 rev=0x03 hdr=0x00
vendor = 'Intel Corporation'
device = 'Dual Port Gigabit Ethernet Controller (82546EB)'
class = network
subclass = ethernet
em2@pci0:7:4:1: class=0x020000 card=0x11798086 chip=0x10798086 rev=0x03 hdr=0x00
vendor = 'Intel Corporation'
device = 'Dual Port Gigabit Ethernet Controller (82546EB)'
class = network
subclass = ethernet
The only thing I noticed was that em0 uses MSI, whereas the pliug
in card does not. The sitiuation under which it stopped was very
heavy receive traffic into the interface. Unfortunately I didnt get much
time to debug it, as it's a pretty critical server. But I did observer that
a quick ifconfig down/up doesn't fix it - the machine needed to reboot.
By 'stop' I mean that the interface looks fine, but doesnt actially
appear to receive packets anymore.
-pete.