Jeremy Chadwick
2007-Mar-28 03:33 UTC
em(4) losing link when if_bridge(4) addm/deletem are used
Firstly, I apologise ahead of time for sounding brash, but for the past
week I have been fighting with the simple concept of Ethernet bridging
over a VPN/secure tunnel, and it seems no matter what solution I find, I
manage to run into bugs or caveats which cause me to look for other
solutions... rinse lather repeat.
In the midst of the madness, I found that when doing either of the
following commands...
ifconfig bridge0 addm em1
ifconfig bridge0 deletem em1
...I see the entire em1 interface completely reset, bringing down link,
then up again shortly after. The logs on my switch also confirm this.
For sake of comparison, the bge(4) driver does not behave this way.
I am aware that if_bridge(4) enables promiscuous mode on whatever
interface is being added, ditto with -promisc on deletem. However, this
does not appear to be the cause of the problem, because I can do
"ifconfig em1 promisc" and "ifconfig em1 -promisc" without
loss of link.
In the case of OpenVPN, it's fairly common for the iface-up/down scripts
to do a "ifconfig bridge0 addm/deletem" when the tunnel goes up and
down. I'm sure one can see the implication from this...
Applicable details:
em1: <Intel(R) PRO/1000 Network Connection Version - 6.2.9> port
0xa800-0xa83f mem 0xf2020000-0xf203ffff irq 23 at device 11.0 on pci2
em1@pci2:11:0: class=0x020000 card=0x11138086 chip=0x10138086 rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82541EI Gigabit Ethernet Controller (Copper)'
class = network
subclass = ethernet
FreeBSD 6.2-STABLE #0: Tue Mar 27 09:19:46 PDT 2007
root@medusa.parodius.com:/usr/obj/usr/src/sys/MEDUSA
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
Andrew Thompson
2007-Mar-28 21:50 UTC
em(4) losing link when if_bridge(4) addm/deletem are used
On Tue, Mar 27, 2007 at 08:33:27PM -0700, Jeremy Chadwick wrote:> In the midst of the madness, I found that when doing either of the > following commands... > > ifconfig bridge0 addm em1 > ifconfig bridge0 deletem em1 > > ...I see the entire em1 interface completely reset, bringing down link, > then up again shortly after. The logs on my switch also confirm this. > > For sake of comparison, the bge(4) driver does not behave this way. > > I am aware that if_bridge(4) enables promiscuous mode on whatever > interface is being added, ditto with -promisc on deletem. However, this > does not appear to be the cause of the problem, because I can do > "ifconfig em1 promisc" and "ifconfig em1 -promisc" without loss of link.The only other thing the bridge does to the interface when removing it is to reinstate IFCAP_TXCSUM, which is disabled when the interface is bridged. If you manually toggle this with ifconfig do you get the same result? Andrew
Jeremy Chadwick
2007-Mar-28 22:30 UTC
em(4) losing link when if_bridge(4) addm/deletem are used
On Thu, Mar 29, 2007 at 09:37:09AM +1200, Andrew Thompson wrote:> On Tue, Mar 27, 2007 at 08:33:27PM -0700, Jeremy Chadwick wrote: > > In the midst of the madness, I found that when doing either of the > > following commands... > > > > ifconfig bridge0 addm em1 > > ifconfig bridge0 deletem em1 > > > > ...I see the entire em1 interface completely reset, bringing down link, > > then up again shortly after. The logs on my switch also confirm this. > > > > For sake of comparison, the bge(4) driver does not behave this way. > > > > I am aware that if_bridge(4) enables promiscuous mode on whatever > > interface is being added, ditto with -promisc on deletem. However, this > > does not appear to be the cause of the problem, because I can do > > "ifconfig em1 promisc" and "ifconfig em1 -promisc" without loss of link. > > The only other thing the bridge does to the interface when removing it > is to reinstate IFCAP_TXCSUM, which is disabled when the interface is > bridged. If you manually toggle this with ifconfig do you get the same > result?Bingo. medusa# ifconfig em1 -txcsum medusa# ifconfig em1 em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 options=8<VLAN_MTU> inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 ether 00:30:48:71:60:6b media: Ethernet autoselect status: no carrier medusa# grep em1 /var/log/messages Mar 28 15:29:34 medusa kernel: em1: link state changed to DOWN Mar 28 15:29:36 medusa kernel: em1: link state changed to UP -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Andrew Thompson
2007-Mar-28 22:59 UTC
em(4) losing link when if_bridge(4) addm/deletem are used
On Wed, Mar 28, 2007 at 03:30:45PM -0700, Jeremy Chadwick wrote:> On Thu, Mar 29, 2007 at 09:37:09AM +1200, Andrew Thompson wrote: > > On Tue, Mar 27, 2007 at 08:33:27PM -0700, Jeremy Chadwick wrote: > > > In the midst of the madness, I found that when doing either of the > > > following commands... > > > > > > ifconfig bridge0 addm em1 > > > ifconfig bridge0 deletem em1 > > > > > > ...I see the entire em1 interface completely reset, bringing down link, > > > then up again shortly after. The logs on my switch also confirm this. > > > > > The only other thing the bridge does to the interface when removing it > > is to reinstate IFCAP_TXCSUM, which is disabled when the interface is > > bridged. If you manually toggle this with ifconfig do you get the same > > result? > > Bingo. > > medusa# ifconfig em1 -txcsum > medusa# ifconfig em1 > em1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 > options=8<VLAN_MTU> > inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 > ether 00:30:48:71:60:6b > media: Ethernet autoselect > status: no carrier > medusa# grep em1 /var/log/messages > Mar 28 15:29:34 medusa kernel: em1: link state changed to DOWN > Mar 28 15:29:36 medusa kernel: em1: link state changed to UPAs a workaround you could remove txcsum at boot, then the bridge will not toggle it. Long term you could prod the em(4) maintainer to see if this can be resolved. Andrew