Volker A. Brandt
2009-Sep-22 22:18 UTC
Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Hello all! It seems that I am unable to use an aggregation as a physical bridge for a CentOS 5.3 client. The dom0 is running SXCE 122 and has the following interfaces defined: root@glaurung# dladm show-link LINK CLASS MTU STATE OVER nge0 phys 1500 up -- nge1 phys 1500 up -- bge0 phys 1500 up -- bge1 phys 1500 up -- public0 aggr 1500 up bge0 nge0 root@glaurung# dladm show-aggr LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS public0 L4 auto off short ----- The three physical interfaces bge0, nge0, and bge1 are all connected to the same gigabit ethernet switch. The domU is running a PVM CentOS 5.3: root@cent-01# uname -a Linux cent-01 2.6.18-128.el5xen #1 SMP Wed Jan 21 11:12:42 EST 2009 x86_64 x86_64 x86_64 GNU/Linux Initially I defined the bridge for the xVM vnic to be public0: <interface type=''bridge''> <mac address=''00:bb:c0:00:40:01''/> <source bridge=''public0''/> <script path=''/usr/lib/xen/scripts/vif-vnic''/> </interface> [Sorry to the forum readers for the broken XML display.] This seriously confuses the CentOS domU. While booting, it will complain about a duplicate address: Sep 22 23:12:01 cent-01 avahi-daemon[1689]: Joining mDNS multicast group on interface eth0.IPv4 with address 192.168.222.157. Sep 22 23:12:01 cent-01 avahi-daemon[1689]: Registering new address record for 192.168.222.157 on eth0. Sep 22 23:12:02 cent-01 kernel: eth0: duplicate address detected! When the domU is up, TCP traffic does not work. ICMP is the only thing that seems to get through. Specifically, NFS is unusable. So I redefine the domU, having changed the vnic to go across bge1: <interface type=''bridge''> <mac address=''00:bb:c0:00:40:01''/> <source bridge=''bge1/> <script path=''/usr/lib/xen/scripts/vif-vnic''/> </interface> And instantly, everything starts working. Using snoop on the dom0, I see that cent-01 traffic indeed goes across bge1. Is this an inherent limitation of Solaris aggregations? If so, any pointer to the documentation of that fact would be appreciated. In that case, it''s strange that only CentOS is affected. WinXP and OpenSolaris show no problems at all. Is there a bug in the Linux ''avahi'' daemon that might trigger the problem? Maybe a bug in Solaris? Simply stopping the ''avahi'' stuff again does not solve anything; just moving the bridge to be bge1 does. Or have I missed some other setting within the CentOS domU? Thanks -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt
Volker A. Brandt
2009-Sep-28 07:06 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Hello all! Last week, I wrote a lengthy email about the following problem:> It seems that I am unable to use an aggregation as a physical bridge > for a CentOS 5.3 client.Is anyone out there running a Centos 5.3 domU under SXCE 122 or 123 and uses an aggregation as physical bridge? If so, could you please tell me so that I know that _someone_ has got it working? :-) Thanks -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt
Florian Manschwetus
2009-Sep-28 10:20 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Am 28.09.2009 09:06, schrieb Volker A. Brandt:> Hello all! > > > Last week, I wrote a lengthy email about the following problem: >> It seems that I am unable to use an aggregation as a physical bridge >> for a CentOS 5.3 client. > > Is anyone out there running a Centos 5.3 domU under SXCE 122 or 123 and > uses an aggregation as physical bridge? If so, could you please tell > me so that I know that _someone_ has got it working? :-) > > > Thanks -- VolkerI haven''t used Centos ever, but I have used for a long time a port-aggregation on x4150 SunFires with xvm. Running hvm Windows (with and without PV-IO) and OpenSolaris guests and also paravirtual Linux (Gentoo x64) and OpenSolaris guests. Never have had trouble with networking. Even not when the OpenSolaris Dom0 has crashed totally, the guests where running fine. So maybe it is a issue with crossbow support for your nics? My X4150s are equipped with intel''s e1000e dualTwin nic. Florian
Volker A. Brandt
2009-Sep-28 10:25 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Thank you for your reply, Florian.> I haven''t used Centos ever, but I have used for a long time a > port-aggregation on x4150 SunFires with xvm. Running hvm Windows (with > and without PV-IO) and OpenSolaris guests and also paravirtual Linux > (Gentoo x64) and OpenSolaris guests. Never have had trouble with > networking. Even not when the OpenSolaris Dom0 has crashed totally, the > guests where running fine. > So maybe it is a issue with crossbow support for your nics? > My X4150s are equipped with intel''s e1000e dualTwin nic.In my second email, I did not provide any context. Maybe I should have. :-) I am running WXP and OpenSolaris just fine across the aggregation. All NICs are crossbow-ready. You mention that you can run Gentoo. So to me it seems to be a CentOS-only problem right now. Thanks again -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt
Florian Manschwetus
2009-Sep-28 10:44 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Am 28.09.2009 12:25, schrieb Volker A. Brandt:> Thank you for your reply, Florian. > >> I haven''t used Centos ever, but I have used for a long time a >> port-aggregation on x4150 SunFires with xvm. Running hvm Windows (with >> and without PV-IO) and OpenSolaris guests and also paravirtual Linux >> (Gentoo x64) and OpenSolaris guests. Never have had trouble with >> networking. Even not when the OpenSolaris Dom0 has crashed totally, the >> guests where running fine. >> So maybe it is a issue with crossbow support for your nics? >> My X4150s are equipped with intel''s e1000e dualTwin nic. > > In my second email, I did not provide any context. Maybe I should > have. :-) I am running WXP and OpenSolaris just fine across the > aggregation. All NICs are crossbow-ready. > > You mention that you can run Gentoo. So to me it seems to be a > CentOS-only problem right now. > > > Thanks again -- VolkerOk, thats interesting. Currently I use a Xvm from 3.4 gate based on build 122. Currently no port aggregation due to other reasons. Futureplan includes the idea to use vlan trunking (thats afaik the term) maybe in combination with port aggregation if possible, need to check this and start testing the idea. Florian
Volker A. Brandt
2009-Sep-28 10:53 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
> Ok, thats interesting. Currently I use a Xvm from 3.4 gate based on > build 122.So you are newer, since I use the xVM that comes with SXCE 122: v3.3.2-xvm chgset ''Tue Aug 18 03:21:41 2009 -0700 18433:7e735e9e9bf6''> Currently no port aggregation due to other reasons. > Futureplan includes the idea to use vlan trunking (thats afaik the term) > maybe in combination with port aggregation if possible, need to check > this and start testing the idea.When you have done some testing I would like to hear the results! :-) Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt
Florian Manschwetus
2009-Sep-28 11:05 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Am 28.09.2009 12:53, schrieb Volker A. Brandt:>> Ok, thats interesting. Currently I use a Xvm from 3.4 gate based on >> build 122. > > So you are newer, since I use the xVM that comes with SXCE 122: > > v3.3.2-xvm chgset ''Tue Aug 18 03:21:41 2009 -0700 18433:7e735e9e9bf6''Uhm maybe that was misleading. I were using port aggregation with SXCE b90 - b109 or so. I stopped using it when gambling around with Xvm-Server EarlyAccess. After that I started testing and using self build xvm from gate (runs rock stable so I would prefer a putback as soon as possible). Florian> >> Currently no port aggregation due to other reasons. >> Futureplan includes the idea to use vlan trunking (thats afaik the term) >> maybe in combination with port aggregation if possible, need to check >> this and start testing the idea. > > When you have done some testing I would like to hear the results! :-) > > > Regards -- Volker
Bjørn Connolly
2009-Sep-29 11:49 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
* Volker A. Brandt (vab@bb-c.de) wrote:> > Ok, thats interesting. Currently I use a Xvm from 3.4 gate based on > > build 122. > > So you are newer, since I use the xVM that comes with SXCE 122: > > v3.3.2-xvm chgset ''Tue Aug 18 03:21:41 2009 -0700 18433:7e735e9e9bf6''I''m using the stock xvm from 121.> > Currently no port aggregation due to other reasons. > > Futureplan includes the idea to use vlan trunking (thats afaik the term) > > maybe in combination with port aggregation if possible, need to check > > this and start testing the idea. > > When you have done some testing I would like to hear the results! :-)It''s been a couple of weeks since I played around with this, so my mental notes might be a little out of whack. I''m using LACP on 2 Intel NIC''s # dladm show-phys LINK MEDIA STATE SPEED DUPLEX DEVICE e1000g1 Ethernet up 1000 full e1000g1 e1000g0 Ethernet up 1000 full e1000g0 # dladm show-aggr LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS aggr1 L4 auto active short ----- On top of that I''m using VLAN''s (vnic''s with vlan option) to do security separation between network segments. # dladm show-vnic LINK OVER SPEED MACADDRESS MACADDRTYPE VID xvm3_0 aggr1 1000 0:16:36:72:ff:cf fixed 11 xvm4_0 aggr1 1000 0:16:36:72:ff:d1 fixed 11 wifi0 aggr1 1000 2:8:20:49:a6:9e random 20 wan0 aggr1 1000 2:8:20:cc:43:e1 random 99 public0 aggr1 1000 2:8:20:a5:39:fa random 100 server0 aggr1 1000 2:8:20:16:ae:a2 random 11 xvm27_0 aggr1 1000 0:16:36:72:ff:df fixed 11 As I recall I was unable to get network traffic running for HVM guests (Windows, FreeBSD, Debian) when using the vlan option for vif-vnic. Using tcpdump in the domU I verified that the domU was receiving traffic and sending traffic (arp requests and responses) but I never saw them in dom0. Creating a static arp on the domU for another host on the same vlan enabled me to ping that host with a RTT of 0.000ms. Anyway I finally gave up and started using etherstub''s and routing on dom0 instead. Not optimal but at least it works. BTW It works perfectly for debian-lenny pv domU''s. Hopefully it will work for FreeBSD''s XEN port when/if that ever happens.... -- Bjørn Connolly Mostly RFC3514 compliant.
Volker A. Brandt
2009-Sep-29 19:48 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
> I''m using LACP on 2 Intel NIC''s > # dladm show-phys > LINK MEDIA STATE SPEED DUPLEX DEVICE > e1000g1 Ethernet up 1000 full e1000g1 > e1000g0 Ethernet up 1000 full e1000g0 > # dladm show-aggr > LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS > aggr1 L4 auto active short ----- > > On top of that I''m using VLAN''s (vnic''s with vlan option) to do security > separation between network segments. > > # dladm show-vnic > LINK OVER SPEED MACADDRESS MACADDRTYPE VID > xvm3_0 aggr1 1000 0:16:36:72:ff:cf fixed 11 > xvm4_0 aggr1 1000 0:16:36:72:ff:d1 fixed 11 > wifi0 aggr1 1000 2:8:20:49:a6:9e random 20 > wan0 aggr1 1000 2:8:20:cc:43:e1 random 99 > public0 aggr1 1000 2:8:20:a5:39:fa random 100 > server0 aggr1 1000 2:8:20:16:ae:a2 random 11 > xvm27_0 aggr1 1000 0:16:36:72:ff:df fixed 11 > > As I recall I was unable to get network traffic running for HVM guests > (Windows, FreeBSD, Debian) when using the vlan option for vif-vnic. > Using tcpdump in the domU I verified that the domU was receiving traffic > and sending traffic (arp requests and responses) but I never saw them in > dom0.These symptoms seem to be different from mine: my HVM guests work OK, the PVM guest NetBSD does so, too (more or less :-), just the PVM domU CentOS 5.3 consistently does not. Over a single physical link, everything is fine.> Creating a static arp on the domU for another host on the same > vlan enabled me to ping that host with a RTT of 0.000ms.Will try that RSN.> Anyway I finally gave up and started using etherstub''s and routing on > dom0 instead. Not optimal but at least it works.This sounds a bit more complicated, so I''ll leave that for later.> > BTW It works perfectly for debian-lenny pv domU''s. Hopefully it will > work for FreeBSD''s XEN port when/if that ever happens....NetBSD is pretty good in that regard. I wish they had a ZFS port and an NFSv4 port. :-) Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt
Florian Manschwetus
2009-Sep-29 20:04 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Am 29.09.2009 21:48, schrieb Volker A. Brandt:>> I''m using LACP on 2 Intel NIC''s >> # dladm show-phys >> LINK MEDIA STATE SPEED DUPLEX DEVICE >> e1000g1 Ethernet up 1000 full e1000g1 >> e1000g0 Ethernet up 1000 full e1000g0 >> # dladm show-aggr >> LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS >> aggr1 L4 auto active short ----- >> >> On top of that I''m using VLAN''s (vnic''s with vlan option) to do security >> separation between network segments. >> >> # dladm show-vnic >> LINK OVER SPEED MACADDRESS MACADDRTYPE VID >> xvm3_0 aggr1 1000 0:16:36:72:ff:cf fixed 11 >> xvm4_0 aggr1 1000 0:16:36:72:ff:d1 fixed 11 >> wifi0 aggr1 1000 2:8:20:49:a6:9e random 20 >> wan0 aggr1 1000 2:8:20:cc:43:e1 random 99 >> public0 aggr1 1000 2:8:20:a5:39:fa random 100 >> server0 aggr1 1000 2:8:20:16:ae:a2 random 11 >> xvm27_0 aggr1 1000 0:16:36:72:ff:df fixed 11 >> >> As I recall I was unable to get network traffic running for HVM guests >> (Windows, FreeBSD, Debian) when using the vlan option for vif-vnic. >> Using tcpdump in the domU I verified that the domU was receiving traffic >> and sending traffic (arp requests and responses) but I never saw them in >> dom0. > > These symptoms seem to be different from mine: my HVM guests work OK, > the PVM guest NetBSD does so, too (more or less :-), just the PVM domU > CentOS 5.3 consistently does not. Over a single physical link, everything > is fine. > >> Creating a static arp on the domU for another host on the same >> vlan enabled me to ping that host with a RTT of 0.000ms. > > Will try that RSN. > >> Anyway I finally gave up and started using etherstub''s and routing on >> dom0 instead. Not optimal but at least it works. > > This sounds a bit more complicated, so I''ll leave that for later. >> >> BTW It works perfectly for debian-lenny pv domU''s. Hopefully it will >> work for FreeBSD''s XEN port when/if that ever happens.... > > NetBSD is pretty good in that regard. I wish they had a ZFS port > and an NFSv4 port. :-)Uhm, worth to give opensolaris a try? Florian> > > Regards -- Volker
Volker A. Brandt
2009-Oct-12 13:48 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Some 2½ weeks ago I wrote:> It seems that I am unable to use an aggregation as a physical bridge > for a CentOS 5.3 client. > > The dom0 is running SXCE 122 and has the following interfaces defined:[...] I have upgraded the machine to SXCE 124, and the problem has disappeared. Checking the changelogs from 122 to 124, I haven''t spotted any relevant putbacks. So I don''t really know why it works now, but it does work. :-) Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab@bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgröße: 45 Geschäftsführer: Rainer J. H. Brandt und Volker A. Brandt
Mark Johnson
2009-Oct-12 13:52 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
Volker A. Brandt wrote:> Some 2½ weeks ago I wrote: > >> It seems that I am unable to use an aggregation as a physical bridge >> for a CentOS 5.3 client. >> >> The dom0 is running SXCE 122 and has the following interfaces defined: > [...] > > I have upgraded the machine to SXCE 124, and the problem has disappeared. > Checking the changelogs from 122 to 124, I haven''t spotted any relevant > putbacks. So I don''t really know why it works now, but it does work. > :-) >http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6855136 MRJ
David Edmondson
2009-Oct-12 14:00 UTC
Re: Using an aggregation as a physical bridge for a CentOS 5.3 vnic?
On 12 Oct 2009, at 2:52pm, Mark Johnson wrote:> Volker A. Brandt wrote: >> Some 2½ weeks ago I wrote: >>> It seems that I am unable to use an aggregation as a physical bridge >>> for a CentOS 5.3 client. >>> >>> The dom0 is running SXCE 122 and has the following interfaces >>> defined: >> [...] >> I have upgraded the machine to SXCE 124, and the problem has >> disappeared. >> Checking the changelogs from 122 to 124, I haven''t spotted any >> relevant >> putbacks. So I don''t really know why it works now, but it does >> work. >> :-) > > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6855136That wouldn''t fit - Volker is running a CentOS guest and that fix is for a Solaris guest.