I''m trying to set up Xen 3.0.2 to serve multiple guest domains, each using a unique VLAN. I start xend with a simple script that executes network-bridge for several domains with different VLANs: dir=$(dirname "$0") "$dir/network-bridge" "$@" vifnum=0 netdev=eth0.140 bridge=xenbr140 "$dir/network-bridge" "$@" vifnum=1 netdev=eth0.160 bridge=xenbr160 However, it doesn''t seem to work - only one guest has network connectivity. When I run tcpdump on dom0, I see that some packets are getting through from domU which has problems with network access, but that''s about it. How should I start multiple guest domains to use different VLANs? I searched the mailing list, and found some references to "vif-vlan" scripts etc., but didn''t find anything that would solve my problem. Is there anyone out there using Xen with a couple of domains, which use different VLANs? I would be thankful for any suggestions on where the problem might be (and no, it''s not checksumming). -- Tomasz Chmielewski wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Tomasz Chmielewski wrote:> I''m trying to set up Xen 3.0.2 to serve multiple guest domains, each > using a unique VLAN.> Is there anyone out there using Xen with a couple of domains, which use > different VLANs? I would be thankful for any suggestions on where the > problem might be (and no, it''s not checksumming).I''m afraid I can''t offer any specific suggestions for your setup, but I will mention that I''m successfully running several domUs with different VLANs. However, both my dom0 and domU are Gentoo, and both our init.d system and its interaction with Xen is a little different from other distros. I''m using tagged VLANs on a single physical interface (eth0), with the vlan interfaces named vlanN . I create a separate bridge for each VLAN, named after the VLAN''s common-name (eg. dmz, external, staff, student, etc). Each vlanN interface is added to the corresponding bridge, and any domU which needs networking on that particular VLAN has its vif assigned to the necessary bridge, in the usual bridge=whatever manner. This is all handled using the stock Gentoo init.d scripts. While this may only be of use to Gentoo users, here is my current config: aross@oak ~ $ ls -la /etc/init.d/ | fgrep net. lrwxrwxrwx 1 root root 8 Aug 28 12:55 net.dmz -> net.eth0 lrwxrwxrwx 1 root root 6 Aug 17 16:00 net.eth0 -> net.lo lrwxrwxrwx 1 root root 12 Aug 2 14:11 net.external -> net.eth0 -rwxr-xr-x 1 root root 30045 Aug 17 16:00 net.lo lrwxrwxrwx 1 root root 8 Aug 2 14:09 net.printers -> net.eth0 aross@oak ~ $ cat /etc/conf.d/net vlans_eth0="3 10 11" vconfig_eth0=( "set_name_type VLAN_PLUS_VID_NO_PAD" ) config_eth0=( "dhcp" ) config_vlan3=( "null" ) config_vlan10=( "null" ) config_vlan11=( "null" ) bridge_external="vlan10" config_external=( "null" ) depend_external() { need net.eth0 } bridge_printers="vlan3" config_printers=( "null" ) depend_printers() { need net.eth0 } bridge_dmz="vlan11" config_dmz=( "null" ) depend_dmz() { need net.eth0 } aross@oak ~ $ cat /etc/xen/printing memory = 128; kernel = "/boot/xen-domU-2.6.16.26"; disk = [ "phy:evms/printing,xvda,w" ]; root = "/dev/xvda ro"; vif = [ "mac=00:16:3E:77:69:07,bridge=dmz", "mac=00:16:3E:77:69:08,bridge=printers" ] aross@oak ~ $ cat /etc/xen/gentoo memory = 128; kernel = "/boot/xen-domU-2.6.16.26"; disk = [ "phy:evms/gentoo,xvda,w" ]; root = "/dev/xvda ro"; vif = [ "mac=00:16:3E:77:69:01,bridge=external" ] Cheers Andrew _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Aug-30 08:30 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Andrew Ross wrote:> Tomasz Chmielewski wrote: >> I''m trying to set up Xen 3.0.2 to serve multiple guest domains, each >> using a unique VLAN. > >> Is there anyone out there using Xen with a couple of domains, which use >> different VLANs? I would be thankful for any suggestions on where the >> problem might be (and no, it''s not checksumming). > > I''m afraid I can''t offer any specific suggestions for your setup, but I > will mention that I''m successfully running several domUs with different > VLANs. > > However, both my dom0 and domU are Gentoo, and both our init.d system > and its interaction with Xen is a little different from other distros.Did you do anything unusual to /etc/xen/scripts/*? I.e., did you change anything there, most likely, network-bridge and/or vif-bridge scripts, which I assume you are running (or perhaps not, as your setup seem a bit customized)? -- Tomasz Chmielewski wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Aug-30 14:11 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Andrew Ross wrote:> Tomasz Chmielewski wrote: >> I''m trying to set up Xen 3.0.2 to serve multiple guest domains, each >> using a unique VLAN. > >> Is there anyone out there using Xen with a couple of domains, which use >> different VLANs? I would be thankful for any suggestions on where the >> problem might be (and no, it''s not checksumming). > > I''m afraid I can''t offer any specific suggestions for your setup, but I > will mention that I''m successfully running several domUs with different > VLANs. > > However, both my dom0 and domU are Gentoo, and both our init.d system > and its interaction with Xen is a little different from other distros. > > I''m using tagged VLANs on a single physical interface (eth0), with the > vlan interfaces named vlanN . I create a separate bridge for each VLAN, > named after the VLAN''s common-name (eg. dmz, external, staff, student, > etc). Each vlanN interface is added to the corresponding bridge, and any > domU which needs networking on that particular VLAN has its vif assigned > to the necessary bridge, in the usual bridge=whatever manner.The only way I could make it work is to set up vlans inside domU domains. It works, but I don''t like it. -- Tomasz Chmielewski wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Sébastien CRAMATTE
2006-Aug-30 15:12 UTC
Re: [Xen-users] can''t get multiple VLANs to work
I''ve configured VLAN under debian sarge (kernel 2.6.16.19 + xen 3.0.2) using this howto felipe-alfaro.org/blog/category/xen The Felipe''s howto explain two approach in depth (vlan config in domU and 1 vlan per bridge) Unfortunately the Andrew''s approach with one vlan per bridge doesn''t work for me ... I must use Felipe''s approach that configure vlan directly in each domU. Morevover I force the mtu of vlan interface to "1496" in each domU I''ve seen a message about this in xen-devel list : lists.xensource.com/archives/html/xen-devel/2006-08/msg00935.html Bonding + vlan not works ! It''s very very slow or I loose packet randomly ... I''ve tried to setup it using methods above... neither of them work ... Regards Tomasz Chmielewski a écrit :> Andrew Ross wrote: >> Tomasz Chmielewski wrote: >>> I''m trying to set up Xen 3.0.2 to serve multiple guest domains, each >>> using a unique VLAN. >> >>> Is there anyone out there using Xen with a couple of domains, which use >>> different VLANs? I would be thankful for any suggestions on where the >>> problem might be (and no, it''s not checksumming). >> >> I''m afraid I can''t offer any specific suggestions for your setup, but I >> will mention that I''m successfully running several domUs with different >> VLANs. >> >> However, both my dom0 and domU are Gentoo, and both our init.d system >> and its interaction with Xen is a little different from other distros. >> >> I''m using tagged VLANs on a single physical interface (eth0), with the >> vlan interfaces named vlanN . I create a separate bridge for each VLAN, >> named after the VLAN''s common-name (eg. dmz, external, staff, student, >> etc). Each vlanN interface is added to the corresponding bridge, and any >> domU which needs networking on that particular VLAN has its vif assigned >> to the necessary bridge, in the usual bridge=whatever manner. > > The only way I could make it work is to set up vlans inside domU domains. > > It works, but I don''t like it. > >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Hello,
I have Xen running with domUs in different VLAN''s. I have xen on a
debian sarge server. My configuration is:
* In dom0, I have a bridge for each vlan (different than the native vlan
of interface eth0, that I don''t need to tag). For each vlan the
configuration is (for vlan 302):
vconfig add eth0 302
/etc/xen/scripts/network-bridge start netdev=eth0.302 bridge=xen-br302
antispoof=no
/sbin/ifconfig eth0.302 up
* In the domU the configuration I have is:
vif = [ ''bridge=xen-br302'' ]
* The only problem is that un domU I have to run "/usr/sbin/ethtool -K
eth0 tx off" after bring it up.
--
Angel L. Mateo Martínez
Sección de Telemática
Área de Tecnologías de la Información _o)
y las Comunicaciones Aplicadas (ATICA) / \\
um.es/atica _(___V
Tfo: 968367590
Fax: 968398337
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
lists.xensource.com/xen-users
Sébastien CRAMATTE
2006-Aug-31 07:13 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Angel L. Mateo a écrit :> Hello, > > I have Xen running with domUs in different VLAN''s. I have xen on a > debian sarge server. My configuration is: > > * In dom0, I have a bridge for each vlan (different than the native vlan > of interface eth0, that I don''t need to tag). For each vlan the > configuration is (for vlan 302): > > vconfig add eth0 302 > /etc/xen/scripts/network-bridge start netdev=eth0.302 bridge=xen-br302 > antispoof=no > /sbin/ifconfig eth0.302 up > > * In the domU the configuration I have is: > > vif = [ ''bridge=xen-br302'' ] > > * The only problem is that un domU I have to run "/usr/sbin/ethtool -K > eth0 tx off" after bring it up. > >You don''t have problem with UDP ? Have you DNS or DHCP services running in your VMs ? For my own I loose UDP traffic If I try to setup one vlan per bridge in the DOM0 To resolve this I must setup VLAN in each VM Have you test bonding + vlans ? Regards _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Aug-31 07:33 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Sébastien CRAMATTE wrote:> Angel L. Mateo a écrit : >> Hello, >> >> I have Xen running with domUs in different VLAN''s. I have xen on a >> debian sarge server. My configuration is: >> >> * In dom0, I have a bridge for each vlan (different than the native vlan >> of interface eth0, that I don''t need to tag). For each vlan the >> configuration is (for vlan 302): >> >> vconfig add eth0 302 >> /etc/xen/scripts/network-bridge start netdev=eth0.302 bridge=xen-br302 >> antispoof=no >> /sbin/ifconfig eth0.302 up >> >> * In the domU the configuration I have is: >> >> vif = [ ''bridge=xen-br302'' ] >> >> * The only problem is that un domU I have to run "/usr/sbin/ethtool -K >> eth0 tx off" after bring it up. >> > You don''t have problem with UDP ? > Have you DNS or DHCP services running in your VMs ?Maybe I have a problem with UDP, but even bigger problem with TCP, as pings are not working (with IP address); not even arpings are working, even with very small packet sizes.> For my own I loose UDP traffic If I try to setup one vlan per bridge > in the DOM0 > To resolve this I must setup VLAN in each VM > > Have you test bonding + vlans ?No, I didn''t try bonding with Xen yet. -- Tomasz Chmielewski wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
El jue, 31-08-2006 a las 09:33 +0200, Tomasz Chmielewski escribió:> >> > > You don''t have problem with UDP ? > > Have you DNS or DHCP services running in your VMs ? > > Maybe I have a problem with UDP, but even bigger problem with TCP, as > pings are not working (with IP address); not even arpings are working, > even with very small packet sizes. >I don''t have any problem with UDP neither TCP. The problem with networking is solved running "/usr/sbin/ethtool -K>> eth0 tx off" in domU-- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información _o) y las Comunicaciones Aplicadas (ATICA) / \\ um.es/atica _(___V Tfo: 968367590 Fax: 968398337 _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Aug-31 08:22 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Angel L. Mateo wrote:> Hello, > > I have Xen running with domUs in different VLAN''s. I have xen on a > debian sarge server. My configuration is: > > * In dom0, I have a bridge for each vlan (different than the native vlan > of interface eth0, that I don''t need to tag). For each vlan the > configuration is (for vlan 302): > > vconfig add eth0 302 > /etc/xen/scripts/network-bridge start netdev=eth0.302 bridge=xen-br302 > antispoof=no > /sbin/ifconfig eth0.302 up > > * In the domU the configuration I have is: > > vif = [ ''bridge=xen-br302'' ] > > * The only problem is that un domU I have to run "/usr/sbin/ethtool -K > eth0 tx off" after bring it up.I this configuration, do you run a "normal" device (like eth0) in domU, or is it also a VLAN (eth0.302 in dom0, but also in domU)? -- Tomasz Chmielewski wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Sébastien CRAMATTE
2006-Aug-31 08:28 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Tomasz Chmielewski a écrit :> Angel L. Mateo wrote: >> Hello, >> >> I have Xen running with domUs in different VLAN''s. I have xen on a >> debian sarge server. My configuration is: >> >> * In dom0, I have a bridge for each vlan (different than the native vlan >> of interface eth0, that I don''t need to tag). For each vlan the >> configuration is (for vlan 302): >> >> vconfig add eth0 302 >> /etc/xen/scripts/network-bridge start netdev=eth0.302 bridge=xen-br302 >> antispoof=no >> /sbin/ifconfig eth0.302 up >> >> * In the domU the configuration I have is: >> >> vif = [ ''bridge=xen-br302'' ] >> >> * The only problem is that un domU I have to run "/usr/sbin/ethtool -K >> eth0 tx off" after bring it up. > > I this configuration, do you run a "normal" device (like eth0) in > domU, or is it also a VLAN (eth0.302 in dom0, but also in domU)? > >I believe that is one or other ... If you setup vlan per bridge in dom0 you just mount eth0 in domU else you setup just the default bridge in dom0 and VLANs in each domU Look at this blog, it''s quite well explained ... felipe-alfaro.org/blog/category/xen Regards _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Tomasz Chmielewski
2006-Aug-31 08:36 UTC
Re: [Xen-users] can''t get multiple VLANs to work
Sébastien CRAMATTE wrote:> Tomasz Chmielewski a écrit : >> Angel L. Mateo wrote: >>> Hello, >>> >>> I have Xen running with domUs in different VLAN''s. I have xen on a >>> debian sarge server. My configuration is: >>> >>> * In dom0, I have a bridge for each vlan (different than the native vlan >>> of interface eth0, that I don''t need to tag). For each vlan the >>> configuration is (for vlan 302): >>> >>> vconfig add eth0 302 >>> /etc/xen/scripts/network-bridge start netdev=eth0.302 bridge=xen-br302 >>> antispoof=no >>> /sbin/ifconfig eth0.302 up >>> >>> * In the domU the configuration I have is: >>> >>> vif = [ ''bridge=xen-br302'' ] >>> >>> * The only problem is that un domU I have to run "/usr/sbin/ethtool -K >>> eth0 tx off" after bring it up. >> >> I this configuration, do you run a "normal" device (like eth0) in >> domU, or is it also a VLAN (eth0.302 in dom0, but also in domU)? >> >> > I believe that is one or other ... > If you setup vlan per bridge in dom0 you just mount eth0 in domU > else you setup just the default bridge in dom0 and VLANs in each domU > > Look at this blog, it''s quite well explained ... > felipe-alfaro.org/blog/category/xenYes, I''ve read it. 1. For me it works if: - dom0 has one bridge, eth0 attached to it - VLAN is made *only* in domUs 2. It doesn''t work if: - dom0 has multiple bridges, with different VLANs attached to it - domU has a normal network setup (no VLANs configured) In the case when it doesn''t work, domU doesn''t receive any packets - "tcpdump -i eth0" shows only packets that domU wants to send, but nothing else. Only domU attached to "default bridge" (xenbr0 + eth0) has network connectivity. -- Tomasz Chmielewski wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users
Just to throw my $0.02 in... I have servers setup with vlans and bridging, in a slightly different setup. I have four NICs in my servers. Two are management network dom0 only. The other two are for the domU vlans. So, eth0/1 are just on one vlan, nothing special. eth2/3 are on multiple vlans. For each vlan, I setup the eth2.vlanid, eth3.vlanid. I create bridges for each vlan (make sure to enable STP, I missed that once...not fun). Attach both eth2.vlanid and eth3.vlanid to their respective bridges for each vlan. The trick for me was to make sure and run ''ip link set dev up'' on all the vlan interfaces AND the bridges. I have no IP configured for dom0 on any of the domU VLANs. In the domU configs, just specify the bridge to attach to, nothing fancy to configure in the domU. I didn''t need the bandwidth of bonding (plus I''m cabled to separate switches for redundancy), and with both interfaces and STP enabled I should get interface redundancy, though I haven''t gone over and pulled the plug on a NIC to test it yet. I''m not using the standard xen networking scripts, as it was simpler for me to just write one that created all the interfaces/bridges. I can share it if anyone is interested, it is a modified version of a script I came across (probably off this list). -- Jason _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com lists.xensource.com/xen-users