Xen devel - Aug 2010 - Patch fixing vlan handling when network-bridge script is used

We had problem using tagged vlans inside domU. Testing shows that this is caused
by changing network interfaces in network-bridge script. Following patch should
allow to use vlans in guest.

Testing scenario:
configure vlan network (e.g. vlan 1 on eth0 => eth0.1)
start xend
create domU and configure same vlan inside it

You can see that vlan is still <interface>.<vlan_id>. You can reach
dom0 but not domU. After patch apply vlan is changed to
<bridge>.<vlan_id> after xend start and you can reach both dom0 and
domu.

Patch:
------
diff -r 9f49667fec71 tools/hotplug/Linux/network-bridge
--- a/tools/hotplug/Linux/network-bridge	Fri Jul 30 15:22:39 2010 +0100
+++ b/tools/hotplug/Linux/network-bridge	Wed Aug 18 13:33:05 2010 +0200
@@ -119,6 +119,36 @@
     fi
 }
 
+handle_vlan_start() {
+    for vlan in `ls /proc/net/vlan/${netdev}* 2>/dev/null`; do
+        vlan_id=$(echo ${vlan} | sed -e "s/.*${netdev}.\(.*\)/\1/")
+        vlan_if=${netdev}.${vlan_id}
+
+        get_ip_info ${vlan_if}
+
+        vconfig rem ${vlan_if}
+        sleep 1
+        vconfig add ${bridge} ${vlan_id}
+
+        do_ifup ${bridge}.${vlan_id}
+    done
+}
+
+handle_vlan_stop() {
+    for vlan in `ls /proc/net/vlan/${bridge}* 2>/dev/null`; do
+        vlan_id=$(echo ${vlan} | sed -e "s/.*${bridge}.\(.*\)/\1/")
+        vlan_if=${bridge}.${vlan_id}
+
+        get_ip_info ${vlan_if}
+
+        vconfig rem ${vlan_if}
+        sleep 1
+        vconfig add ${netdev} ${vlan_id}
+
+        do_ifup ${netdev}.${vlan_id}
+    done
+}
+
 # Usage: transfer_addrs src dst
 # Copy all IP addresses (including aliases) from device $src to device $dst.
 transfer_addrs () {
@@ -254,6 +284,8 @@
     add_to_bridge2 ${bridge} ${pdev}
     do_ifup ${bridge}
 
+    handle_vlan_start
+
     if [ ${antispoof} = ''yes'' ] ; then
 	antispoofing
     fi
@@ -289,6 +321,8 @@
     ip link set ${pdev} name ${netdev}
     do_ifup ${netdev}
 
+    handle_vlan_stop
+
     brctl delbr ${tdev}
 
     release_lock "network-bridge"
-- 
Miroslav Rezanina
Software Engineer - Virtualization Team - XEN kernel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Miroslav Rezanina writes ("[Xen-devel] Patch fixing vlan handling when
network-bridge script is used"):
> We had problem using tagged vlans inside domU. Testing shows that
> this is caused by changing network interfaces in network-bridge
> script. Following patch should allow to use vlans in guest.
Thanks for reporting this.

This is just another way in which the network-bridge approach, of
trying to transfer all the details from one interface to another,
won''t work well.

I think you would be much better served if you just used your
operating system''s features (/etc/network/interfaces on Debian) to set
up the bridge and call it xenbr0, and disabled network-bridge
entirely.

Rather than apply your patch, I would prefer to change network-bridge
so that it explicitly fails when vlans are detected and tells you to
do it the other way.

Anyone else have an opinion /

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> Miroslav Rezanina writes ("[Xen-devel] Patch fixing vlan handling when
> network-bridge script is used"):
> > We had problem using tagged vlans inside domU. Testing shows that
> > this is caused by changing network interfaces in network-bridge
> > script. Following patch should allow to use vlans in guest.
> 
> Thanks for reporting this.
> 
> This is just another way in which the network-bridge approach, of
> trying to transfer all the details from one interface to another,
> won''t work well.
> 
> I think you would be much better served if you just used your
> operating system''s features (/etc/network/interfaces on Debian) to
set
> up the bridge and call it xenbr0, and disabled network-bridge
> entirely.
> 
> Rather than apply your patch, I would prefer to change network-bridge
> so that it explicitly fails when vlans are detected and tells you to
> do it the other way.
> 
> Anyone else have an opinion /
> 
I always set up networking in /etc/network/interfaces. I think that xend
doing it for you is nice but it breaks so easily that I wonder if it''s
worth it at all. IMO, the default should be that xend doesn''t touch the
bridge at all, but leave the option to do so in the config but commented
out.

Of course, you don''t hear from the people that are all using it with no
problem at all...

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

James Harper writes ("RE: [Xen-devel] Patch fixing vlan handling when
network-bridge scriptis used"):
> I always set up networking in /etc/network/interfaces. I think that xend
> doing it for you is nice but it breaks so easily that I wonder if
it''s
> worth it at all. IMO, the default should be that xend doesn''t
touch the
> bridge at all, but leave the option to do so in the config but commented
> out.
The new xl toolset in xen-unstable doesn''t set up the bridge for you.
(I think the one in 4.0-testing might; the scripts there are
confusingly halfway between 4.0.0 and xen-unstable following the last
series of libxl backports.)

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thursday 19 August 2010 06:20:47 James Harper wrote:
> > Miroslav Rezanina writes ("[Xen-devel] Patch fixing vlan handling
when
> > 
> > network-bridge script is used"):
> > > We had problem using tagged vlans inside domU. Testing shows that
> > > this is caused by changing network interfaces in network-bridge
> > > script. Following patch should allow to use vlans in guest.
> > 
> > Thanks for reporting this.
> > 
> > This is just another way in which the network-bridge approach, of
> > trying to transfer all the details from one interface to another,
> > won''t work well.
> > 
> > I think you would be much better served if you just used your
> > operating system''s features (/etc/network/interfaces on
Debian) to set
> > up the bridge and call it xenbr0, and disabled network-bridge
> > entirely.
> > 
> > Rather than apply your patch, I would prefer to change network-bridge
> > so that it explicitly fails when vlans are detected and tells you to
> > do it the other way.
> > 
> > Anyone else have an opinion /
> 
> I always set up networking in /etc/network/interfaces. I think that xend
> doing it for you is nice but it breaks so easily that I wonder if
it''s
> worth it at all. IMO, the default should be that xend doesn''t
touch the
> bridge at all, but leave the option to do so in the config but commented
> out.
> 
> Of course, you don''t hear from the people that are all using it
with no
> problem at all...
Well, I did have a few problems with the network-script, too, and then decided 
to drop it and do it all in network/interfaces. I didn''t report it
because the
main problem for me was initialization order and there is nothing that I found 
that could be done to fix that script - when xend starts it''s already
too late.

But I had to add the "ethtool -K $vif tx off" to vif-bridge to fix the
checksums
error. Maybe this could be added by default with a check for ethtool?

cheers,
-- 
Michael Brade;                                              KDE Developer
  |-mail: echo brade !#|tr -d "c oh"|s\e\d
''s/e/\@/2;s/$/.org/;s/bra/k/2''
  °--web: http://www.behindkde.org/people/michaelb/

KDE 4: Beyond Your Expectations


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael Brade writes ("Re: [Xen-devel] Patch fixing vlan handling when
network-bridge scriptis used"):
> But I had to add the "ethtool -K $vif tx off" to vif-bridge to
fix the checksums
> error. Maybe this could be added by default with a check for ethtool?
Are we still having this bug ?  I tripped over this about 4 years ago
when I was just a user of Xen ...

Can you describe the setup in which you had to do the -K tx off ?

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thursday 19 August 2010 19:59:06 you wrote:
> Michael Brade writes ("Re: [Xen-devel] Patch fixing vlan handling when
network-bridge scriptis used"):
> > But I had to add the "ethtool -K $vif tx off" to vif-bridge
to fix the
> > checksums error. Maybe this could be added by default with a check for
> > ethtool?
> 
> Are we still having this bug ?  I tripped over this about 4 years ago
> when I was just a user of Xen ...
wow... I also thought this must have been fixed already but I didn''t
find a
message saying so, nor did I see a patch for it so I concluded ethtool was the 
way to go.
> Can you describe the setup in which you had to do the -K tx off ?
yup: 
  dom0:
    * running xen-testing from staging, kernel 2.6.32.19 with pvops
    * network interfaces: peth0, peth1, ppp0
    * bridges: eth0, eth1
    * local net is on bridge eth1 with domUs connected there
    * internet is on ppp0
  domU:
    * running PV-GRUB, kernel 2.6.32.19 as above, but stripped down to 
      the bare minimum
    * only one network interface, dhcp to dom0

relevant domU config:

kernel      = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra       = "(hd0)/boot/grub/menu.lst"
disk        = [ ''tap2:tapdisk:aio:/..../root.img,xvda1,w'', 
                ''tap2:tapdisk:aio:/..../var.img,xvda2,w'' ]

boot=''c''
sdl=0
vnc=1
vncunused=1
stdvga=1

vif         = [ ''mac=...., bridge=eth1'' ]


grub menu.lst:

title vmlinuz-2.6.32.19-domU
   root (hd0)
   kernel /boot/vmlinuz-2.6.32.19 console=hvc0 root=/dev/xvda1 ro

If I leave the -K tx off out then I won''t even get a dhcp connection in
domU.

And btw, vnc doesn''t work either, despite the vnc=1. There is no open
port, no
idea why yet. Only xm console is available. If I start a domU with HVM it 
works.

Here''s xm list --long:

(domain
    (domid 2)
    (cpu_weight 256)
    (cpu_cap 0)
    (on_crash restart)
    (uuid .....)
    (bootloader_args )
    (vcpus 1)
    (name .....)
    (on_poweroff destroy)
    (on_reboot restart)
    (cpus (()))
    (description )
    (bootloader )
    (maxmem 512)
    (memory 512)
    (shadow_memory 0)
    (features )
    (on_xend_start ignore)
    (on_xend_stop ignore)
    (start_time 1282234728.43)
    (cpu_time 16.594413607)
    (online_vcpus 1)
    (image
        (linux
            (kernel /usr/lib/xen/boot/pv-grub-x86_64.gz)
            (args ''(hd0)/boot/grub/menu.lst'')
            (superpages 0)
            (tsc_mode 0)
            (videoram 4)
            (pci ())
            (nomigrate 0)
            (notes)
        )
    )
    (status 2)
    (state -b----)
    (store_mfn 1815884)
    (console_mfn 1815883)
    (device
        (vif
            (bridge eth1)
            (mac ....)
            (script ''/etc/xen/scripts/vif-bridge bridge=eth1'')
            (uuid ....)
            (backend 0)
        )
    )
    (device
        (console
            (protocol vt100)
            (location 2)
            (uuid ....)
        )
    )
    (device
        (tap2
            (protocol x86_64-abi)
            (uuid ....)
            (bootable 1)
            (dev xvda1:disk)
            (uname
                tap:tapdisk:aio:/..../root.img
            )
            (mode w)
            (backend 0)
            (VDI )
        )
    )
    (device
        (tap2
            (protocol x86_64-abi)
            (uuid ....)
            (bootable 0)
            (dev xvda2:disk)
            (uname
                tap:tapdisk:aio:/..../var.img
            )
            (mode w)
            (backend 0)
            (VDI )
        )
    )
)

Hope this helps!

thanks,
-- 
Michael Brade;                                              KDE Developer
  |-mail: echo brade !#|tr -d "c oh"|s\e\d
''s/e/\@/2;s/$/.org/;s/bra/k/2''
  °--web: http://www.behindkde.org/people/michaelb/

KDE 4: Beyond Your Expectations


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 08/19/2010 08:59 AM, Michael Brade wrote:
> But I had to add the "ethtool -K $vif tx off" to vif-bridge to
fix the checksums
> error. Maybe this could be added by default with a check for ethtool?
This was a problem in pvops dom0 kernels a long time ago, but it has
been fixed for... maybe a year?  If you''re still having problems
without
the ethtool, please report them.

Thanks,
    J

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Friday 20 August 2010 01:11:18 Jeremy Fitzhardinge
wrote:
>  On 08/19/2010 08:59 AM, Michael Brade wrote:
> > But I had to add the "ethtool -K $vif tx off" to vif-bridge
to fix the
> > checksums error. Maybe this could be added by default with a check for
> > ethtool?
> 
> This was a problem in pvops dom0 kernels a long time ago, but it has
> been fixed for... maybe a year?  If you''re still having problems
without
> the ethtool, please report them.
I did in my last mail and now I also added it to Bugzilla. Do you have enough 
information in that entry or do I need to add other things as well?

Btw, it did work with HVM where the guest communicates over the tap1.0 
interface, it just doesn''t work with PV-guests on the vif1.0 interface.
What
is the difference between the two, why does HVM use one and PV the other?

cheers,
-- 
Michael Brade;                                              KDE Developer
  |-mail: echo brade !#|tr -d "c oh"|s\e\d
''s/e/\@/2;s/$/.org/;s/bra/k/2''
  °--web: http://www.behindkde.org/people/michaelb/

KDE 4: Beyond Your Expectations


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel