Xen users - Jan 2012 - Some VLAN ideas for discussion

Hi all,

a question to the Xen networking geeks.

Currently my hosts are running with LACP bonded VLAN trunks that are
then broken into bridges in the dom0.
This is setup quite similar to the oracle wiki article.

I would like to build something that includes:
    - quite a few VLANs
    - multiple VMs that need access to some of them
    - multiple VMs that need access to almost all of them (routers,
vpn portals... things)

This isn''t easily done with Xen so far as if you don''t want to
terminate the VLANs in a bridge in dom0 but also want to have virtual
machines doing routing.
Dedicating a physical nic to this cause (or a bond of course) isn''t so
great either, if, for example your host only has two interfaces :)

Some ideas I''m looking at:
    - NICs that support multiple PCI functions (Intel 1000PT,
similar), give virtual functions to each of the routers.
    - Buy Solarflare NICs and use their netback driver (might work. I
don''t know it; wish more people would be using them so they end up
soldered onto mainboards)
    - OpenVSwitch, I''m not sure if it is able to pass like 100 VLANs
into a domU? I don''t have experience with it yet, sadly. It being a
softswitch it might be able to run LACP over two nics on it''s own,
outside of the kernel. How about MSTP and native QinQ? *grin* Yes,
there''s a few interesting points there.

Attaching a number of virtual nics that each carry a single VLAN is
not acceptable (management overhead that would make the Solarflares
look *cheap* and doesn''t scale anyway)

On the other hand, I''ve had the feeling I''m missing something.
For one, how about QinQ, how about L2TPv3 -
the standard linux bridge could not mess up^W^W strip away VLAN tags
         - that it can''t see due to encapsulation in a proper tunnel
that just passes the bridge as IP and is extracted in dom0 and domUs
         - that have a specified outer VLAN type (note that some
postings about linux qinq use the same ethertype, that is not qinq,
that''s just vlan injection ;))


My testbed is currently looking like this, if anyone considers reproducing:

a vm named "start"
  a xen host
    a journey through the internet
  a xen host
a vm named "goal"

The next step is to add two alpine linux router VMs (for failover
options) in each host and then I want to somehow build the
interconnection - ideally not by passing a bridge(or softswitch) in
the xen host all too often.

Any comments?
(besides "you have too much time" - I don''t ;)


Greetings,
Florian

-- 
the purpose of libvirt is to provide an abstraction layer hiding all
xen features added since 2006 until they were finally understood and
copied by the kvm devs.

On 10.01.2012 18:12, Florian Heigl wrote:
> Hi all,
>
> a question to the Xen networking geeks.
> ...
> Some ideas I''m looking at:
>     - NICs that support multiple PCI functions (Intel 1000PT,
> similar), give virtual functions to each of the routers.
>     - Buy Solarflare NICs and use their netback driver (might work. I
> don''t know it; wish more people would be using them so they end up
> soldered onto mainboards)
>     - OpenVSwitch, I''m not sure if it is able to pass like 100
VLANs
> into a domU? I don''t have experience with it yet, sadly. It being
a
> softswitch it might be able to run LACP over two nics on it''s own,
> outside of the kernel.
I''ve been using OpenVSwitch for my bridges in Xen for quite a while, 
almost a year.  I run them at the house, but with no bells or whistles.  
I do run them at work and use them for my LACP trunks to my real 
switches, so I know that will work.  You can also setup your vifs to be 
trunk ports and then add the sub interfaces using ip link inside the VM, 
I do that also.  From reading over their technical documentation and 
running performance tests in my environment, I doubt that you''d have 
much difficulty running 100s of VLANs through them.  I have been able to 
saturate a 10Gbe adapter using it with iSCSI traffic inside 20 VMs, so 
it can handle the through-put.  There are some issues, mainly none of 
the Xen hotplug scripts and udev rules work well with it, so I''ve had
to
rewrite them and add udev rules to get vifs added and removed from an 
OVS bridge, once you get past that it''s smooth sailing.  With all the 
features that OVS supports, GRE, SPAN/RSPAN, and openflow/NOX, you can 
really do some interesting things with it.  QinQ is not supported yet, 
but probably will be in the future.  Hope this helps.

> How about MSTP and native QinQ? *grin* Yes,
> there''s a few interesting points there.
>
> Attaching a number of virtual nics that each carry a single VLAN is
> not acceptable (management overhead that would make the Solarflares
> look *cheap* and doesn''t scale anyway)
>
> On the other hand, I''ve had the feeling I''m missing
something.
> For one, how about QinQ, how about L2TPv3 -
> the standard linux bridge could not mess up^W^W strip away VLAN tags
>          - that it can''t see due to encapsulation in a proper
tunnel
> that just passes the bridge as IP and is extracted in dom0 and domUs
>          - that have a specified outer VLAN type (note that some
> postings about linux qinq use the same ethertype, that is not qinq,
> that''s just vlan injection ;))
>
>
> My testbed is currently looking like this, if anyone considers 
> reproducing:
>
> a vm named "start"
>   a xen host
>     a journey through the internet
>   a xen host
> a vm named "goal"
>
> The next step is to add two alpine linux router VMs (for failover
> options) in each host and then I want to somehow build the
> interconnection - ideally not by passing a bridge(or softswitch) in
> the xen host all too often.
>
> Any comments?
> (besides "you have too much time" - I don''t ;)
>
>
> Greetings,
> Florian
Mike