Xen devel - Feb 2004 - RE: Creating a local network within the GuestOS and r outing to an ext ernal network

Ian:

How bad is the inter-VM performance?  Are we talking about sub 10 Mbps or
just not blazing given that it is really a memory-net?

Is there any reason you can think of that IPv6 won''t work with the
current
Xen network support?  We are going to use IPv6 (at least for off-box
communications).  I am still working out the details of if/how to support
QoS for applications that go off box.  One option is an application proxy on
one domain and let it fully own the network interface, but that ends up
being a lot of work for an essentially bent-pipe.  I suppose I could do some
NAT over IPv6, .... Using the 169.254/16 for communications to the NAT point
(does that have to be Domain0?) and assume that there is enough internal
bandwidth (hence the above question) to avoid major performance issues until
we hit the IPv6 interface (which is, by the way, a relatively slow link, < 1
Mb/s).

Charlie

Charlie Woloszynski
Innovative Concepts Inc.
703-893-2007 x506 
charles.woloszynski@innocon.com

-----Original Message-----
From: Ian Pratt [mailto:Ian.Pratt@cl.cam.ac.uk] 
Sent: Thursday, February 19, 2004 11:46 AM
To: Woloszynski, Charles
Cc: ''xen-devel@lists.sourceforge.net''; Ian.Pratt@cl.cam.ac.uk
Subject: Re: [Xen-devel] Creating a local network within the GuestOS and
routing to an ext ernal network 

> Is it possible to create a local network within a set of Xenolinux 
> machines and then allow one of them to route to the external network 
> over an actual network interface?
> 
> I am working on using the TC (traffic control) toolset to do QoS 
> enforcement and planning on using Xen/Xenolinux.  Since the Xen packet 
> interface simply shares access across the system, I wanted to use one 
> GuestOS as the router and have it enforce the queuing policies and let 
> the other GuestOSes talk to the external network through this GuestOS.  
> I was wondering if anyone has done anything like this?  Can we create 
> an in-memory network for the local subnet?
Yes, the 169.254/16 network is entirely intra-machine. 

There are a couple of bugs that mean that the performance of inter VM
communication isn''t what it should be (too many virtual interrupts),
but it
works. This will be fixed in somewhere in the 1.3/1.4 series when the new IO
stuff goes in.

See the xen_nat_enable script (which should actually be called
"dom0_nat_enable").


Ian

> How bad is the inter-VM performance?  Are we talking about sub 10 Mbps or
> just not blazing given that it is really a memory-net?
Last time I measured I think we were only getting about 800Mb/s
with 1500 byte packets (less than we get going over a LAN !).

We should be getting multiple Gb/s.
> Is there any reason you can think of that IPv6 won''t work with the
current
> Xen network support?  We are going to use IPv6 (at least for off-box
> communications).  
The VFR in the mainline tree only supports IPv4. However, Mike
Wray has developed a L2 network switch support for Xen, which
will hopefully be checked in soon. One side effect of L2 Ethernet
MAC switching support would be that I expect IPv6 will just work
as Xen is no longer needing to look deeper than the Ethernet
header. (Xen won''t be able to enforce IP firewalling for you, but
if you''re passing it through a domain for NAT you can do it there
anyhow.)
> I am still working out the details of if/how to support
> QoS for applications that go off box.  One option is an application proxy
on
> one domain and let it fully own the network interface, but that ends up
> being a lot of work for an essentially bent-pipe.  I suppose I could do
some
> NAT over IPv6, .... Using the 169.254/16 for communications to the NAT
point
> (does that have to be Domain0?) 
No, any domain can be the NAT point.
> and assume that there is enough internal
> bandwidth (hence the above question) to avoid major performance issues
until
> we hit the IPv6 interface (which is, by the way, a relatively slow link,
< 1
> Mb/s).
All this will be much easier in the new IO world...

Ian


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

" Xen won''t be able to enforce IP firewalling for you, but

But this is a feature!   We want that external IP layer enforcement.
For our purposes, full layer 2 network access by any domain is a bad
thing.    

Will this layer 2 switch supplant the current code, or be an addition?



-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> Will this layer 2 switch supplant the current code, or be an addition?
An addition. Xen is being used by a lot of different groups in a
lot of different ways.
> " Xen won''t be able to enforce IP firewalling for you, but
> 
> But this is a feature!   We want that external IP layer enforcement.
> For our purposes, full layer 2 network access by any domain is a bad
> thing.
Mike''s code also has a concept of ''virtual network'';
only hosts
on the same virtual network are visible to each other.

It''s probably a fairly simple modification to only allow hosts to
talk to e.g. domain 0, then implement IPv6 firewalling using
iptables in the domain.

The "correct" solution would be to implement an IPv6-capable VFR
in Xen (which is actually very easy), but since this code is all
going to change in the new IO world I wouldn''t advise embarking
on such a project right now unless you need it urgently.

Ian


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel