Xen users - Aug 2005 - DomU Bridged vs. Routed Networking?

Is there a performance difference between bridged and routed DomU networking?

Seems like most users opt for the bridged approach.  Perhaps it''s
because
it is easier to setup and it is the default setting.  For route, I spent 
several days to learn that /proc/sys/net/ipv4/conf/eth0/proxy_arp needs to 
be set to 0.  It is not (Debian Sarge).  It should probably be added to 
/etc/xen/scripts/network-route.

I also modified scripts/vif-route (using iptables) to forward only packets 
belonging to each domU''s IP address, thereby preventing domU''s
from using
IP addresses not assigned to them.  With bridge, I''d need to install 
etables - one extra program to install and learn.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Andy Lee wrote:
> Is there a performance difference between bridged and routed DomU 
> networking?
> 
> Seems like most users opt for the bridged approach.  Perhaps it''s 
> because it is easier to setup and it is the default setting.  For route, 
> I spent several days to learn that 
> /proc/sys/net/ipv4/conf/eth0/proxy_arp needs to be set to 0.  It is not 
> (Debian Sarge).  It should probably be added to 
> /etc/xen/scripts/network-route.
> 
> I also modified scripts/vif-route (using iptables) to forward only 
> packets belonging to each domU''s IP address, thereby preventing
domU''s
> from using IP addresses not assigned to them.  With bridge, I''d
need to
> install etables - one extra program to install and learn.
There is one other factor which some people have noticed
and pointed out on this list: the interface is in promiscuous mode
in the bridging scenario, which deteriorates performance.
Your mileage may vary.

thanks,
Nivedita


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > Seems like most users opt for the bridged approach.  Perhaps
it''s
> > because it is easier to setup and it is the default setting.  For 
> > route, I spent several days to learn that 
> > /proc/sys/net/ipv4/conf/eth0/proxy_arp needs to be set to 0.  It is 
> > not (Debian Sarge).  It should probably be added to 
> > /etc/xen/scripts/network-route.
> > 
> > I also modified scripts/vif-route (using iptables) to forward only 
> > packets belonging to each domU''s IP address, thereby 
> preventing domU''s 
> > from using IP addresses not assigned to them.  
Please can you post diffs and we''ll update the example scripts.
> > With bridge, I''d need 
> > to install etables - one extra program to install and learn.
That''s not actually true -- you can use iptables to do packet filtering
in bridge mode. You only need ebtables if you want to do matches on MAC
addrs. 
> There is one other factor which some people have noticed and 
> pointed out on this list: the interface is in promiscuous 
> mode in the bridging scenario, which deteriorates performance.
> Your mileage may vary.
All modern Ethernet networks are switched rather than shared media.
Putting the interface in promiscous mode will make NO difference to
performance unless you have lots of *multicast* traffic on your network
that this host isn''t interested in.

Ian

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ian Pratt wrote:
>>>Seems like most users opt for the bridged approach.  Perhaps
it''s
>>>because it is easier to setup and it is the default setting.  For 
>>>route, I spent several days to learn that 
>>>/proc/sys/net/ipv4/conf/eth0/proxy_arp needs to be set to 0.  It is 
>>>not (Debian Sarge).  It should probably be added to 
>>>/etc/xen/scripts/network-route.
>>>
>>>I also modified scripts/vif-route (using iptables) to forward only 
>>>packets belonging to each domU''s IP address, thereby 
>>
>>preventing domU''s 
>>
>>>from using IP addresses not assigned to them.  
> 
> 
> Please can you post diffs and we''ll update the example scripts.
There are network-nat and vif-nat scripts, but they need
a little tweaking. Would be good to merge Andy''s changes.
>>>With bridge, I''d need 
>>>to install etables - one extra program to install and learn.
> 
> 
> That''s not actually true -- you can use iptables to do packet
filtering
> in bridge mode. You only need ebtables if you want to do matches on MAC
> addrs. 
> 
> 
>>There is one other factor which some people have noticed and 
>>pointed out on this list: the interface is in promiscuous 
>>mode in the bridging scenario, which deteriorates performance.
>>Your mileage may vary.
> 
> 
> All modern Ethernet networks are switched rather than shared media.
> Putting the interface in promiscous mode will make NO difference to
> performance unless you have lots of *multicast* traffic on your network
> that this host isn''t interested in.
Yep, mostly multicast traffic, since that''s the only additional bucket
(unless you turn off icmp broadcast echo replies when not promiscuous)
but I measured this earlier this morning with a tcp netperf stream
and it was about 4% difference. Admittedly, today was an exception,
the network was getting hammered due to a misconfigured router,
congestion on the net due to the worm and some heavy multicast
traffic. I didn''t see this till now but it was logging a bunch
of those, which exacerbated the load on the system.

thanks,
Nivedita


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

...on Tue, Aug 16, 2005 at 11:47:16PM -0700, Andy Lee wrote:

 > Seems like most users opt for the bridged approach.  Perhaps it''s
because
 > it is easier to setup and it is the default setting.  For route, I spent 
 > several days to learn that /proc/sys/net/ipv4/conf/eth0/proxy_arp needs to
 > be set to 0.  It is not (Debian Sarge).  It should probably be added to 
 > /etc/xen/scripts/network-route.

Hm, I also just managed to set up a routed xenU network, 
after hitting probably almost any possible pitfall, but 
proxy_arp being set wrong was not amongst them (also 
Debian Sarge). It''s set for the vif''s here, but not for 
eth0.

Also, the routing setup seems to be documented nowhere 
(couldn''t find it, at least), except that the network 
scripts are being referenced in xend-config.sxp.

(And the NetBSD howto confused me by assigning an 
ip address to the bridge parameter for the vif in 
their example.)

Alex.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ian Pratt wrote:
 >
 > Please can you post diffs and we''ll update the example scripts.

Of course!  I still need a few days to finish the script (clean it up and 
add comments), then I''ll post the diffs.

Regards,
Andy


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users