Xen users - Apr 2008 - How does one create a bridge interfaces in dom0 without IPs?

I have a Debian Etch box running with Linux kernel 2.6.18 and Xen 3.0.3.  
On that box, in dom0, I have three active interfaces, eth0, eth1, and 
eth2.  However, only eth2 has an IP (this is the management interface).

eth0 and eth1 are simply "up" ("iface eth0 inet manual" 
in /etc/network/interfaces).

I have a custom network-bridge script with the following:
#!/bin/sh
/etc/xen/scripts/network-bridge $1 vifnum=0 bridge=xenbr0 netdev=eth0
/etc/xen/scripts/network-bridge $1 vifnum=1 bridge=xenbr1 netdev=eth1

which creates xenbr0 and xenbr1 at boot, attached to eth0 and eth1 
respectively.

My domUs are configured to use xenbr0 or xenbr1.  Everything works great!  
The physical interfaces, the vif0.X interfaces, and the bridge interfaces 
all have the same numbers and match up nicely (eth0, peth0, vif0.0, 
xenbr0 all link together).


I have an Ubuntu Hardy box running with Linux kernel 2.6.24 and Xen 3.2, 
and I cannot replicate this setup.  The included network-bridge script no 
longer accepts any parameters except "start/stop" and doesn''t
seem to
create bridges for interfaces without IPs in dom0.

How can I replicate the bridge setup that I have on my Xen 3.0 box, where 
bridges are created at boot on all the interfaces that are marked
"up"?

I want eth0 to be the management interface with an IP in dom0 only.  And I 
want eth3 through eth6 to be bridged interfaces with no IPs in dom0, just 
up and running so I can attach the different domUs to the different 
bridges.  Ideally, I''d also like to name the bridges something other
than
ethX.

I''ve tried using the included network-bridge script, but it
doesn''t seem
to work on interfaces without IPs, and it (or xend) is not linking the 
vif0.X numbers with the interfaces numbers (vif0.0 is linked to eth3 for 
example).

I''ve tried using a wrapper around network-bridge that passes
parameters,
and another that exports env variables.  No go.

I''ve tried creating the bridges automatically via
/etc/network/interfaces,
and manually.  I can get either the domUs to connect or the dom0 to 
connect to the network, but not both at the same time.

What am I missing?  How does one create bridges for the domUs to use 
without wasting a bunch of IPs for the dom0?

-- 
Freddie Cash
fjwcash@gmail.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On April 9, 2008 10:04 am Freddie Cash wrote:
> I have an Ubuntu Hardy box running with Linux kernel 2.6.24 and Xen
> 3.2, and I cannot replicate this setup.  The included network-bridge
> script no longer accepts any parameters except "start/stop" and
doesn''t
> seem to create bridges for interfaces without IPs in dom0.
>
> How can I replicate the bridge setup that I have on my Xen 3.0 box,
> where bridges are created at boot on all the interfaces that are marked
> "up"?
What a royal pain to get bridging on multiple interfaces (not starting 
with eth0) working on Xen 3.2!  (At least in the version included in 
Ubuntu Hardy Heron.)

Why is it that the default network-bridge script assumes you only have 1 
ethernet interface, that it is called eth0, that it has an active IP, and 
that you want the bridge device named after the interface?

Why can''t one pass parameters into it, like the virtual interface to
use
(vifnum=), the physical interface to use (netdev=), and the name of the 
bridge to create (bridge=)?  Like the Xen 3.0 script does.

In order to get things working like I have in Xen 3.0, I had to:
  - configure udev in dom0 to rename all my interfaces to make my 
quad-port gigabit NIC use eth0-eth3
  - configure /etc/network/interfaces to assign bogus 127.0.0.x/24 IPs to 
eth0-eth3 (and bring up eth6 using DHCP for the management interface)
  - write a wrapper script to call network-bridge 4 times with no 
parameters (since it doesn''t use them anymore)
  - re-write all my Xen 3.0 config files to use bridge=ethX instead of 
bridge=xenbrX
  - purchase a toupee to cover up the bald spot I got tearing my hair out 
over the past three days trying to get this to work
  - plaster over the holes in the drywall from banging my head against the 
wall due to the general lack of documentation for all things Xen

Anyone with enough sh-foo willing to write a more usable, configurable 
network-bridge script that works the way the Xen 3.0 one does (dropping 
the 3.0 script in place or even the scripts/ directory doesn''t work
with
Xen 3.2)?  Something that will correctly take vifnum=, netdev=, and 
bridge= options, and use them accordingly?  Something that can be used by 
those of us that have beefy servers with multiple NICs that we want to 
configure using multiple bridges.

Or be willing to explain to me how the network-script works, such that I 
could re-write it to accept and *use* those options?  (I can''t read the
current script well enough to follow what it is doing, and my attempts to 
write a custom one using the "ip", "brctl", and what not
commands have
all failed horribly.)

-- 
Freddie Cash
fjwcash@gmail.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Freddie Cash wrote:
> Anyone with enough sh-foo willing to write a more usable, configurable 
> network-bridge script that works the way the Xen 3.0 one does (dropping 
> the 3.0 script in place or even the scripts/ directory doesn''t
work with
> Xen 3.2)?  Something that will correctly take vifnum=, netdev=, and 
> bridge= options, and use them accordingly?  Something that can be used by 
> those of us that have beefy servers with multiple NICs that we want to 
> configure using multiple bridges.
> 
> Or be willing to explain to me how the network-script works, such that I 
> could re-write it to accept and *use* those options?  (I can''t
read the
> current script well enough to follow what it is doing, and my attempts to 
> write a custom one using the "ip", "brctl", and what
not commands have
> all failed horribly.)
Hi Freddie,

I don''t know if my shell fu is good enough, but I have multiple bridges
working.
I posted about it quite recently:
<http://article.gmane.org/gmane.comp.emulators.xen.user/33926>

In short, I created a custom network script:

# cat /etc/xen/scripts/my-network-script
#!/bin/sh
dir=$(dirname "$0")
"$dir/network-bridge" "$@" vifnum=0
"$dir/network-bridge" "$@" vifnum=1

... which is then used by xen:

# egrep -v "^(#|$)" /etc/xen/xend-config.sxp
(network-script my-network-script)
(vif-script vif-bridge)
(dom0-min-mem 196)
(dom0-cpus 0)

.. and in the domU:s I just use "bridge=xenbr0" or
"bridge=xenbr1". Very simple.
  No requirement to assign IP addresses to the dom0 interfaces.

Good luck!
/Martin Leben


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Apr 9, 2008 at 4:16 PM, Martin Leben <ml060223@leben.nu>
wrote:
> Freddie Cash wrote:
>
> > Anyone with enough sh-foo willing to write a more usable, configurable
> network-bridge script that works the way the Xen 3.0 one does (dropping the
> 3.0 script in place or even the scripts/ directory doesn''t work
with Xen
> 3.2)?  Something that will correctly take vifnum=, netdev=, and bridge>
options, and use them accordingly?  Something that can be used by those of
> us that have beefy servers with multiple NICs that we want to configure
> using multiple bridges.
> >
> > Or be willing to explain to me how the network-script works, such that
I
> could re-write it to accept and *use* those options?  (I can''t
read the
> current script well enough to follow what it is doing, and my attempts to
> write a custom one using the "ip", "brctl", and what
not commands have all
> failed horribly.)
>  I don''t know if my shell fu is good enough, but I have multiple
bridges
> working. I posted about it quite recently:
>  <http://article.gmane.org/gmane.comp.emulators.xen.user/33926>
>
>  In short, I created a custom network script:
Doesn''t work with Xen 3.2 on Ubuntu Hardy.  The network-bridge script
that comes with Xen 3.2 on Ubunut Hardy does not accept or use any
parameters beyond "start".  No vifnum, no netdev, no bridge, nothing.
It doesn''t work.

And it doesn''t work if the interfaces don''t have IP addresses
assigned to them.

And the order that network cables are plugged into the system after
boot makes a difference (the vif0.X devices are created based on the
order that the cables are connected and the link detected ... and not
on the interface name/number).  So if you connect two NICs to two
separate switches, and the switches power on in a different order,
your vif0.X are numbered differently.  Which means the bridges are
configured differently.

On a system with 7 NIC ports, this is a hassle and a configuration
nightmare (using Xen 3.2 on Ubuntu Hardy).

On my other system, with Xen 3.0 on Debian Etch, these things don''t
happen.  But the Etch kernels don''t support the quad-port NIC.
>  .. and in the domU:s I just use "bridge=xenbr0" or
"bridge=xenbr1". Very
> simple.  No requirement to assign IP addresses to the dom0 interfaces.
Except it doesn''t work that way on Xen 3.2 on Ubuntu Hardy.  :(

-- 
Freddie Cash
fjwcash@gmail.com

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users