Xen users - Jul 2012 - Xen networking experiment (with custom scripts and OpenVSwitch)

Hello everyone,

Recently I have been testing my customized Xen 4.2 networking setup. It
works pretty good and I would like to share it with anyone who is
interested.

The relevant files can be found here:
https://github.com/slacks42/xenscripts

Benefit: configure the Xen related networking devices with one
understandable bash script.

Please note that this is all still work in progress. For example, some
logging entries should be deleted or modified, and some routines could
be cleaned up. Still, I think it''s nice enough to share.

Description of the files:

xen.conf is basically a copy of /etc/xen/hotplugpath.sh and should be
put in /etc/xen/.

udev/xen-backend.rules is a modified version of the file supplied by
Xen. Line 6-9 take care of creation and deletion of
networking interfaces. As you can see, upon creation/deletion of a
network interface, the file /etc/xen/scripts/xennet is called. This
file can be found on github in the scripts directory.

Xennet is a replacement of vif-bridge. In the old scenario, udev
calls vif-setup, vif-setup calls vif-bridge, vif-bridge calls
vif-common and numerous other scripts. With xennet, I wanted to have one
script to take care of the networking. So xennet requires no other files
from /etc/xen/scripts.

As you can see xennet uses bash. It is not POSIX compliant but works
fine with bash. This takes us to the second millennium ;-) 

Here is a rundown of xennet:

Line 4 takes care of all error output from the script. This is a handy
way to debug, especially if you alter line 1 to ''#!/bin/bash
-x'' so
that you can see exactly what the script does.

Then on line 8, the $unique variable is set to a random 6 character
wide string, a ''cookie'' of sorts, this is used as a log prefix
so that
you can see which particular instance of the script does what.

Then a couple of functions are initialized. The checklog() and
logmsg() functions take care of logging at the requested loglevel (set
in xen.conf). This is different from line 4; logmsg is a function that
logs to the console, syslog, or a file (xen.conf).

The sigerr(), fatal(), success(), xenstore_read_default(),
findCommand() and evalVariables() functions are modified versions of the
same functions in Xen''s xen-script-common.sh file.

I added line 191-198 for debugging, rather than calling the actual
commands they fake their execution and only add a log entry. That''s why
they are commented out.

The actual routine starts at line 200. Line 201 sets $command to
online/offline/add/remove depending on how the script was called from
xen-backend.rules. evalVariables (line 202) searches for arguments with
an ''='', like ''foo=bar'', and sets those
variables accordingly (like
$foo == ''bar''). This is a nice trick I found in
xen-script-common.sh.

On to line 220-292. This searches for the vifname and bridge name. If
$command is ''offline'' or ''remove'' then I
found that it does not know
the vifname so it needs a way to find that. In all cases $vifname is
set to the requested vifname. Openvswitch does not require a bridge
name if you remove a device. So $bridge is not required with
''offline''
or ''remove''.

Line 297-337 adds or removes the vif from the switch. With openvswitch
this can be a "fake bridge", i.e. a VLAN tagged bridge, or an
unmanaged
switch. One could easily replace the ovs-vswitch commands with brctl
commands if desired, I *think*.

Xen 4.2, when used with xl, does not setup or change your networking
(as we saw with older Xen and network-bridge). So you need to do that
yourself which is a much better idea imho anyway. In my case, my init
scripts start up openvswitch when the system boots and my custom
networking script creates the relevant switches and interfaces and
configures those.

Finally -- openvswitch can have a lot of messy output that can fill up
your syslog files. syslog-ng.conf is something I use to limit
openvswitch''s output to /var/log/openvswitch.log.

Note that I seem to use the word ''switch'' and
''bridge'' while I am
talking about the same thing. Don''t let this confuse you.

Comments are appreciated!

Mark