Based on Waldi''s RFC at
http://lists.xen.org/archives/html/xen-devel/2012-09/msg00943.html
To use it set vif.default.script="vif-openvswitch" in /etc/xen/xl.conf
or use
script=vif-openvswitch in the vif configuration.
Appears to do the right thing for PV and HVM guests (including tap devices)
and with stubdomains.
In order to support VLAN tagging and trunking the "bridge" specified
in the
configuration can have a special syntax, that is:
BRIDGE_NAME[.VLAN][:TRUNK:TRUNK]
e.g.
- xenbr0.99
add the VIF to VLAN99 on xenbr0
- xenbr0:99:100:101
add the VIF to xenbr0 as a trunk port receiving VLANs 99, 100 & 101
Waldi, can you confirm if I have correctly reverse engineered the syntax from
the regexp please ;-)
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Bastian Blank <waldi@debian.org>
Cc: dev@openvswitch.org
---
v2: Correct syntax description in commitlog.
Remove erroneous xl.conf change.
Added Bastian''s S-o-b.
Agreed with Ben Pfaff that the Xen tree is a good home for this script.
---
tools/hotplug/Linux/Makefile | 1 +
tools/hotplug/Linux/vif-openvswitch | 99 +++++++++++++++++++++++++++++++++++
2 files changed, 100 insertions(+), 0 deletions(-)
create mode 100644 tools/hotplug/Linux/vif-openvswitch
diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
index 0605559..99bf87f 100644
--- a/tools/hotplug/Linux/Makefile
+++ b/tools/hotplug/Linux/Makefile
@@ -13,6 +13,7 @@ XENCOMMONS_SYSCONFIG = init.d/sysconfig.xencommons
XEN_SCRIPTS = network-bridge vif-bridge
XEN_SCRIPTS += network-route vif-route
XEN_SCRIPTS += network-nat vif-nat
+XEN_SCRIPTS += vif-openvswitch
XEN_SCRIPTS += vif2
XEN_SCRIPTS += vif-setup
XEN_SCRIPTS += block
diff --git a/tools/hotplug/Linux/vif-openvswitch
b/tools/hotplug/Linux/vif-openvswitch
new file mode 100644
index 0000000..53c2f50
--- /dev/null
+++ b/tools/hotplug/Linux/vif-openvswitch
@@ -0,0 +1,99 @@
+#!/bin/bash
+#===========================================================================+#
${XEN_SCRIPT_DIR}/vif-openvswitch
+#
+# Script for configuring a vif in openvswitch mode.
+# The hotplugging system will call this script if it is specified either in
+# the device configuration given to Xend, or the default Xend configuration
+# in ${XEN_CONFIG_DIR}/xend-config.sxp. If the script is specified in
+# neither of those places, then this script is the default.
+#
+# Usage:
+# vif-openvswitch (add|remove|online|offline)
+#
+# Environment vars:
+# vif vif interface name (required).
+# XENBUS_PATH path to this device''s details in the XenStore
(required).
+#
+# Read from the store:
+# bridge openvswitch to add the vif to (required).
+# ip list of IP networks for the vif, space-separated (optional).
+#
+# up:
+# Enslaves the vif interface to the bridge and adds iptables rules
+# for its ip addresses (if any).
+#
+# down:
+# Removes the vif interface from the bridge and removes the iptables
+# rules for its ip addresses (if any).
+#===========================================================================+
+dir=$(dirname "$0")
+. "$dir/vif-common.sh"
+
+openvswitch_external_id() {
+ local dev=$1
+ local key=$2
+ local value=$3
+
+ echo "-- set interface $dev
external-ids:\"$key\"=\"$value\""
+}
+
+openvswitch_external_id_all() {
+ local dev=$1
+ local frontend_id=$(xenstore_read "$XENBUS_PATH/frontend-id")
+ local vm_path=$(xenstore_read "/local/domain/${frontend_id}/vm")
+ local name=$(xenstore_read "${vm_path}/name")
+ openvswitch_external_id $dev "xen-vm-name" "$name"
+ local uuid=$(xenstore_read "${vm_path}/uuid")
+ openvswitch_external_id $dev "xen-vm-uuid" "$uuid"
+ local mac=$(xenstore_read "$XENBUS_PATH/mac")
+ openvswitch_external_id $dev "attached-mac" "$mac"
+}
+
+add_to_openvswitch () {
+ local dev=$1
+ local bridge="$(xenstore_read_default "$XENBUS_PATH/bridge"
"$bridge")"
+ local tag trunk
+
+ if [[ $bridge =~
^([^.:]+)(\.([[:digit:]]+))?(:([[:digit:]]+(:[[:digit:]]+)*))?$ ]]; then
+ bridge="${BASH_REMATCH[1]}"
+ tag="${BASH_REMATCH[3]}"
+ trunk="${BASH_REMATCH[5]//:/,}"
+ else
+ fatal "No valid bridge was specified"
+ fi
+
+ if [ $trunk ]; then
+ local trunk_arg="trunk=$trunk"
+ fi
+
+ if [ $tag ]; then
+ local tag_arg="tag=$tag"
+ fi
+
+ local vif_details="$(openvswitch_external_id_all $dev)"
+
+ ovs-vsctl --timeout=30 -- --if-exists del-port $dev -- add-port
"$bridge" $dev $tag_arg $trunk_arg $vif_details
+ ip link set $dev up
+}
+
+case "$command" in
+ add|online)
+ setup_virtual_bridge_port $dev
+ add_to_openvswitch $dev
+ ;;
+
+ offline)
+ ovs-vsctl --timeout=30 -- --if-exists del-port $dev
+ ;;
+esac
+
+if [ "$type_if" = vif ]; then
+ handle_iptable
+fi
+
+log debug "Successful vif-openvswitch $command for $dev."
+if [ "$type_if" = vif -a "$command" = "online" ];
then
+ success
+fi
--
1.7.2.5
On 22/04/13 14:36, Ian Campbell wrote:> Based on Waldi''s RFC at > http://lists.xen.org/archives/html/xen-devel/2012-09/msg00943.html > > To use it set vif.default.script="vif-openvswitch" in /etc/xen/xl.conf or use > script=vif-openvswitch in the vif configuration. > > Appears to do the right thing for PV and HVM guests (including tap devices) > and with stubdomains. > > In order to support VLAN tagging and trunking the "bridge" specified in the > configuration can have a special syntax, that is: > > BRIDGE_NAME[.VLAN][:TRUNK:TRUNK] > > e.g. > - xenbr0.99 > add the VIF to VLAN99 on xenbr0 > - xenbr0:99:100:101 > add the VIF to xenbr0 as a trunk port receiving VLANs 99, 100 & 101 > > Waldi, can you confirm if I have correctly reverse engineered the syntax from > the regexp please ;-) > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > Signed-off-by: Bastian Blank <waldi@debian.org> > Cc: dev@openvswitch.org > --- > v2: Correct syntax description in commitlog. > Remove erroneous xl.conf change. > Added Bastian''s S-o-b. > Agreed with Ben Pfaff that the Xen tree is a good home for this script. > --- > tools/hotplug/Linux/Makefile | 1 + > tools/hotplug/Linux/vif-openvswitch | 99 +++++++++++++++++++++++++++++++++++ > 2 files changed, 100 insertions(+), 0 deletions(-) > create mode 100644 tools/hotplug/Linux/vif-openvswitch > > diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile > index 0605559..99bf87f 100644 > --- a/tools/hotplug/Linux/Makefile > +++ b/tools/hotplug/Linux/Makefile > @@ -13,6 +13,7 @@ XENCOMMONS_SYSCONFIG = init.d/sysconfig.xencommons > XEN_SCRIPTS = network-bridge vif-bridge > XEN_SCRIPTS += network-route vif-route > XEN_SCRIPTS += network-nat vif-nat > +XEN_SCRIPTS += vif-openvswitch > XEN_SCRIPTS += vif2 > XEN_SCRIPTS += vif-setup > XEN_SCRIPTS += block > diff --git a/tools/hotplug/Linux/vif-openvswitch b/tools/hotplug/Linux/vif-openvswitch > new file mode 100644 > index 0000000..53c2f50 > --- /dev/null > +++ b/tools/hotplug/Linux/vif-openvswitch > @@ -0,0 +1,99 @@ > +#!/bin/bash > +#===========================================================================> +# ${XEN_SCRIPT_DIR}/vif-openvswitch > +# > +# Script for configuring a vif in openvswitch mode. > +# The hotplugging system will call this script if it is specified either in > +# the device configuration given to Xend, or the default Xend configuration > +# in ${XEN_CONFIG_DIR}/xend-config.sxp. If the script is specified in > +# neither of those places, then this script is the default. > +# > +# Usage: > +# vif-openvswitch (add|remove|online|offline) > +# > +# Environment vars: > +# vif vif interface name (required). > +# XENBUS_PATH path to this device''s details in the XenStore (required). > +# > +# Read from the store: > +# bridge openvswitch to add the vif to (required). > +# ip list of IP networks for the vif, space-separated (optional). > +# > +# up: > +# Enslaves the vif interface to the bridge and adds iptables rules > +# for its ip addresses (if any). > +# > +# down: > +# Removes the vif interface from the bridge and removes the iptables > +# rules for its ip addresses (if any). > +#===========================================================================> + > +dir=$(dirname "$0") > +. "$dir/vif-common.sh" > + > +openvswitch_external_id() { > + local dev=$1 > + local key=$2 > + local value=$3 > + > + echo "-- set interface $dev external-ids:\"$key\"=\"$value\"" > +} > + > +openvswitch_external_id_all() { > + local dev=$1 > + local frontend_id=$(xenstore_read "$XENBUS_PATH/frontend-id") > + local vm_path=$(xenstore_read "/local/domain/${frontend_id}/vm") > + local name=$(xenstore_read "${vm_path}/name") > + openvswitch_external_id $dev "xen-vm-name" "$name" > + local uuid=$(xenstore_read "${vm_path}/uuid") > + openvswitch_external_id $dev "xen-vm-uuid" "$uuid" > + local mac=$(xenstore_read "$XENBUS_PATH/mac") > + openvswitch_external_id $dev "attached-mac" "$mac" > +} > + > +add_to_openvswitch () { > + local dev=$1 > + local bridge="$(xenstore_read_default "$XENBUS_PATH/bridge" "$bridge")" > + local tag trunk > + > + if [[ $bridge =~ ^([^.:]+)(\.([[:digit:]]+))?(:([[:digit:]]+(:[[:digit:]]+)*))?$ ]]; then > + bridge="${BASH_REMATCH[1]}" > + tag="${BASH_REMATCH[3]}" > + trunk="${BASH_REMATCH[5]//:/,}" > + else > + fatal "No valid bridge was specified" > + fi > + > + if [ $trunk ]; then > + local trunk_arg="trunk=$trunk" > + fi > + > + if [ $tag ]; then > + local tag_arg="tag=$tag" > + fi > + > + local vif_details="$(openvswitch_external_id_all $dev)" > + > + ovs-vsctl --timeout=30 -- --if-exists del-port $dev -- add-port "$bridge" $dev $tag_arg $trunk_arg $vif_details > + ip link set $dev up > +} > + > +case "$command" in > + add|online) > + setup_virtual_bridge_port $dev > + add_to_openvswitch $dev > + ;; > + > + offline)Don''t you need a "remove" here? remove|offline) ... Or it doesn''t matter because when the tap device is destroyed everything is automagically cleaned?> + ovs-vsctl --timeout=30 -- --if-exists del-port $dev > + ;; > +esac > + > +if [ "$type_if" = vif ]; then > + handle_iptable > +fi > + > +log debug "Successful vif-openvswitch $command for $dev." > +if [ "$type_if" = vif -a "$command" = "online" ]; then > + success > +fi >
On Mon, 2013-04-22 at 17:46 +0100, Roger Pau Monne wrote:> > + offline) > > Don''t you need a "remove" here? > > remove|offline) > ... > > Or it doesn''t matter because when the tap device is destroyed everything > is automagically cleaned?Even if that were the case I wouldn''t want to rely on it, so we should try and clean up explicitly. Is this tap devices which only generates add/remove and not online/offline? Do you happen to know the sequence of events for both types of device? Ian.
On 22/04/13 19:05, Ian Campbell wrote:> On Mon, 2013-04-22 at 17:46 +0100, Roger Pau Monne wrote: >>> + offline) >> >> Don''t you need a "remove" here? >> >> remove|offline) >> ... >> >> Or it doesn''t matter because when the tap device is destroyed everything >> is automagically cleaned? > > Even if that were the case I wouldn''t want to rely on it, so we should > try and clean up explicitly. > > Is this tap devices which only generates add/remove and not > online/offline? Do you happen to know the sequence of events for both > types of device?It''s all in libxl_linux.c, libxl__hotplug_nic in case you want to take a look at the implementation, the sequence of events for HVM guests should be: - online vif - add tap ... - offline vif - remove tap
On Mon, 2013-04-22 at 18:20 +0100, Roger Pau Monne wrote:> On 22/04/13 19:05, Ian Campbell wrote: > > On Mon, 2013-04-22 at 17:46 +0100, Roger Pau Monne wrote: > >>> + offline) > >> > >> Don''t you need a "remove" here? > >> > >> remove|offline) > >> ... > >> > >> Or it doesn''t matter because when the tap device is destroyed everything > >> is automagically cleaned? > > > > Even if that were the case I wouldn''t want to rely on it, so we should > > try and clean up explicitly. > > > > Is this tap devices which only generates add/remove and not > > online/offline? Do you happen to know the sequence of events for both > > types of device? > > It''s all in libxl_linux.c, libxl__hotplug_nic in case you want to take a > look at the implementation, the sequence of events for HVM guests should be:Ah, yes, I''d forgotten I wasn''t worrying about udev anymore. Anyway, I confirmed that I was leaking the vifX.y-emu devices into the OVS configuration, since unlike bridges OVS ports are persistent and not tied to the existence of the actual device. With the change you suggested things are fine. V3 on its way... Thanks, Ian.