Based on Waldi''s RFC at
http://lists.xen.org/archives/html/xen-devel/2012-09/msg00943.html
To use it set vif.default.script="vif-openvswitch" in /etc/xen/xl.conf
or use
script=vif-openvswitch in the vif configuration.
Appears to do the right thing for PV and HVM guests (including tap devices)
and with stubdomains.
In order to support VLAN tagging and trunking the "bridge" specified
in the
configuration can have a special syntax, that is:
BRIDGE_NAME[.VLAN][:TRUNK:TRUNK]
e.g.
- xenbr0.99
add the VIF to VLAN99 on xenbr0
- xenbr0:99:100:101
add the VIF to xenbr0 as a trunk port receiving VLANs 99, 100 & 101
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Bastian Blank <waldi@debian.org>
Cc: dev@openvswitch.org
---
v4: Handle errors when adding or removing the device, with the former being
fatal. Tested by chmod -x /usr/bin/ovs-vsctl.
Bring the device down after removing it from the bridge.
Check for ovs-vsctl and ip on the path. Tested by renaming the binary.
v3: Handle the remove case as well to properly support tap devices
v2: Correct syntax description in commitlog.
Remove erroneous xl.conf change.
Added Bastian''s S-o-b.
Agreed with Ben Pfaff that the Xen tree is a good home for this script.
---
tools/hotplug/Linux/Makefile | 1 +
tools/hotplug/Linux/vif-openvswitch | 113 +++++++++++++++++++++++++++++++++++
2 files changed, 114 insertions(+), 0 deletions(-)
create mode 100644 tools/hotplug/Linux/vif-openvswitch
diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile
index 0605559..99bf87f 100644
--- a/tools/hotplug/Linux/Makefile
+++ b/tools/hotplug/Linux/Makefile
@@ -13,6 +13,7 @@ XENCOMMONS_SYSCONFIG = init.d/sysconfig.xencommons
XEN_SCRIPTS = network-bridge vif-bridge
XEN_SCRIPTS += network-route vif-route
XEN_SCRIPTS += network-nat vif-nat
+XEN_SCRIPTS += vif-openvswitch
XEN_SCRIPTS += vif2
XEN_SCRIPTS += vif-setup
XEN_SCRIPTS += block
diff --git a/tools/hotplug/Linux/vif-openvswitch
b/tools/hotplug/Linux/vif-openvswitch
new file mode 100644
index 0000000..a8e4fc7
--- /dev/null
+++ b/tools/hotplug/Linux/vif-openvswitch
@@ -0,0 +1,113 @@
+#!/bin/bash
+#===========================================================================+#
${XEN_SCRIPT_DIR}/vif-openvswitch
+#
+# Script for configuring a vif in openvswitch mode.
+# The hotplugging system will call this script if it is specified either in
+# the device configuration given to Xend, or the default Xend configuration
+# in ${XEN_CONFIG_DIR}/xend-config.sxp. If the script is specified in
+# neither of those places, then this script is the default.
+#
+# Usage:
+# vif-openvswitch (add|remove|online|offline)
+#
+# Environment vars:
+# vif vif interface name (required).
+# XENBUS_PATH path to this device''s details in the XenStore
(required).
+#
+# Read from the store:
+# bridge openvswitch to add the vif to (required).
+# ip list of IP networks for the vif, space-separated (optional).
+#
+# up:
+# Enslaves the vif interface to the bridge and adds iptables rules
+# for its ip addresses (if any).
+#
+# down:
+# Removes the vif interface from the bridge and removes the iptables
+# rules for its ip addresses (if any).
+#===========================================================================+
+dir=$(dirname "$0")
+. "$dir/vif-common.sh"
+
+check_tools()
+{
+ if ! command -v ovs-vsctl > /dev/null 2>&1; then
+ fatal "Unable to find ovs-vsctl tool"
+ fi
+ if ! command -v ip > /dev/null 2>&1; then
+ fatal "Unable to find ip tool"
+ fi
+}
+openvswitch_external_id() {
+ local dev=$1
+ local key=$2
+ local value=$3
+
+ echo "-- set interface $dev
external-ids:\"$key\"=\"$value\""
+}
+
+openvswitch_external_id_all() {
+ local dev=$1
+ local frontend_id=$(xenstore_read "$XENBUS_PATH/frontend-id")
+ local vm_path=$(xenstore_read "/local/domain/${frontend_id}/vm")
+ local name=$(xenstore_read "${vm_path}/name")
+ openvswitch_external_id $dev "xen-vm-name" "$name"
+ local uuid=$(xenstore_read "${vm_path}/uuid")
+ openvswitch_external_id $dev "xen-vm-uuid" "$uuid"
+ local mac=$(xenstore_read "$XENBUS_PATH/mac")
+ openvswitch_external_id $dev "attached-mac" "$mac"
+}
+
+add_to_openvswitch () {
+ local dev=$1
+ local bridge="$(xenstore_read_default "$XENBUS_PATH/bridge"
"$bridge")"
+ local tag trunk
+
+ if [[ $bridge =~
^([^.:]+)(\.([[:digit:]]+))?(:([[:digit:]]+(:[[:digit:]]+)*))?$ ]]; then
+ bridge="${BASH_REMATCH[1]}"
+ tag="${BASH_REMATCH[3]}"
+ trunk="${BASH_REMATCH[5]//:/,}"
+ else
+ fatal "No valid bridge was specified"
+ fi
+
+ if [ $trunk ]; then
+ local trunk_arg="trunk=$trunk"
+ fi
+
+ if [ $tag ]; then
+ local tag_arg="tag=$tag"
+ fi
+
+ local vif_details="$(openvswitch_external_id_all $dev)"
+
+ do_or_die ovs-vsctl --timeout=30 \
+ -- --if-exists del-port $dev \
+ -- add-port "$bridge" $dev $tag_arg $trunk_arg $vif_details
+ do_or_die ip link set $dev up
+}
+
+case "$command" in
+ add|online)
+ check_tools
+ setup_virtual_bridge_port $dev
+ add_to_openvswitch $dev
+ ;;
+
+ remove|offline)
+ do_without_error ovs-vsctl --timeout=30 \
+ -- --if-exists del-port $dev
+ do_without_error ip link set $dev down
+ ;;
+esac
+
+if [ "$type_if" = vif ]; then
+ handle_iptable
+fi
+
+log debug "Successful vif-openvswitch $command for $dev."
+if [ "$type_if" = vif -a "$command" = "online" ];
then
+ success
+fi
--
1.7.2.5
On 23/04/13 12:00, Ian Campbell wrote:> Based on Waldi''s RFC at > http://lists.xen.org/archives/html/xen-devel/2012-09/msg00943.html > > To use it set vif.default.script="vif-openvswitch" in /etc/xen/xl.conf or use > script=vif-openvswitch in the vif configuration. > > Appears to do the right thing for PV and HVM guests (including tap devices) > and with stubdomains. > > In order to support VLAN tagging and trunking the "bridge" specified in the > configuration can have a special syntax, that is: > > BRIDGE_NAME[.VLAN][:TRUNK:TRUNK] > > e.g. > - xenbr0.99 > add the VIF to VLAN99 on xenbr0 > - xenbr0:99:100:101 > add the VIF to xenbr0 as a trunk port receiving VLANs 99, 100 & 101 > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > Signed-off-by: Bastian Blank <waldi@debian.org>Acked-by: Roger Pau Monné <roger.pau@citrix.com>> Cc: dev@openvswitch.org > --- > v4: Handle errors when adding or removing the device, with the former being > fatal. Tested by chmod -x /usr/bin/ovs-vsctl. > Bring the device down after removing it from the bridge. > Check for ovs-vsctl and ip on the path. Tested by renaming the binary. > v3: Handle the remove case as well to properly support tap devices > v2: Correct syntax description in commitlog. > Remove erroneous xl.conf change. > Added Bastian''s S-o-b. > Agreed with Ben Pfaff that the Xen tree is a good home for this script. > --- > tools/hotplug/Linux/Makefile | 1 + > tools/hotplug/Linux/vif-openvswitch | 113 +++++++++++++++++++++++++++++++++++ > 2 files changed, 114 insertions(+), 0 deletions(-) > create mode 100644 tools/hotplug/Linux/vif-openvswitch > > diff --git a/tools/hotplug/Linux/Makefile b/tools/hotplug/Linux/Makefile > index 0605559..99bf87f 100644 > --- a/tools/hotplug/Linux/Makefile > +++ b/tools/hotplug/Linux/Makefile > @@ -13,6 +13,7 @@ XENCOMMONS_SYSCONFIG = init.d/sysconfig.xencommons > XEN_SCRIPTS = network-bridge vif-bridge > XEN_SCRIPTS += network-route vif-route > XEN_SCRIPTS += network-nat vif-nat > +XEN_SCRIPTS += vif-openvswitch > XEN_SCRIPTS += vif2 > XEN_SCRIPTS += vif-setup > XEN_SCRIPTS += block > diff --git a/tools/hotplug/Linux/vif-openvswitch b/tools/hotplug/Linux/vif-openvswitch > new file mode 100644 > index 0000000..a8e4fc7 > --- /dev/null > +++ b/tools/hotplug/Linux/vif-openvswitch > @@ -0,0 +1,113 @@ > +#!/bin/bash > +#===========================================================================> +# ${XEN_SCRIPT_DIR}/vif-openvswitch > +# > +# Script for configuring a vif in openvswitch mode. > +# The hotplugging system will call this script if it is specified either in > +# the device configuration given to Xend, or the default Xend configuration > +# in ${XEN_CONFIG_DIR}/xend-config.sxp. If the script is specified in > +# neither of those places, then this script is the default. > +# > +# Usage: > +# vif-openvswitch (add|remove|online|offline) > +# > +# Environment vars: > +# vif vif interface name (required). > +# XENBUS_PATH path to this device''s details in the XenStore (required). > +# > +# Read from the store: > +# bridge openvswitch to add the vif to (required). > +# ip list of IP networks for the vif, space-separated (optional). > +# > +# up: > +# Enslaves the vif interface to the bridge and adds iptables rules > +# for its ip addresses (if any). > +# > +# down: > +# Removes the vif interface from the bridge and removes the iptables > +# rules for its ip addresses (if any). > +#===========================================================================> + > +dir=$(dirname "$0") > +. "$dir/vif-common.sh" > + > +check_tools() > +{ > + if ! command -v ovs-vsctl > /dev/null 2>&1; then > + fatal "Unable to find ovs-vsctl tool" > + fi > + if ! command -v ip > /dev/null 2>&1; then > + fatal "Unable to find ip tool" > + fi > +} > +openvswitch_external_id() { > + local dev=$1 > + local key=$2 > + local value=$3 > + > + echo "-- set interface $dev external-ids:\"$key\"=\"$value\"" > +} > + > +openvswitch_external_id_all() { > + local dev=$1 > + local frontend_id=$(xenstore_read "$XENBUS_PATH/frontend-id") > + local vm_path=$(xenstore_read "/local/domain/${frontend_id}/vm") > + local name=$(xenstore_read "${vm_path}/name") > + openvswitch_external_id $dev "xen-vm-name" "$name" > + local uuid=$(xenstore_read "${vm_path}/uuid") > + openvswitch_external_id $dev "xen-vm-uuid" "$uuid" > + local mac=$(xenstore_read "$XENBUS_PATH/mac") > + openvswitch_external_id $dev "attached-mac" "$mac" > +} > + > +add_to_openvswitch () { > + local dev=$1 > + local bridge="$(xenstore_read_default "$XENBUS_PATH/bridge" "$bridge")" > + local tag trunk > + > + if [[ $bridge =~ ^([^.:]+)(\.([[:digit:]]+))?(:([[:digit:]]+(:[[:digit:]]+)*))?$ ]]; then > + bridge="${BASH_REMATCH[1]}" > + tag="${BASH_REMATCH[3]}" > + trunk="${BASH_REMATCH[5]//:/,}" > + else > + fatal "No valid bridge was specified" > + fi > + > + if [ $trunk ]; then > + local trunk_arg="trunk=$trunk" > + fi > + > + if [ $tag ]; then > + local tag_arg="tag=$tag" > + fi > + > + local vif_details="$(openvswitch_external_id_all $dev)" > + > + do_or_die ovs-vsctl --timeout=30 \ > + -- --if-exists del-port $dev \ > + -- add-port "$bridge" $dev $tag_arg $trunk_arg $vif_details > + do_or_die ip link set $dev up > +} > + > +case "$command" in > + add|online) > + check_tools > + setup_virtual_bridge_port $dev > + add_to_openvswitch $dev > + ;; > + > + remove|offline) > + do_without_error ovs-vsctl --timeout=30 \ > + -- --if-exists del-port $dev > + do_without_error ip link set $dev down > + ;; > +esac > + > +if [ "$type_if" = vif ]; then > + handle_iptable > +fi > + > +log debug "Successful vif-openvswitch $command for $dev." > +if [ "$type_if" = vif -a "$command" = "online" ]; then > + success > +fi >
On Tue, 2013-04-23 at 14:57 +0100, Roger Pau Monne wrote:> On 23/04/13 12:00, Ian Campbell wrote: > > Based on Waldi's RFC at > > http://lists.xen.org/archives/html/xen-devel/2012-09/msg00943.html > > > > To use it set vif.default.script="vif-openvswitch" in /etc/xen/xl.conf or use > > script=vif-openvswitch in the vif configuration. > > > > Appears to do the right thing for PV and HVM guests (including tap devices) > > and with stubdomains. > > > > In order to support VLAN tagging and trunking the "bridge" specified in the > > configuration can have a special syntax, that is: > > > > BRIDGE_NAME[.VLAN][:TRUNK:TRUNK] > > > > e.g. > > - xenbr0.99 > > add the VIF to VLAN99 on xenbr0 > > - xenbr0:99:100:101 > > add the VIF to xenbr0 as a trunk port receiving VLANs 99, 100 & 101 > > > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > > Signed-off-by: Bastian Blank <waldi@debian.org> > > Acked-by: Roger Pau Monné <roger.pau@citrix.com>Applied, thanks. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Does any one have test this script performance with Xen live migration? what is the performance result compared with brctl. I hope any body want to share it. I want to use this script with my Xen 4.2.1 using xend/xm toostack. Best Regards, Agya On Wed, Apr 24, 2013 at 1:57 PM, Ian Campbell <Ian.Campbell@citrix.com>wrote:> On Tue, 2013-04-23 at 14:57 +0100, Roger Pau Monne wrote: > > On 23/04/13 12:00, Ian Campbell wrote: > > > Based on Waldi''s RFC at > > > http://lists.xen.org/archives/html/xen-devel/2012-09/msg00943.html > > > > > > To use it set vif.default.script="vif-openvswitch" in /etc/xen/xl.conf > or use > > > script=vif-openvswitch in the vif configuration. > > > > > > Appears to do the right thing for PV and HVM guests (including tap > devices) > > > and with stubdomains. > > > > > > In order to support VLAN tagging and trunking the "bridge" specified > in the > > > configuration can have a special syntax, that is: > > > > > > BRIDGE_NAME[.VLAN][:TRUNK:TRUNK] > > > > > > e.g. > > > - xenbr0.99 > > > add the VIF to VLAN99 on xenbr0 > > > - xenbr0:99:100:101 > > > add the VIF to xenbr0 as a trunk port receiving VLANs 99, 100 & > 101 > > > > > > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > > > Signed-off-by: Bastian Blank <waldi@debian.org> > > > > Acked-by: Roger Pau Monné <roger.pau@citrix.com> > > Applied, thanks. > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 02/05/13 17:09, agya naila wrote:> Does any one have test this script performance with Xen live migration? > what is the performance result compared with brctl. I hope any body want > to share it. I want to use this script with my Xen 4.2.1 using xend/xm > toostack.I have not tried it myself, but I''m not sure this will play nice with xend/xm, AFAIK xend had a race regarding device destruction and hotplug execution, that''s one of the points why we call hotplug scripts from libxl directly instead of calling them from udev. Any reason to use xm instead of xl?
On Thu, 2013-05-02 at 16:17 +0100, Roger Pau Monne wrote:> On 02/05/13 17:09, agya naila wrote: > > Does any one have test this script performance with Xen live migration? > > what is the performance result compared with brctl. I hope any body want > > to share it. I want to use this script with my Xen 4.2.1 using xend/xm > > toostack. > > I have not tried it myself, but I''m not sure this will play nice with > xend/xm, AFAIK xend had a race regarding device destruction and hotplug > execution, that''s one of the points why we call hotplug scripts from > libxl directly instead of calling them from udev. Any reason to use xm > instead of xl?Indeed, I only wrote this script for xl and did not consider xend at all. Ian.
On Thu, May 2, 2013 at 5:18 PM, Ian Campbell <Ian.Campbell@citrix.com>wrote:> On Thu, 2013-05-02 at 16:17 +0100, Roger Pau Monne wrote: > > On 02/05/13 17:09, agya naila wrote: > > > Does any one have test this script performance with Xen live migration? > > > what is the performance result compared with brctl. I hope any body > want > > > to share it. I want to use this script with my Xen 4.2.1 using xend/xm > > > toostack. > > > > I have not tried it myself, but I''m not sure this will play nice with > > xend/xm, AFAIK xend had a race regarding device destruction and hotplug > > execution, that''s one of the points why we call hotplug scripts from > > libxl directly instead of calling them from udev. Any reason to use xm > > instead of xl? >Thank you Roger, actually I have some short experiment using Remus to enables HA and I read Remus in xl still in experiment without support for network or disk buffering so there is no best choice for me right now instead roll back to the xm toolstack. Indeed, I only wrote this script for xl and did not consider xend at> all. > > Thank you Ian> Ian. > > >Best Regards, Agya _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
steve_1991@hushmail.com
2013-May-02 15:32 UTC
Re: [PATCH V4] hotplug: add openvswitch script
On Thu, May 2, 2013 at 11:17 AM, Roger Pau Monné <roger.pau@citrix.com> wrote:> On 02/05/13 17:09, agya naila wrote: >> Does any one have test this script performance with Xen live migration? >> what is the performance result compared with brctl. I hope any body want >> to share it. I want to use this script with my Xen 4.2.1 using xend/xm >> toostack. > > I have not tried it myself, but I'm not sure this will play nice with > xend/xm, AFAIK xend had a race regarding device destruction and hotplug > execution, that's one of the points why we call hotplug scripts from > libxl directly instead of calling them from udev. Any reason to use xm > instead of xl?I think I should ask in separate email but anyways. Does xl with run_hotplug_scripts=0 (forcing hotplug through udev) has race with udev? - S _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 02/05/13 17:32, steve_1991@hushmail.com wrote:> On Thu, May 2, 2013 at 11:17 AM, Roger Pau Monné <roger.pau@citrix.com> wrote: >> On 02/05/13 17:09, agya naila wrote: >>> Does any one have test this script performance with Xen live migration? >>> what is the performance result compared with brctl. I hope any body want >>> to share it. I want to use this script with my Xen 4.2.1 using xend/xm >>> toostack. >> >> I have not tried it myself, but I'm not sure this will play nice with >> xend/xm, AFAIK xend had a race regarding device destruction and hotplug >> execution, that's one of the points why we call hotplug scripts from >> libxl directly instead of calling them from udev. Any reason to use xm >> instead of xl? > > I think I should ask in separate email but anyways. Does xl with run_hotplug_scripts=0 > (forcing hotplug through udev) has race with udev?AFAIK the hotplug script could be called by udev when the xenstore entries are already gone, because xm and libxl (with run_hotplug_scripts=0) removes the entries from xenstore, but has no control of when udev calls the hotplug script. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel