libvirt users - Apr 2013 - Problem with net-define using Open vSwitch bridge

Hi all,

I used "net-define" to create a network to use to attach VMs to an
Open vSwitch bridge. The net-define worked fine, I can see the network in
"net-list" (I set it to started and to autostart) but when I try to
attach a new VM's NIC to this network (via virt-manager), I'm getting
the following traceback:

Unable to complete install: 'Unable to add bridge vl10-ovsbr0 port vnet0:
Operation not supported'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in
cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1943, in
do_install
    guest.start_install(False, meter=meter)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1249, in start_install
    noboot)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1317, in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in
createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed',
conn=self)
libvirtError: Unable to add bridge vl10-ovsbr0 port vnet0: Operation not
supported

I'm going to assume the problem is that the bridge is not a linux bridge,
but an Open vSwitch bridge. But I did have a line in the XML file that I used
for net-define as follows:

<virtualport type='openvswitch'/>

But when I ran net-define, it failed to create the network with the error:

XML error: unexpected virtualport type -1

Removing that line and re-running the net-define worked, as I have stated above.
How do I tell libvirt that this bridge is an Open vSwitch bridge and not a
regular Linux bridge?

Thanks,
Will

Update: I have upgraded from libvirt 0.9.8 on the virtualization server
to libvirt 1.0.4. Now the net-define command works with the "virtualport
type='openvswitch'" directive in it. So I did a
"net-edit" on the
networks, added that line in, and saved them. So now I am assuming
libvirt knows that the underlying bridge type is Open vSwitch. However,
when I try to use virt-manager (which is 0.9.4 running on Fedora 18) to
spin up a new VM and try to attach it to one of the networks I defined
on the virtualization host (runs Ubuntu 12.04 Server with libvirt 1.0.4
as I had mentioned above), I am now getting the following different
traceback:

Unable to complete install: 'Unable to add port vnet0 to OVS bridge
vl20-ovsbr0: Operation not permitted'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in
cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1943, in
do_install
    guest.start_install(False, meter=meter)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1249,
in start_install
    noboot)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1317,
in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in
createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed',
conn=self)
libvirtError: Unable to add port vnet0 to OVS bridge vl20-ovsbr0:
Operation not permitted

So the error went from "Operation not supported" to "Operation
not
permitted"... Is this some sort of a permissions error now??

If someone could please help me figure this out, that would be great, as
I'm up against a bit of a deadline with this to get it working...

Thanks,
Will

-----Original Message-----
From: Will Dennis 
Sent: Friday, April 05, 2013 12:12 PM
To: libvirt-users at redhat.com
Subject: Problem with net-define using Open vSwitch bridge

Hi all,

I used "net-define" to create a network to use to attach VMs to an
Open
vSwitch bridge. The net-define worked fine, I can see the network in
"net-list" (I set it to started and to autostart) but when I try to
attach a new VM's NIC to this network (via virt-manager), I'm getting
the following traceback:

Unable to complete install: 'Unable to add bridge vl10-ovsbr0 port
vnet0: Operation not supported'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in
cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1943, in
do_install
    guest.start_install(False, meter=meter)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1249,
in start_install
    noboot)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1317,
in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in
createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed',
conn=self)
libvirtError: Unable to add bridge vl10-ovsbr0 port vnet0: Operation not
supported

I'm going to assume the problem is that the bridge is not a linux
bridge, but an Open vSwitch bridge. But I did have a line in the XML
file that I used for net-define as follows:

<virtualport type='openvswitch'/>

But when I ran net-define, it failed to create the network with the
error:

XML error: unexpected virtualport type -1

Removing that line and re-running the net-define worked, as I have
stated above. How do I tell libvirt that this bridge is an Open vSwitch
bridge and not a regular Linux bridge?

Thanks,
Will

Thanks, Laine, for your reply. The OVS bridge named 'vl20-ovsbr0' does
exist, but it is an Open vSwitch "fake bridge" -- see
http://blog.scottlowe.org/2012/10/19/vlans-with-open-vswitch-fake-bridge
s/ for why I'm doing this.
Here's some relevant output from my system:

# ifconfig | grep encap
eth0      Link encap:Ethernet  HWaddr 00:25:90:aa:bb:cc
lo        Link encap:Local Loopback
ovsbr0    Link encap:Ethernet  HWaddr a2:c6:37:cd:90:4a
vl10-ovsbr0 Link encap:Ethernet  HWaddr b6:c4:8c:69:2b:06
vl20-ovsbr0 Link encap:Ethernet  HWaddr d6:d0:be:6e:b6:fb

# ovs-vsctl --real list-br
ovsbr0

# ovs-vsctl --fake list-br
vl10-ovsbr0
vl20-ovsbr0

# ovs-vsctl show
3e0d861b-efb7-46b1-af1b-4a76cd833558
    Bridge "ovsbr0"
        Controller "tcp:127.0.0.1:6633"
            is_connected: true
        Port "ovsbr0"
            Interface "ovsbr0"
                type: internal
        Port "vl10-ovsbr0"
            tag: 10
            Interface "vl10-ovsbr0"
                type: internal
        Port "vl20-ovsbr0"
            tag: 20
            Interface "vl20-ovsbr0"
                type: internal

# ovs-vsctl br-to-parent vl20-ovsbr0
ovsbr0

virsh # net-dumpxml 20-net
<network>
  <name>20-net</name>
  <uuid>9e2c2ce7-a193-17ec-d6c0-3f115e759e29</uuid>
  <forward mode='bridge'/>
  <bridge name='vl20-ovsbr0' />
  <virtualport type='openvswitch'/>
</network>

(I have no XML from the guest, as virt-manager will not let me create it
with the desired network...)

Is the problem that libvirt only can define networks with "real" OVS
bridges? That would be a bummer, since I need the VM's port tagged with
a specific VLAN... Or is there another workable way to do this?

Thanks,
Will


On 04/08/2013 04:15 PM, Laine Stump wrote:
> Please send a copy of your network definition, as well as the
> <interface> element of your guest's definition so that we can see
what
> you're working with.
> 
> Does the OVS bridge named vl20-ovsbr0 exist?
-----Original Message-----
From: Will Dennis 
Sent: Monday, April 08, 2013 2:45 PM
To: libvirt-users at redhat.com
Subject: RE: Problem with net-define using Open vSwitch bridge

Update: I have upgraded from libvirt 0.9.8 on the virtualization server
to libvirt 1.0.4. Now the net-define command works with the "virtualport
type='openvswitch'" directive in it. So I did a
"net-edit" on the
networks, added that line in, and saved them. So now I am assuming
libvirt knows that the underlying bridge type is Open vSwitch. However,
when I try to use virt-manager (which is 0.9.4 running on Fedora 18) to
spin up a new VM and try to attach it to one of the networks I defined
on the virtualization host (runs Ubuntu 12.04 Server with libvirt 1.0.4
as I had mentioned above), I am now getting the following different
traceback:

Unable to complete install: 'Unable to add port vnet0 to OVS bridge
vl20-ovsbr0: Operation not permitted'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in
cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1943, in
do_install
    guest.start_install(False, meter=meter)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1249,
in start_install
    noboot)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1317,
in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in
createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed',
conn=self)
libvirtError: Unable to add port vnet0 to OVS bridge vl20-ovsbr0:
Operation not permitted

So the error went from "Operation not supported" to "Operation
not
permitted"... Is this some sort of a permissions error now??

If someone could please help me figure this out, that would be great, as
I'm up against a bit of a deadline with this to get it working...

Thanks,
Will

-----Original Message-----
From: Will Dennis 
Sent: Friday, April 05, 2013 12:12 PM
To: libvirt-users at redhat.com
Subject: Problem with net-define using Open vSwitch bridge

Hi all,

I used "net-define" to create a network to use to attach VMs to an
Open
vSwitch bridge. The net-define worked fine, I can see the network in
"net-list" (I set it to started and to autostart) but when I try to
attach a new VM's NIC to this network (via virt-manager), I'm getting
the following traceback:

Unable to complete install: 'Unable to add bridge vl10-ovsbr0 port
vnet0: Operation not supported'

Traceback (most recent call last):
  File "/usr/share/virt-manager/virtManager/asyncjob.py", line 96, in
cb_wrapper
    callback(asyncjob, *args, **kwargs)
  File "/usr/share/virt-manager/virtManager/create.py", line 1943, in
do_install
    guest.start_install(False, meter=meter)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1249,
in start_install
    noboot)
  File "/usr/lib/python2.7/site-packages/virtinst/Guest.py", line
1317,
in _create_guest
    dom = self.conn.createLinux(start_xml or final_xml, 0)
  File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2606, in
createLinux
    if ret is None:raise libvirtError('virDomainCreateLinux() failed',
conn=self)
libvirtError: Unable to add bridge vl10-ovsbr0 port vnet0: Operation not
supported

I'm going to assume the problem is that the bridge is not a linux
bridge, but an Open vSwitch bridge. But I did have a line in the XML
file that I used for net-define as follows:

<virtualport type='openvswitch'/>

But when I ran net-define, it failed to create the network with the
error:

XML error: unexpected virtualport type -1

Removing that line and re-running the net-define worked, as I have
stated above. How do I tell libvirt that this bridge is an Open vSwitch
bridge and not a regular Linux bridge?

Thanks,
Will