libvirt users - Feb 2013 - Networking problem - multiple guests using MacVTap in bridge mode

Dear list,

I am getting strange problems using a setup outlined in the subject. I
want to run three guests on a single kvm host. The guests are connected
through macvtap directly to the network device 'eth0' of the host using
this configuration:

  <interface type='direct'>
    <mac address='52:54:00:86:3f:24'/>
    <source dev='eth0' mode='bridge'/>
    <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
  </interface>

Each guest is assigned a static ip address, no DHCP is involved.

When I start the guests the macvtap links are generated automatically,
ifconfig outputs this:

  macvtap2  Link encap:Ethernet  HWaddr 52:54:00:86:3f:24
            inet6 addr: fe80::5054:ff:fe86:3f24/64 Scope:Link
            UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
            RX packets:98 errors:0 dropped:0 overruns:0 frame:0
            TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
            collisions:0 txqueuelen:500
            RX bytes:10946 (10.6 KiB)  TX bytes:2184 (2.1 KiB)

However, depending on the startup order, only the first two guests are
able to reach the network! I can't access the third one. After the
initial startup, the last host won't connect to the network even if I
restart
all the guests in a differnt order. I can only reset this by restarting
the host.

As far as I have tested this, it really depends on the initial startup order
of the guests after a host restart. This seems very strange to me.

As I am quite new to all of this, I have no clue how to start searching for the
problem here. Can anyone provide me with some hints?

On 02/01/2013 08:26 AM, Filip Korzeniowski wrote:
> Dear list,
>
> I am getting strange problems using a setup outlined in the subject. I
> want to run three guests on a single kvm host. The guests are connected
> through macvtap directly to the network device 'eth0' of the host
using
> this configuration:
>
>   <interface type='direct'>
>     <mac address='52:54:00:86:3f:24'/>
>     <source dev='eth0' mode='bridge'/>
>     <address type='pci' domain='0x0000'
bus='0x00' slot='0x03' function='0x0'/>
>   </interface>
>
> Each guest is assigned a static ip address, no DHCP is involved.
>
> When I start the guests the macvtap links are generated automatically,
> ifconfig outputs this:
>
>   macvtap2  Link encap:Ethernet  HWaddr 52:54:00:86:3f:24
>             inet6 addr: fe80::5054:ff:fe86:3f24/64 Scope:Link
>             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>             RX packets:98 errors:0 dropped:0 overruns:0 frame:0
>             TX packets:31 errors:0 dropped:0 overruns:0 carrier:0
>             collisions:0 txqueuelen:500
>             RX bytes:10946 (10.6 KiB)  TX bytes:2184 (2.1 KiB)
>
> However, depending on the startup order, only the first two guests are
> able to reach the network! I can't access the third one. After the
> initial startup, the last host won't connect to the network even if I
restart
> all the guests in a differnt order. I can only reset this by restarting
> the host.
Assuming that each guest's interface definition has a unique MAC
address, this really sounds like a bug in macvtap on your kernel. There
is probably some limit on the number of macvtap interfaces attached to a
single physical interface, but it's certainly much higher than 3, and if
you hit it you'd be more likely to get an error when attempting to
create the macvtap device.