libvirt users - May 2022 - macvtap with disconnected physical interface

On 5/3/22 4:31 AM, Daniel P. Berrang? wrote:
> On Mon, May 02, 2022 at 03:42:05AM +0200, Gionatan Danti wrote:
>> Dear list,
>> I just discovered the hard way that if the lower lever physical
interface of
>> a macvlan bridge is disconnected (ie: by unplugging the eth cable,
resulting
>> in no carrier), inter-guest network traffic from all virtual machines
bound
>> to the disconnected interface is dropped.
>>
>> This behavior surprises me, as with the classic bridges I can
disconnect the
>> underlying physical interface without causing any harm to inter-guest
>> traffic.
>>
>> Am I doing something wrong, or this really is the expected behavior? If
so,
>> can I force the macvtap interfaces to bridge traffic even when the
>> underlying physical interface is disconnected?
> 
> Can you share the <interface> configuration for your guest NIC so we
> can see how it is setup.
I can't say that I've ever tried this, since my only reason for using 
macvtap is to provide the guests with direct connectivity to the 
physical network, and unplugging the physdev negates that. The behavior 
you describe doesn't surprise me all that much though, since the 
physical device in the case of a host bridge isn't an integral part of 
the bridge (it's just one more device attached to a port), while the 
physical device and macvlan bridge a much more closely associated.

I'm Cc'ing Michael Tsirkin to see if he has more authoritative 
information on whether or not the macvtaps connected to a macvlan bridge 
can communicate amongst themselves when the physdev is disconnected.

In the meantime, is there a reason you don't want to just use a standard 
host bridge that's not connected to any physdev? The one thing I can 
think of is that you might not want to allow communication between the 
host and guests, but as long as the bridge itself isn't given an IP 
address, that won't be possible (at least at the level of IP).

Il 2022-05-03 14:58 Laine Stump ha scritto:
> I can't say that I've ever tried this, since my only reason for
using
> macvtap is to provide the guests with direct connectivity to the
> physical network, and unplugging the physdev negates that. The
> behavior you describe doesn't surprise me all that much though, since
> the physical device in the case of a host bridge isn't an integral
> part of the bridge (it's just one more device attached to a port),
> while the physical device and macvlan bridge a much more closely
> associated.
Yeah, it sound very plausible.
> I'm Cc'ing Michael Tsirkin to see if he has more authoritative
> information on whether or not the macvtaps connected to a macvlan
> bridge can communicate amongst themselves when the physdev is
> disconnected.
Thanks.
> In the meantime, is there a reason you don't want to just use a
> standard host bridge that's not connected to any physdev? The one
> thing I can think of is that you might not want to allow communication
> between the host and guests, but as long as the bridge itself isn't
> given an IP address, that won't be possible (at least at the level of
> IP).
I generally use plain bridge for my KVM setup. Specifically, when using 
VLANs I setup the following:
eth -> eth.xx -> bridge -> vnet

This time, however, I need *both* a trunk-enabled VM (a virtual 
firewall) and other segregated virtual machines. A "plain" bridge
setup
would be something as:
eth -> bridge -> bridge.xx -> bridge -> vnet

Notice the two bridges, needed because bridge.xx is a VLAN interface 
when no vnet can be directly attached. To avoid the double bridges, I 
tried the following:
eth -> bridge -> bridge.xx -> macvtap

It seems to work very well but, during testing, I discovered that if the 
interface under the macvtap one (in this case the bridge itself) goes 
down, inter-guest networking is lost. As a side note, in the specific 
scenario I described above, such issues can not really happen: as a vnet 
interface is going to be always bound to the first bridge, it will be 
*always* up due to the vnet interface itself being always up 
(irrespective of the physical link status) and forcing the bridge up.

However, working so well, I thought to change my classical bridge setup 
with a macvtap based one even for simpler installation. In short, going 
from:
eth -> bridge -> vnet
to:
eth -> macvtap

But this very simple setup is going deny all guest traffic should the 
physical interface become disconnected. A very crude solution would be 
to issues "ip link set macvtap0 protodown off" when the physical link 
goes down, but I wonder if a better solution exists.

That said, is replacing classical bridges with macvtap interfaces a bad 
idea? Anything I should know before doing that?
Regards.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti at assyoma.it - info at assyoma.it
GPG public key ID: FF5F32A8