Il 2022-05-03 14:58 Laine Stump ha scritto:> I can't say that I've ever tried this, since my only reason for using > macvtap is to provide the guests with direct connectivity to the > physical network, and unplugging the physdev negates that. The > behavior you describe doesn't surprise me all that much though, since > the physical device in the case of a host bridge isn't an integral > part of the bridge (it's just one more device attached to a port), > while the physical device and macvlan bridge a much more closely > associated.Yeah, it sound very plausible.> I'm Cc'ing Michael Tsirkin to see if he has more authoritative > information on whether or not the macvtaps connected to a macvlan > bridge can communicate amongst themselves when the physdev is > disconnected.Thanks.> In the meantime, is there a reason you don't want to just use a > standard host bridge that's not connected to any physdev? The one > thing I can think of is that you might not want to allow communication > between the host and guests, but as long as the bridge itself isn't > given an IP address, that won't be possible (at least at the level of > IP).I generally use plain bridge for my KVM setup. Specifically, when using VLANs I setup the following: eth -> eth.xx -> bridge -> vnet This time, however, I need *both* a trunk-enabled VM (a virtual firewall) and other segregated virtual machines. A "plain" bridge setup would be something as: eth -> bridge -> bridge.xx -> bridge -> vnet Notice the two bridges, needed because bridge.xx is a VLAN interface when no vnet can be directly attached. To avoid the double bridges, I tried the following: eth -> bridge -> bridge.xx -> macvtap It seems to work very well but, during testing, I discovered that if the interface under the macvtap one (in this case the bridge itself) goes down, inter-guest networking is lost. As a side note, in the specific scenario I described above, such issues can not really happen: as a vnet interface is going to be always bound to the first bridge, it will be *always* up due to the vnet interface itself being always up (irrespective of the physical link status) and forcing the bridge up. However, working so well, I thought to change my classical bridge setup with a macvtap based one even for simpler installation. In short, going from: eth -> bridge -> vnet to: eth -> macvtap But this very simple setup is going deny all guest traffic should the physical interface become disconnected. A very crude solution would be to issues "ip link set macvtap0 protodown off" when the physical link goes down, but I wonder if a better solution exists. That said, is replacing classical bridges with macvtap interfaces a bad idea? Anything I should know before doing that? Regards. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8
On Tue, May 03, 2022 at 11:15:26PM +0200, Gionatan Danti wrote:> Il 2022-05-03 14:58 Laine Stump ha scritto: > > I can't say that I've ever tried this, since my only reason for using > > macvtap is to provide the guests with direct connectivity to the > > physical network, and unplugging the physdev negates that. The > > behavior you describe doesn't surprise me all that much though, since > > the physical device in the case of a host bridge isn't an integral > > part of the bridge (it's just one more device attached to a port), > > while the physical device and macvlan bridge a much more closely > > associated. > > Yeah, it sound very plausible. > > > I'm Cc'ing Michael Tsirkin to see if he has more authoritative > > information on whether or not the macvtaps connected to a macvlan > > bridge can communicate amongst themselves when the physdev is > > disconnected. > > Thanks. > > > In the meantime, is there a reason you don't want to just use a > > standard host bridge that's not connected to any physdev? The one > > thing I can think of is that you might not want to allow communication > > between the host and guests, but as long as the bridge itself isn't > > given an IP address, that won't be possible (at least at the level of > > IP). > > I generally use plain bridge for my KVM setup. Specifically, when using > VLANs I setup the following: > eth -> eth.xx -> bridge -> vnet > > This time, however, I need *both* a trunk-enabled VM (a virtual firewall) > and other segregated virtual machines. A "plain" bridge setup would be > something as: > eth -> bridge -> bridge.xx -> bridge -> vnet > > Notice the two bridges, needed because bridge.xx is a VLAN interface when no > vnet can be directly attached. To avoid the double bridges, I tried the > following: > eth -> bridge -> bridge.xx -> macvtap > > It seems to work very well but, during testing, I discovered that if the > interface under the macvtap one (in this case the bridge itself) goes down, > inter-guest networking is lost. As a side note, in the specific scenario I > described above, such issues can not really happen: as a vnet interface is > going to be always bound to the first bridge, it will be *always* up due to > the vnet interface itself being always up (irrespective of the physical link > status) and forcing the bridge up.well this is kind of by design. if we don't force guest's link down it will think it can access internet, with all packets being lost. I see how you might want that in some cases - maybe we should consider an option to disable this behaviour.> However, working so well, I thought to change my classical bridge setup with > a macvtap based one even for simpler installation. In short, going from: > eth -> bridge -> vnet > to: > eth -> macvtap > > But this very simple setup is going deny all guest traffic should the > physical interface become disconnected. A very crude solution would be to > issues "ip link set macvtap0 protodown off" when the physical link goes > down, but I wonder if a better solution exists. > > That said, is replacing classical bridges with macvtap interfaces a bad > idea? Anything I should know before doing that? > Regards. > > -- > Danti Gionatan > Supporto Tecnico > Assyoma S.r.l. - www.assyoma.it > email: g.danti at assyoma.it - info at assyoma.it > GPG public key ID: FF5F32A8
Il 2022-05-04 00:03 Michael S. Tsirkin ha scritto:> well this is kind of by design. if we don't force guest's link > down it will think it can access internet, with all packets > being lost.I understand. But in my opinion, inter-guest communication should be preserved because: a) this is how the classical bridge mode works; b) modern guest have internal checks for internet connection; c) guest-to-guest communication should not be impaired by (lack of) internet access.> I see how you might want that in some cases - maybe we > should consider an option to disable this behaviour.It would be great - especially as default behavior. In the meantime, do we have some virsh command (or libvirt xml snippet) to work around the issue, rather than messing with "ip link protodown off"? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8
Il 2022-05-03 23:15 Gionatan Danti ha scritto:> I generally use plain bridge for my KVM setup. Specifically, when > using VLANs I setup the following: > eth -> eth.xx -> bridge -> vnet > > This time, however, I need *both* a trunk-enabled VM (a virtual > firewall) and other segregated virtual machines. A "plain" bridge > setup would be something as: > eth -> bridge -> bridge.xx -> bridge -> vnet > > Notice the two bridges, needed because bridge.xx is a VLAN interface > when no vnet can be directly attached. To avoid the double bridges, I > tried the following: > eth -> bridge -> bridge.xx -> macvtap > > It seems to work very well but, during testing, I discovered that if > the interface under the macvtap one (in this case the bridge itself) > goes down, inter-guest networking is lost. As a side note, in the > specific scenario I described above, such issues can not really > happen: as a vnet interface is going to be always bound to the first > bridge, it will be *always* up due to the vnet interface itself being > always up (irrespective of the physical link status) and forcing the > bridge up. > > However, working so well, I thought to change my classical bridge > setup with a macvtap based one even for simpler installation. In > short, going from: > eth -> bridge -> vnet > to: > eth -> macvtap > > But this very simple setup is going deny all guest traffic should the > physical interface become disconnected. A very crude solution would be > to issues "ip link set macvtap0 protodown off" when the physical link > goes down, but I wonder if a better solution exists. > > That said, is replacing classical bridges with macvtap interfaces a > bad idea? Anything I should know before doing that? > Regards.Hi all, any comment / suggestion on the steps described above? Does a simpler approach exists? Thanks. -- Danti Gionatan Supporto Tecnico Assyoma S.r.l. - www.assyoma.it email: g.danti at assyoma.it - info at assyoma.it GPG public key ID: FF5F32A8