libvirt users - Jun 2022 - vepa-mode directly attached interface

On 6/11/22 8:54 AM, Gionatan Danti wrote:
> Hi all,
> I just realized that libvirt default for directly attached interface is 
> vepa mode.
> 
> I discovered it now because virt-manager automatically enables bridge 
> mode, while cockpit-machines get the default vepa mode. This is 
> unfortunate because, being vepa switches very rare, it means that using 
> cockpit to configure directly attached interfaces causes guests to not 
> talk each other.
> 
> I have some questions:
> 
> - can libvirt default be changed/configured somehow (ie: to 
> automatically create bridge-mode directly attached interface when no 
> mode is specified)?
libvirt's defaults are defaults, and can't be changed with config. 
libvirt also will never changed the hardcoded default, because that 
could break existing installations.

I'm not sure why vepa was chosen as the default when macvtap support was 
added to libvirt, but my best guess is that it was just the bias of the 
person doing the work who assumed their usage would be the most common 
(IIRC it was done by someone who was specifically wanting to support 
connection to VEPA-capable switches, and since it was a new feature 
nobody else had experience with / opinions about which mode would be 
most common, so the reviewers just accepted this default)
> 
> - how can I use virsh to discover machines with vepa-mode interfaces 
> (virsh domiflist <domain> does not return the interface mode)?
I guess you'll need to do a "virsh dumpxml --inactive" for each
guest
and parse it out of there.
> 
> - can I change the interface type at runtime (virt-xml <domain>
--edit
> --network type=direct,source.mode=bridge works for inactive domains only)?
No, I think the mode of the macvtap interface is set when the interface 
is created, and can't be changed later (i.e. it's a limitation of 
macvtap, not libvirt)

Il 2022-06-11 19:32 Laine Stump ha scritto:
> libvirt's defaults are defaults, and can't be changed with config.
> libvirt also will never changed the hardcoded default, because that
> could break existing installations.
Thanks for the direct confirmation that default can not be changed with 
config. I opened a cockpit-machine bug to ask for a change there: 
https://bugzilla.redhat.com/show_bug.cgi?id=2095967
> I'm not sure why vepa was chosen as the default when macvtap support
> was added to libvirt, but my best guess is that it was just the bias
> of the person doing the work who assumed their usage would be the most
> common (IIRC it was done by someone who was specifically wanting to
> support connection to VEPA-capable switches, and since it was a new
> feature nobody else had experience with / opinions about which mode
> would be most common, so the reviewers just accepted this default)
Sound reasonable. Do you know if directly attached bridge-mode interface 
is common and tested into production workload, or if "pure" bridge
mode
is the way to go?

Full disclosure: I always used "pure" bridges, but I like the idea to 
deny any guest->host traffic by design - hence my interest in 
bridge-mode macvtap interfaces. Moreover, by the virtue of not requiring 
a dedicated bridge interface, they can simplify the host network 
configuration.
> I guess you'll need to do a "virsh dumpxml --inactive" for
each guest
> and parse it out of there.
It matches my results. However, virsh dumpxml is an expensive operation 
- polkitd can easily burn >30% of a single core for 1s polling interval 
with virsh dumpxml. I settled (for testing) with grepping "mode..vepa"
in /etc/libvirtd/qemu and it seems to work well (1s polling not even 
register in top).

Does libvirt support calling some external script when a new virtual 
machine is defined?
> No, I think the mode of the macvtap interface is set when the
> interface is created, and can't be changed later (i.e. it's a
> limitation of macvtap, not libvirt)
Again, it matches my results (ip link showing "operation not
supported"
when trying to change the mode at runtime). I hoped to miss something, 
but it does not seem the case...

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti at assyoma.it - info at assyoma.it
GPG public key ID: FF5F32A8