libvirt users - Oct 2022 - Predictable and consistent net interface naming in guests

Hi Igor and Laine,

I would like to revive a 2 years old discussion [1] about consistent network
interfaces in the guest.

That discussion mentioned that a guest PCI address may change in two cases:
- The PCI topology changes.
- The machine type changes.

Usually, the machine type is not expected to change, especially if one
wants to allow migrations between nodes.
I would hope to argue this should not be problematic in practice, because
guest images would be made per a specific machine type.

Regarding the PCI topology, I am not sure I understand what changes
need to occur to the domxml for a defined guest PCI address to change.
The only think that I can think of is a scenario where hotplug/unplug is
used,
but even then I would expect existing devices to preserve their PCI address
and the plug/unplug device to have a reserved address managed by the one
acting on it (the management system).

Could you please help clarify in which scenarios the PCI topology can cause
a mess to the naming of interfaces in the guest?

Are there any plans to add the acpi_index support?

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1874096#c15

Thank you,
Edy.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20221031/f8e55998/attachment.htm>

On Mon, Oct 31, 2022 at 16:32:27 +0200, Edward Haas wrote:

[...]
> Are there any plans to add the acpi_index support?
https://www.libvirt.org/formatdomain.html#network-interfaces

"Since 7.3.0, one can set the ACPI index against network interfaces. With
 some operating systems (eg Linux with systemd), the ACPI index is used
 to provide network interface device naming, that is stable across
 changes in PCI addresses assigned to the device. This value is required
 to be unique across all devices and be between 1 and (16*1024-1)."

On Mon, Oct 31, 2022 at 04:32:27PM +0200, Edward Haas
wrote:
> Hi Igor and Laine,
> 
> I would like to revive a 2 years old discussion [1] about consistent
network
> interfaces in the guest.
> 
> That discussion mentioned that a guest PCI address may change in two cases:
> - The PCI topology changes.
> - The machine type changes.
> 
> Usually, the machine type is not expected to change, especially if one
> wants to allow migrations between nodes.
> I would hope to argue this should not be problematic in practice, because
> guest images would be made per a specific machine type.
> 
> Regarding the PCI topology, I am not sure I understand what changes
> need to occur to the domxml for a defined guest PCI address to change.
> The only think that I can think of is a scenario where hotplug/unplug is
> used,
> but even then I would expect existing devices to preserve their PCI address
> and the plug/unplug device to have a reserved address managed by the one
> acting on it (the management system).
> 
> Could you please help clarify in which scenarios the PCI topology can cause
> a mess to the naming of interfaces in the guest?
> 
> Are there any plans to add the acpi_index support?
This was implemented a year & a half ago

  https://libvirt.org/formatdomain.html#network-interfaces

though due to QEMU limitations this only works for the old
i440fx chipset, not Q35 yet.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

On Mon, Oct 31, 2022 at 04:32:27PM +0200, Edward Haas
wrote:
> That discussion mentioned that a guest PCI address may change in two cases:
> - The PCI topology changes.
> - The machine type changes.
>
> Usually, the machine type is not expected to change, especially if one
> wants to allow migrations between nodes.
> I would hope to argue this should not be problematic in practice, because
> guest images would be made per a specific machine type.
The machine type might not change from q35 to i440fx and vice versa,
but since the domain XML is constructed every time a KubeVirt VM is
started, the machine type might be q35-6.0 on one boot and q35-7.0
the next one if a KubeVirt upgrade that comes with a new version of
QEMU has happened in between.

This is unlikely to make a difference in terms of PCI addresses seen
in the guest OS, but it's still not accurate to say that the machine
type will not change.

Live migration is a separate matter, as the machine type will
definitely not change while the VM is running.
> Regarding the PCI topology, I am not sure I understand what changes
> need to occur to the domxml for a defined guest PCI address to change.
> The only think that I can think of is a scenario where hotplug/unplug is
> used,
> but even then I would expect existing devices to preserve their PCI address
> and the plug/unplug device to have a reserved address managed by the one
> acting on it (the management system).
>
> Could you please help clarify in which scenarios the PCI topology can cause
> a mess to the naming of interfaces in the guest?
A change in libvirt (again, due to a KubeVirt upgrade in between two
boots of the same VM) might result in different PCI addresses being
assigned to devices despite the same input XML.

We generally try fairly hard to avoid this kind of situation, but we
can only really guarantee stable PCI addresses for the lifetime of a
VM that has been defined and can't promise that the same input XML
will result in the same guest ABI when using different versions of
libvirt.

-- 
Andrea Bolognani / Red Hat / Virtualization