Richard W.M. Jones
2023-Feb-06 12:49 UTC
[Libguestfs] [PATCH v2v] convert: linux: Require host cpu for all RHEL-alike >= 9
On Mon, Feb 06, 2023 at 12:37:30PM +0000, Daniel P. Berrang? wrote:> On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote: > > RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using > > the default qemu CPU. You will see an error at boot: > > > > Fatal glibc error: CPU does not support x86-64-v2 > > > > Instead you need to use -cpu host. > > > > In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false" > > for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9. > > > > This commit extends the same fix to all RHEL family distros. > > If v2v is going to use host CPU for a bunch of distros, why not use > it for all distros ? What's the downside in -cpu host, instead of > the default qemu64 CPU ? > > Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much > all benefit from NOT using the default QEMU CPU model, if only for > the fact that they gain the 'aesni' feature which increases crypto > performance massively.I don't know what we should do, but the current code does this: (1) If a particular CPU model is specified by the source hypervisor (eg VMware), use that. (2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough (3) Otherwise don't specify any -cpu or <cpu> section The third option seems to cause qemu / libvirt to use qemu64. However it has the advantage that the guest is migratable afterwards. There may be a case for changing this so: (a) Rule (2) is changed so we use host-model (for the libvirt target). (b) Rule (3) is changed so we always specify a minimum CPU, eg. for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??) (a) seems like a no-brainer. Not sure why we chose host-passthrough instead of host-model? The explanation in the commit message is not very convincing: https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5646586c6c7886 I think (b) is fraught because it is my understanding that there is no good choice for a minimum CPU since we don't know anything about the target hardware (eg. if it's Intel or AMD). And on !x86-64 we really have no idea at all what's a good choice. Other management systems like RHV, KubeVirt are probably ignoring our CPU metadata anyway ... Rich.> > > > Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850 > > Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619 > > Reported-by: Ming Xie > > Thanks: Laszlo Ersek > > --- > > convert/convert_linux.ml | 4 ++-- > > 1 file changed, 2 insertions(+), 2 deletions(-) > > > > diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml > > index e20cafa551..460cbff0ed 100644 > > --- a/convert/convert_linux.ml > > +++ b/convert/convert_linux.ml > > @@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware keep_serial_console _ > > > > (* RHEL >= 9.0 on x86_64 requires the processor to support the "x86-64-v2" > > * microarchitecture level, which the default QEMU VCPU model does not > > - * satisfy. Refer to RHBZ#2076013. > > + * satisfy. Refer to RHBZ#2076013 RHBZ#2166619. > > *) > > - let default_cpu_suffices = inspect.i_distro <> "rhel" || > > + let default_cpu_suffices = family <> `RHEL_family || > > inspect.i_arch <> "x86_64" || > > inspect.i_major_version < 9 in > > > > -- > > 2.39.0 > > > > _______________________________________________ > > Libguestfs mailing list > > Libguestfs at redhat.com > > https://listman.redhat.com/mailman/listinfo/libguestfs > > > > With regards, > Daniel > -- > |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| > |: https://libvirt.org -o- https://fstop138.berrange.com :| > |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|-- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html
Laszlo Ersek
2023-Feb-07 07:23 UTC
[Libguestfs] [PATCH v2v] convert: linux: Require host cpu for all RHEL-alike >= 9
On 2/6/23 13:49, Richard W.M. Jones wrote:> On Mon, Feb 06, 2023 at 12:37:30PM +0000, Daniel P. Berrang? wrote: >> On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote: >>> RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using >>> the default qemu CPU. You will see an error at boot: >>> >>> Fatal glibc error: CPU does not support x86-64-v2 >>> >>> Instead you need to use -cpu host. >>> >>> In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false" >>> for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9. >>> >>> This commit extends the same fix to all RHEL family distros. >> >> If v2v is going to use host CPU for a bunch of distros, why not use >> it for all distros ? What's the downside in -cpu host, instead of >> the default qemu64 CPU ? >> >> Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much >> all benefit from NOT using the default QEMU CPU model, if only for >> the fact that they gain the 'aesni' feature which increases crypto >> performance massively. > > I don't know what we should do, but the current code does this: > > (1) If a particular CPU model is specified by the source hypervisor > (eg VMware), use that. > > (2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough > > (3) Otherwise don't specify any -cpu or <cpu> section > > The third option seems to cause qemu / libvirt to use qemu64. However > it has the advantage that the guest is migratable afterwards. > > There may be a case for changing this so: > > (a) Rule (2) is changed so we use host-model (for the libvirt target). > > (b) Rule (3) is changed so we always specify a minimum CPU, eg. > for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??) > > (a) seems like a no-brainer. Not sure why we chose host-passthrough > instead of host-model? The explanation in the commit message is not > very convincing: > https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5646586c6c7886I think it's the result of the previous discussion we had about it. I don't remember it -- nor do I feel very strongly about it.> > I think (b) is fraught because it is my understanding that there is no > good choice for a minimum CPU since we don't know anything about the > target hardware (eg. if it's Intel or AMD). And on !x86-64 we really > have no idea at all what's a good choice. > > Other management systems like RHV, KubeVirt are probably ignoring our > CPU metadata anyway ...Yes, that's why gcaps_default_cpu is only considered in the QEMU and libvirt outputs. Laszlo> > Rich. > >>> >>> Updates: commit f28757c6d100060c65212ea55cfa59d308dcb850 >>> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=2166619 >>> Reported-by: Ming Xie >>> Thanks: Laszlo Ersek >>> --- >>> convert/convert_linux.ml | 4 ++-- >>> 1 file changed, 2 insertions(+), 2 deletions(-) >>> >>> diff --git a/convert/convert_linux.ml b/convert/convert_linux.ml >>> index e20cafa551..460cbff0ed 100644 >>> --- a/convert/convert_linux.ml >>> +++ b/convert/convert_linux.ml >>> @@ -201,9 +201,9 @@ let convert (g : G.guestfs) source inspect i_firmware keep_serial_console _ >>> >>> (* RHEL >= 9.0 on x86_64 requires the processor to support the "x86-64-v2" >>> * microarchitecture level, which the default QEMU VCPU model does not >>> - * satisfy. Refer to RHBZ#2076013. >>> + * satisfy. Refer to RHBZ#2076013 RHBZ#2166619. >>> *) >>> - let default_cpu_suffices = inspect.i_distro <> "rhel" || >>> + let default_cpu_suffices = family <> `RHEL_family || >>> inspect.i_arch <> "x86_64" || >>> inspect.i_major_version < 9 in >>> >>> -- >>> 2.39.0 >>> >>> _______________________________________________ >>> Libguestfs mailing list >>> Libguestfs at redhat.com >>> https://listman.redhat.com/mailman/listinfo/libguestfs >>> >> >> With regards, >> Daniel >> -- >> |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| >> |: https://libvirt.org -o- https://fstop138.berrange.com :| >> |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :| >
Daniel P. Berrangé
2023-Feb-07 08:56 UTC
[Libguestfs] [PATCH v2v] convert: linux: Require host cpu for all RHEL-alike >= 9
On Mon, Feb 06, 2023 at 12:49:41PM +0000, Richard W.M. Jones wrote:> On Mon, Feb 06, 2023 at 12:37:30PM +0000, Daniel P. Berrang? wrote: > > On Mon, Feb 06, 2023 at 12:19:40PM +0000, Richard W.M. Jones wrote: > > > RHEL >= 9 and compatible distros like Rocky >= 9 will not boot using > > > the default qemu CPU. You will see an error at boot: > > > > > > Fatal glibc error: CPU does not support x86-64-v2 > > > > > > Instead you need to use -cpu host. > > > > > > In commit f28757c6d1 ("convert_linux: set "gcaps_default_cpu = false" > > > for x86_64 RHEL-9.0+ guests") we fixed this specifically for RHEL >= 9. > > > > > > This commit extends the same fix to all RHEL family distros. > > > > If v2v is going to use host CPU for a bunch of distros, why not use > > it for all distros ? What's the downside in -cpu host, instead of > > the default qemu64 CPU ? > > > > Even if the distro doesn't require x86_64-v2 ABI, they'll pretty much > > all benefit from NOT using the default QEMU CPU model, if only for > > the fact that they gain the 'aesni' feature which increases crypto > > performance massively. > > I don't know what we should do, but the current code does this: > > (1) If a particular CPU model is specified by the source hypervisor > (eg VMware), use that.Reasonable, if there's a match> (2) If the guest is RHEL family >= 9, use -cpu host / host-passthrough > > (3) Otherwise don't specify any -cpu or <cpu> section > > The third option seems to cause qemu / libvirt to use qemu64. However > it has the advantage that the guest is migratable afterwards.I vaguely recall in previous discussion that we decided not to care if the guest is migratable or not, on the basis that wasn't really domain knowledge of v2v nor an user request. Thus the use of 'host-passthrough' rather than 'host-model' to give a "perfect" match for the host, rather than approximation.> There may be a case for changing this so: > > (a) Rule (2) is changed so we use host-model (for the libvirt target).All hinges on whether users expect the output of v2v to be migratable or not.> (b) Rule (3) is changed so we always specify a minimum CPU, eg. > for x86-64 guests, choose x86-64-v2 (is that an actual CPU model??)In the case of x86-64-v2 it is the QEMU Nehalem model is a perfect match, and is capable of running on both Intel and AMD hosts. This is mostly luck though. For x86-64-v3 / -v4 ABIs there is no viable model that will run on both AMD and Intel. So once there's a guest OS that mandates -v3, we'll be back to wanting host-model/passthrough.> (a) seems like a no-brainer. Not sure why we chose host-passthrough > instead of host-model? The explanation in the commit message is not > very convincing: > https://github.com/libguestfs/virt-v2v/commit/a3e45b3ea5b45e682cb4b7ad3a5646586c6c7886 > > I think (b) is fraught because it is my understanding that there is no > good choice for a minimum CPU since we don't know anything about the > target hardware (eg. if it's Intel or AMD). And on !x86-64 we really > have no idea at all what's a good choice.Yeah, I'd recommend against (b). IMHO mgmt apps / provisioning tools should only ever choose host-model or host-passthrough for KVM. Use of an explicit named CPU model should be something left to admins to decide based on their global knowledge of what machines they need to migrate across. Ignoring named models also nicely avoids the problem of needing knowledge across all arches. If you care about TCG, you could use 'maximum' instead of 'host-passthrough'. They are identical in the case of KVM, and for TCG 'maximum' just gives you all possible features. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|