Tanmoy Sinha
2018-Mar-23 06:10 UTC
[Libguestfs] Issue with libguestfs-test-tool on a guest hosted on VMWare ESXi
I am using a debian 9 guest, hosted on a ESXi platform with nested
virtualisation enabled.
On this debian 9 guest when I run libguesfs-test-tool, it fails with an
error:
"qemu-system-x86_64:
/build/qemu-DqynNa/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs:
Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed."
Instead when I use a wrapper script and hook it with the env
variable LIBGUESTFS_HV=/tmp/qemu.wrapper, things work fine, as in the
script I just spawn the guest with the specific machine type pc-i440fx-2.1
over KVM.
exec qemu-system-x86_64 -enable-kvm -machine *pc-i440fx-2.**1*,accel
=kvm,dump-guest-core=off "${args[@]}"
The script is attached as well as the libguestfs-test-tool logs.
Can anybody help me on this. Why is the latest machine type, i.e.
pc-i440fx-2.8 on my debian 9 guest not working out? If I do a force_tcg it
works even with the latest machine type, but that is not what I desire, I
don't intend libguestfs to run on emulation.
========================================================================================================================
System configuration:
*$> uname -a*
Linux XXXX 4.9.0-6-amd64 #1 SMP Debian 4.9.82-1+deb9u3 (2018-03-02) x86_64
GNU/Linux
*$> kvm -machine help*
Supported machines are:
pc Standard PC (i440FX + PIIX, 1996) (alias of
pc-i440fx-2.8)
pc-i440fx-2.8 Standard PC (i440FX + PIIX, 1996) (default)
pc-i440fx-2.7 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.6 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.5 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.4 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.3 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.2 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.1 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-2.0 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.7 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.6 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.5 Standard PC (i440FX + PIIX, 1996)
pc-i440fx-1.4 Standard PC (i440FX + PIIX, 1996)
pc-1.3 Standard PC (i440FX + PIIX, 1996)
pc-1.2 Standard PC (i440FX + PIIX, 1996)
pc-1.1 Standard PC (i440FX + PIIX, 1996)
pc-1.0 Standard PC (i440FX + PIIX, 1996)
pc-0.15 Standard PC (i440FX + PIIX, 1996)
pc-0.14 Standard PC (i440FX + PIIX, 1996)
pc-0.13 Standard PC (i440FX + PIIX, 1996)
pc-0.12 Standard PC (i440FX + PIIX, 1996)
pc-0.11 Standard PC (i440FX + PIIX, 1996)
pc-0.10 Standard PC (i440FX + PIIX, 1996)
q35 Standard PC (Q35 + ICH9, 2009) (alias of pc-q35-2.8)
pc-q35-2.8 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.7 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.6 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.5 Standard PC (Q35 + ICH9, 2009)
pc-q35-2.4 Standard PC (Q35 + ICH9, 2009)
isapc ISA-only PC
none empty machine
xenfv Xen Fully-virtualized PC
xenpv Xen Para-virtualized PC
*$> kvm -cpu help*
Available CPUs:
x86 486
x86 Broadwell-noTSX Intel Core Processor (Broadwell, no TSX)
x86 Broadwell Intel Core Processor (Broadwell)
x86 Conroe Intel Celeron_4x0 (Conroe/Merom Class Core 2)
x86 Haswell-noTSX Intel Core Processor (Haswell, no TSX)
x86 Haswell Intel Core Processor (Haswell)
x86 IvyBridge Intel Xeon E3-12xx v2 (Ivy Bridge)
x86 Nehalem Intel Core i7 9xx (Nehalem Class Core i7)
x86 Opteron_G1 AMD Opteron 240 (Gen 1 Class Opteron)
x86 Opteron_G2 AMD Opteron 22xx (Gen 2 Class Opteron)
x86 Opteron_G3 AMD Opteron 23xx (Gen 3 Class Opteron)
x86 Opteron_G4 AMD Opteron 62xx class CPU
x86 Opteron_G5 AMD Opteron 63xx class CPU
x86 Penryn Intel Core 2 Duo P9xxx (Penryn Class Core 2)
x86 SandyBridge Intel Xeon E312xx (Sandy Bridge)
x86 Skylake-Client Intel Core Processor (Skylake)
x86 Westmere Westmere E56xx/L56xx/X56xx (Nehalem-C)
x86 athlon QEMU Virtual CPU version 2.5+
x86 core2duo Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
x86 coreduo Genuine Intel(R) CPU T2600 @ 2.16GHz
x86 kvm32 Common 32-bit KVM processor
x86 kvm64 Common KVM processor
x86 n270 Intel(R) Atom(TM) CPU N270 @ 1.60GHz
x86 pentium
x86 pentium2
x86 pentium3
x86 phenom AMD Phenom(tm) 9550 Quad-Core Processor
x86 qemu32 QEMU Virtual CPU version 2.5+
x86 qemu64 QEMU Virtual CPU version 2.5+
x86 host KVM processor with all supported host features (only
available in KVM mode)
Recognized CPUID flags:
fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36
pn clflush ds acpi mmx fxsr sse sse2 ss ht tm ia64 pbe
pni pclmulqdq dtes64 monitor ds-cpl vmx smx est tm2 ssse3 cid fma cx16
xtpr pdcm pcid dca sse4.1 sse4.2 x2apic movbe popcnt tsc-deadline aes xsave
osxsave avx f16c rdrand hypervisor
fsgsbase tsc-adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f
avx512dq rdseed adx smap avx512ifma pcommit clflushopt clwb avx512pf
avx512er avx512cd avx512bw avx512vl
avx512vbmi umip pku ospke rdpid
avx512-4vnniw avx512-4fmaps
syscall nx mmxext fxsr-opt pdpe1gb rdtscp lm 3dnowext 3dnow
lahf-lm cmp-legacy svm extapic cr8legacy abm sse4a misalignsse
3dnowprefetch osvw ibs xop skinit wdt lwp fma4 tce nodeid-msr tbm topoext
perfctr-core perfctr-nb
invtsc
xstore xstore-en xcrypt xcrypt-en ace2 ace2-en phe phe-en pmm pmm-en
kvmclock kvm-nopiodelay kvm-mmu kvmclock kvm-asyncpf kvm-steal-time
kvm-pv-eoi kvm-pv-unhalt kvmclock-stable-bit
npt lbrv svm-lock nrip-save tsc-scale vmcb-clean flushbyasid
decodeassists pause-filter pfthreshold
xsaveopt xsavec xgetbv1 xsaves
arat
Pino Toscano
2018-Mar-23 09:07 UTC
Re: [Libguestfs] Issue with libguestfs-test-tool on a guest hosted on VMWare ESXi
On Friday, 23 March 2018 07:10:00 CET Tanmoy Sinha wrote:> I am using a debian 9 guest, hosted on a ESXi platform with nested > virtualisation enabled. > > On this debian 9 guest when I run libguesfs-test-tool, it fails with an > error: > > "qemu-system-x86_64: > /build/qemu-DqynNa/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs: > Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed."See also https://bugzilla.redhat.com/show_bug.cgi?id=1456407 This is basically a bug in VMware and the qemu bug https://bugs.launchpad.net/qemu/+bug/1661386 has a couple of hints about what it can be changed in VMware to avoid this situation. -- Pino Toscano
Tanmoy Sinha
2018-Mar-23 17:35 UTC
Re: [Libguestfs] Issue with libguestfs-test-tool on a guest hosted on VMWare ESXi
Thanks. Reading the defect and the associated thread in
https://bugs.launchpad.net/qemu/+bug/1661386, I enabled performance
counters in VMWare guest settings. Now the kvm assertion `ret
=cpu->kvm_msr_buf->nmsrs' is gone. But the qemu/kvm hangs after
printing
"SeaBIOS (version 1.10.2-1)"
I presume this too is a qemu/kvm issue for vmware platforms. Is there any
existing issue reported?
Regards,
Tanmoy Sinha
============================================
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu x86_64
libguestfs: command: run: \ /usr/lib/x86_64-linux-gnu/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.1.17
supermin: package handler: debian/dpkg
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: if-newer: output does not need rebuilding
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: checking for previously cached test results of
/usr/bin/qemu-system-x86_64, in /var/tmp/.guestfs-0
libguestfs: loading previously cached test results
libguestfs: qemu version 2.8
libguestfs: finished testing qemu features
[00012ms] /usr/bin/qemu-system-x86_64 \
-global virtio-blk-pci.scsi=off \
-nodefconfig \
-enable-fips \
-nodefaults \
-display none \
-machine accel=kvm:tcg \
-cpu host \
-m 500 \
-no-reboot \
-rtc driftfix=slew \
-no-hpet \
-global kvm-pit.lost_tick_policy=discard \
-kernel /var/tmp/.guestfs-0/appliance.d/kernel \
-initrd /var/tmp/.guestfs-0/appliance.d/initrd \
-object rng-random,filename=/dev/urandom,id=rng0 \
-device virtio-rng-pci,rng=rng0 \
-device virtio-scsi-pci,id=scsi \
-drive
file=/tmp/libguestfswJEdlG/scratch.1,cache=unsafe,format=raw,id=hd0,if=none
\
-device scsi-hd,drive=hd0 \
-drive
file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none,format=raw
\
-device scsi-hd,drive=appliance \
-device virtio-serial-pci \
-serial stdio \
-device sga \
-chardev socket,path=/tmp/libguestfsj95RNX/guestfsd.sock,id=channel0 \
-device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
-append 'panic=1 console=ttyS0 edd=off udevtimeout=6000
udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory
usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=/dev/sdb
selinux=0 guestfs_verbose=1 TERM=screen'
libguestfs: responding to serial console Device Status Report
\x1b[1;256r\x1b[256;256H\x1b[6n
Google, Inc.
Serial Graphics Adapter 12/07/13
SGABIOS $Id: sgabios.S 8 2010-04-22 00:03:40Z nlaredo $ (mjt@gandalf) Sat
Dec 7 15:39:53 UTC 2013
Term: 80x24
4 0
SeaBIOS (version 1.10.2-1)
On Fri, 23 Mar 2018 at 14:37 Pino Toscano <ptoscano@redhat.com> wrote:
> On Friday, 23 March 2018 07:10:00 CET Tanmoy Sinha wrote:
> > I am using a debian 9 guest, hosted on a ESXi platform with nested
> > virtualisation enabled.
> >
> > On this debian 9 guest when I run libguesfs-test-tool, it fails with
an
> > error:
> >
> > "qemu-system-x86_64:
> > /build/qemu-DqynNa/qemu-2.8+dfsg/target-i386/kvm.c:1805: kvm_put_msrs:
> > Assertion `ret == cpu->kvm_msr_buf->nmsrs' failed."
>
> See also https://bugzilla.redhat.com/show_bug.cgi?id=1456407
> This is basically a bug in VMware and the qemu bug
> https://bugs.launchpad.net/qemu/+bug/1661386
> has a couple of hints about what it can be changed in VMware to avoid
> this situation.
>
> --
> Pino Toscano