[moderator note: I'm forwarding a stripped down version of the original mail which was rejected in the moderator queue. I stripped the 3.3 megabyte .tar.bz2 of the log file attachment, which is inappropriate for a technical list. Either trim the log to the relevant portion, or host the log externally and have your list email merely give a URL of the externally-hosted file]> ForwardedMessage.eml > > Subject: > Tunnelled migrate Windows7 VMs halted > From: > 邓林文 <dlworld0@163.com> > Date: > 04/25/2017 11:12 PM > > To: > libvirt-users@redhat.com > > > > I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted on the target although the status is running. > > > [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system > > > But when migrated with qemu native mode, the VM runs well. > > > [root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system > > > System Info: > Release: Centos 7.2 > Kernel: 3.10.0-327.28.3.el7.x86_64 > Qemu: qemu-kvm-rhev-2.3.0 > Libvirt: libvirt-1.2.17/libvirt-2.0.0 > CPU: AMD Opteron(TM) Processor 6212 > > > As CPUFrequency may lead windows migrate halt, I have disabled Power Management. > > > Does anyone have some sugestions? > > > Thanks, > Linwen Deng > > > vm.xml > > > <domain type='kvm' id='8'> > <name>i-000000ac</name> > <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> > <metadata> > <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> > <nova:package version="12.0.0-4"/> > <nova:name>c7-vm15-test15-7</nova:name> > <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> > <nova:flavor name="oeflavor-4-2048-20"> > <nova:memory>2048</nova:memory> > <nova:disk>20</nova:disk> > <nova:swap>0</nova:swap> > <nova:ephemeral>0</nova:ephemeral> > <nova:vcpus>4</nova:vcpus> > </nova:flavor> > <nova:owner> > <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> > <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> > </nova:owner> > </nova:instance> > </metadata> > <memory unit='KiB'>2097152</memory> > <currentMemory unit='KiB'>2097152</currentMemory> > <memoryBacking> > <hugepages> > <page size='2048' unit='KiB' nodeset='0'/> > </hugepages> > </memoryBacking> > <vcpu placement='static'>2</vcpu> > <cputune> > <shares>4096</shares> > <vcpupin vcpu='0' cpuset='4-7'/> > <vcpupin vcpu='1' cpuset='4-7'/> > <emulatorpin cpuset='4-7'/> > </cputune> > <resource> > <partition>/machine</partition> > </resource> > <sysinfo type='smbios'> > <system> > <entry name='manufacturer'>Fedora Project</entry> > <entry name='product'>OpenStack Nova</entry> > <entry name='version'>12.0.0-4</entry> > <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> > <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> > <entry name='family'>Virtual Machine</entry> > </system> > </sysinfo> > <os> > <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> > <boot dev='cdrom'/> > <boot dev='hd'/> > <boot dev='fd'/> > <smbios mode='sysinfo'/> > </os> > <features> > <acpi/> > <apic/> > </features> > <cpu mode='host-model'> > <model fallback='allow'/> > <topology sockets='2' cores='1' threads='1'/> > </cpu> > <clock offset='utc'> > <timer name='pit' tickpolicy='delay'/> > <timer name='rtc' tickpolicy='catchup'/> > <timer name='hpet' present='no'/> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <devices> > <emulator>/usr/libexec/qemu-kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='qcow2' cache='none'/> > <source file='/opt/ssd/win7.qcow2'/> > <backingStore type='file' index='1'> > <format type='raw'/> > <source file='/opt/ssd/_base/win7.qcow2'/> > <backingStore/> > </backingStore> > <target dev='vda' bus='virtio'/> > <serial>8164a82d-6066-46d7-8a92-8391620fc58c</serial> > <alias name='virtio-disk0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > </disk> > <controller type='usb' index='0' model='ich9-ehci1'> > <alias name='usb'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x7'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci1'> > <alias name='usb'/> > <master startport='0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0' multifunction='on'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci2'> > <alias name='usb'/> > <master startport='2'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x1'/> > </controller> > <controller type='usb' index='0' model='ich9-uhci3'> > <alias name='usb'/> > <master startport='4'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x2'/> > </controller> > <controller type='pci' index='0' model='pci-root'> > <alias name='pci.0'/> > </controller> > <controller type='virtio-serial' index='0'> > <alias name='virtio-serial0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/> > </controller> > <serial type='file'> > <source path='/var/lib/nova/instances/53f0710f-b25e-4f47-a7cf-c15a9409fdc3/console.log'/> > <target port='0'/> > <alias name='serial0'/> > </serial> > <serial type='pty'> > <source path='/dev/pts/2'/> > <target port='1'/> > <alias name='serial1'/> > </serial> > <console type='file'> > <source path='/var/lib/nova/instances/53f0710f-b25e-4f47-a7cf-c15a9409fdc3/console.log'/> > <target type='serial' port='0'/> > <alias name='serial0'/> > </console> > <channel type='spicevmc'> > <target type='virtio' name='com.redhat.spice.0' state='connected'/> > <alias name='channel0'/> > <address type='virtio-serial' controller='0' bus='0' port='1'/> > </channel> > <input type='tablet' bus='usb'> > <alias name='input0'/> > </input> > <input type='mouse' bus='ps2'> > <alias name='input1'/> > </input> > <input type='keyboard' bus='ps2'> > <alias name='input2'/> > </input> > <graphics type='vnc' port='5900' autoport='yes' listen='0.0.0.0' keymap='en-us'> > <listen type='address' address='0.0.0.0'/> > </graphics> > <graphics type='spice' port='5901' autoport='yes' listen='0.0.0.0' keymap='en-us'> > <listen type='address' address='0.0.0.0'/> > <image compression='auto_glz'/> > <streaming mode='filter'/> > <mouse mode='client'/> > </graphics> > <sound model='ich6'> > <alias name='sound0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> > </sound> > <video> > <model type='qxl' ram='65536' vram='65536' vgamem='16384' primary='yes'/> > <alias name='video0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/> > </video> > <memballoon model='virtio'> > <stats period='10'/> > <alias name='balloon0'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > </memballoon> > </devices> > <seclabel type='none' model='none'/> > <seclabel type='dynamic' model='dac' relabel='yes'> > <label>+0:+107</label> > <imagelabel>+0:+107</imagelabel> > </seclabel> > </domain> >-- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org
Daniel P. Berrange
2017-Apr-26 13:59 UTC
Re: [libvirt-users] Tunnelled migrate Windows7 VMs halted
On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote:> > > > I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted > > on the target although the status is running.What do you mean by halted ? The guest OS has shutdown, or QEMU has crashed, or something else ?> > > > > > [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system > > > > > > But when migrated with qemu native mode, the VM runs well. > > > > > > [root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system>From QEMU's POV there's no difference between these modes - in both casesQEMU is just getting a file descriptor from libvirt. So any problems with the guest after migration are> > System Info: > > Release: Centos 7.2 > > Kernel: 3.10.0-327.28.3.el7.x86_64 > > Qemu: qemu-kvm-rhev-2.3.0 > > Libvirt: libvirt-1.2.17/libvirt-2.0.0Does the 2 libvirt versions mean you are live-migrating between two different versions of CentOS ? If so, this also likely means two different versions of QEMU involved.> > CPU: AMD Opteron(TM) Processor 6212 > > > > > > As CPUFrequency may lead windows migrate halt, I have disabled Power Management. > > > > > > Does anyone have some sugestions? > > > > > > Thanks, > > Linwen Deng > > > > > > vm.xml > > > > > > <domain type='kvm' id='8'> > > <name>i-000000ac</name> > > <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> > > <metadata> > > <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> > > <nova:package version="12.0.0-4"/> > > <nova:name>c7-vm15-test15-7</nova:name> > > <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> > > <nova:flavor name="oeflavor-4-2048-20"> > > <nova:memory>2048</nova:memory> > > <nova:disk>20</nova:disk> > > <nova:swap>0</nova:swap> > > <nova:ephemeral>0</nova:ephemeral> > > <nova:vcpus>4</nova:vcpus> > > </nova:flavor> > > <nova:owner> > > <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> > > <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> > > </nova:owner> > > </nova:instance> > > </metadata> > > <memory unit='KiB'>2097152</memory> > > <currentMemory unit='KiB'>2097152</currentMemory> > > <memoryBacking> > > <hugepages> > > <page size='2048' unit='KiB' nodeset='0'/> > > </hugepages> > > </memoryBacking> > > <vcpu placement='static'>2</vcpu> > > <cputune> > > <shares>4096</shares> > > <vcpupin vcpu='0' cpuset='4-7'/> > > <vcpupin vcpu='1' cpuset='4-7'/> > > <emulatorpin cpuset='4-7'/> > > </cputune> > > <resource> > > <partition>/machine</partition> > > </resource> > > <sysinfo type='smbios'> > > <system> > > <entry name='manufacturer'>Fedora Project</entry> > > <entry name='product'>OpenStack Nova</entry> > > <entry name='version'>12.0.0-4</entry> > > <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> > > <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> > > <entry name='family'>Virtual Machine</entry> > > </system> > > </sysinfo> > > <os> > > <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> > > <boot dev='cdrom'/> > > <boot dev='hd'/> > > <boot dev='fd'/> > > <smbios mode='sysinfo'/> > > </os> > > <features> > > <acpi/> > > <apic/> > > </features> > > <cpu mode='host-model'> > > <model fallback='allow'/> > > <topology sockets='2' cores='1' threads='1'/> > > </cpu> > > <clock offset='utc'> > > <timer name='pit' tickpolicy='delay'/> > > <timer name='rtc' tickpolicy='catchup'/> > > <timer name='hpet' present='no'/> > > </clock>If nothing else, this guest is incorrectly configured for running the Windows OS. I suspect you forget to set the Glance image property needed to tell Nova that it is windows. Specifically it is missing hyperv paravirt <timer>, and many hyperv paravirt <feature> elements. This will make Windows guests unstable and liable to hit BSOD Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
At 2017-04-26 21:59:46, "Daniel P. Berrange" <berrange@redhat.com> wrote:>On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote: >> > >> > I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted >> > on the target although the status is running. > >What do you mean by halted ? The guest OS has shutdown, or QEMU >has crashed, or something else ? >I can still connect to the guest OS and see the windows opened before migrate on the target, but cann't click any more, and there's no effect when send "ctrl+alt+del" keys by virsh.>> > >> > >> > [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system >> > >> > >> > But when migrated with qemu native mode, the VM runs well. >> > >> > >> > [root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system > >From QEMU's POV there's no difference between these modes - in both cases >QEMU is just getting a file descriptor from libvirt. > >So any problems with the guest after migration are>When I destroy the guest and restart, the guest OS works well. The attachment is a stripped version log.>> > System Info: >> > Release: Centos 7.2 >> > Kernel: 3.10.0-327.28.3.el7.x86_64 >> > Qemu: qemu-kvm-rhev-2.3.0 >> > Libvirt: libvirt-1.2.17/libvirt-2.0.0 > >Does the 2 libvirt versions mean you are live-migrating between two different >versions of CentOS ? If so, this also likely means two different versions >of QEMU involved. > >> > CPU: AMD Opteron(TM) Processor 6212 >> > >> > >> > As CPUFrequency may lead windows migrate halt, I have disabled Power Management. >> > >> > >> > Does anyone have some sugestions? >> > >> > >> > Thanks, >> > Linwen Deng >> > >> > >> > vm.xml >> > >> > >> > <domain type='kvm' id='8'> >> > <name>i-000000ac</name> >> > <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> >> > <metadata> >> > <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> >> > <nova:package version="12.0.0-4"/> >> > <nova:name>c7-vm15-test15-7</nova:name> >> > <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> >> > <nova:flavor name="oeflavor-4-2048-20"> >> > <nova:memory>2048</nova:memory> >> > <nova:disk>20</nova:disk> >> > <nova:swap>0</nova:swap> >> > <nova:ephemeral>0</nova:ephemeral> >> > <nova:vcpus>4</nova:vcpus> >> > </nova:flavor> >> > <nova:owner> >> > <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> >> > <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> >> > </nova:owner> >> > </nova:instance> >> > </metadata> >> > <memory unit='KiB'>2097152</memory> >> > <currentMemory unit='KiB'>2097152</currentMemory> >> > <memoryBacking> >> > <hugepages> >> > <page size='2048' unit='KiB' nodeset='0'/> >> > </hugepages> >> > </memoryBacking> >> > <vcpu placement='static'>2</vcpu> >> > <cputune> >> > <shares>4096</shares> >> > <vcpupin vcpu='0' cpuset='4-7'/> >> > <vcpupin vcpu='1' cpuset='4-7'/> >> > <emulatorpin cpuset='4-7'/> >> > </cputune> >> > <resource> >> > <partition>/machine</partition> >> > </resource> >> > <sysinfo type='smbios'> >> > <system> >> > <entry name='manufacturer'>Fedora Project</entry> >> > <entry name='product'>OpenStack Nova</entry> >> > <entry name='version'>12.0.0-4</entry> >> > <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> >> > <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> >> > <entry name='family'>Virtual Machine</entry> >> > </system> >> > </sysinfo> >> > <os> >> > <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> >> > <boot dev='cdrom'/> >> > <boot dev='hd'/> >> > <boot dev='fd'/> >> > <smbios mode='sysinfo'/> >> > </os> >> > <features> >> > <acpi/> >> > <apic/> >> > </features> >> > <cpu mode='host-model'> >> > <model fallback='allow'/> >> > <topology sockets='2' cores='1' threads='1'/> >> > </cpu> >> > <clock offset='utc'> >> > <timer name='pit' tickpolicy='delay'/> >> > <timer name='rtc' tickpolicy='catchup'/> >> > <timer name='hpet' present='no'/> >> > </clock> > >If nothing else, this guest is incorrectly configured for running >the Windows OS. I suspect you forget to set the Glance image >property needed to tell Nova that it is windows. > >Specifically it is missing hyperv paravirt <timer>, and many hyperv >paravirt <feature> elements. > >This will make Windows guests unstable and liable to hit BSOD > > >Regards, >Daniel >-- >|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| >|: https://libvirt.org -o- https://fstop138.berrange.com :| >|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
At 2017-04-26 21:59:46, "Daniel P. Berrange" <berrange@redhat.com> wrote:>On Wed, Apr 26, 2017 at 08:51:39AM -0500, Eric Blake wrote: >> > >> > I migrated a Windows 7 VM with libvirtd tunnelled, the VM halted >> > on the target although the status is running. > >What do you mean by halted ? The guest OS has shutdown, or QEMU >has crashed, or something else ? > >> > >> > >> > [root@test15 ~]# virsh migrate --live --p2p --tunnelled i-000000ac qemu+tcp://192.168.65.13/system >> > >> > >> > But when migrated with qemu native mode, the VM runs well. >> > >> > >> > [root@test15 ~]# virsh migrate --live --p2p i-000000ac qemu+tcp://192.168.65.13/system > >From QEMU's POV there's no difference between these modes - in both cases >QEMU is just getting a file descriptor from libvirt. > >So any problems with the guest after migration are > >> > System Info: >> > Release: Centos 7.2 >> > Kernel: 3.10.0-327.28.3.el7.x86_64 >> > Qemu: qemu-kvm-rhev-2.3.0 >> > Libvirt: libvirt-1.2.17/libvirt-2.0.0 > >Does the 2 libvirt versions mean you are live-migrating between two different >versions of CentOS ? If so, this also likely means two different versions >of QEMU involved.>Migrate between 2 same version, and tried to upgrade the libvirt version.>> > CPU: AMD Opteron(TM) Processor 6212 >> > >> > >> > As CPUFrequency may lead windows migrate halt, I have disabled Power Management. >> > >> > >> > Does anyone have some sugestions? >> > >> > >> > Thanks, >> > Linwen Deng >> > >> > >> > vm.xml >> > >> > >> > <domain type='kvm' id='8'> >> > <name>i-000000ac</name> >> > <uuid>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</uuid> >> > <metadata> >> > <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0"> >> > <nova:package version="12.0.0-4"/> >> > <nova:name>c7-vm15-test15-7</nova:name> >> > <nova:creationTime>2017-04-14 06:48:54</nova:creationTime> >> > <nova:flavor name="oeflavor-4-2048-20"> >> > <nova:memory>2048</nova:memory> >> > <nova:disk>20</nova:disk> >> > <nova:swap>0</nova:swap> >> > <nova:ephemeral>0</nova:ephemeral> >> > <nova:vcpus>4</nova:vcpus> >> > </nova:flavor> >> > <nova:owner> >> > <nova:user uuid="b0f21b1d16b147ffb3f7713716cc894a">admin</nova:user> >> > <nova:project uuid="ae883f160c3c41db850d5cde8de8208b">service</nova:project> >> > </nova:owner> >> > </nova:instance> >> > </metadata> >> > <memory unit='KiB'>2097152</memory> >> > <currentMemory unit='KiB'>2097152</currentMemory> >> > <memoryBacking> >> > <hugepages> >> > <page size='2048' unit='KiB' nodeset='0'/> >> > </hugepages> >> > </memoryBacking> >> > <vcpu placement='static'>2</vcpu> >> > <cputune> >> > <shares>4096</shares> >> > <vcpupin vcpu='0' cpuset='4-7'/> >> > <vcpupin vcpu='1' cpuset='4-7'/> >> > <emulatorpin cpuset='4-7'/> >> > </cputune> >> > <resource> >> > <partition>/machine</partition> >> > </resource> >> > <sysinfo type='smbios'> >> > <system> >> > <entry name='manufacturer'>Fedora Project</entry> >> > <entry name='product'>OpenStack Nova</entry> >> > <entry name='version'>12.0.0-4</entry> >> > <entry name='serial'>95637afe-453f-42a8-b198-df673ab59c91</entry> >> > <entry name='uuid'>53f0710f-b25e-4f47-a7cf-c15a9409fdc3</entry> >> > <entry name='family'>Virtual Machine</entry> >> > </system> >> > </sysinfo> >> > <os> >> > <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> >> > <boot dev='cdrom'/> >> > <boot dev='hd'/> >> > <boot dev='fd'/> >> > <smbios mode='sysinfo'/> >> > </os> >> > <features> >> > <acpi/> >> > <apic/> >> > </features> >> > <cpu mode='host-model'> >> > <model fallback='allow'/> >> > <topology sockets='2' cores='1' threads='1'/> >> > </cpu> >> > <clock offset='utc'> >> > <timer name='pit' tickpolicy='delay'/> >> > <timer name='rtc' tickpolicy='catchup'/> >> > <timer name='hpet' present='no'/> >> > </clock> > >If nothing else, this guest is incorrectly configured for running >the Windows OS. I suspect you forget to set the Glance image >property needed to tell Nova that it is windows. > >Specifically it is missing hyperv paravirt <timer>, and many hyperv >paravirt <feature> elements. > >This will make Windows guests unstable and liable to hit BSOD>I have tried to update the xml, the problem also exists, although there's one success in 5 tries. But the guest can accept ctrl-alt-delete, further operations still not available. <features> <acpi/> <apic/> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> </hyperv> </features> <cpu mode='host-model'> <model fallback='allow'/> <topology sockets='1' cores='2' threads='1'/> </cpu> <clock offset='utc'> <timer name='pit' tickpolicy='delay'/> <timer name='rtc' tickpolicy='catchup'/> <timer name='hpet' present='no'/> <timer name='hypervclock' present='yes'/> </clock>> >Regards, >Daniel >-- >|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| >|: https://libvirt.org -o- https://fstop138.berrange.com :| >|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|