Matthew Schumacher
2015-Apr-21 00:45 UTC
[libvirt-users] QemuDomainObjEndJob called when libvirtd is started and libvirt insists qemu is using the wrong disk source.
List, I was under the impression that I could restart libvirtd without it destroying my VMs, but am not finding that to be true. When I killall libvirtd then my VM's keep running, but then when I start libvirtd it calls qemuDomainObjEndJob:1542 : Stopping job: modify (async=none vm=0x7fb8cc0d8510 name=test) and my domain gets whacked. Any way to disable this behavior? Also, while I'm at it, due to issues with snapshotting, I ended up with two domains where libvirt insists the disk source is incorrect which causes many things to break like this: root@wasvirt1:/etc/libvirt# virsh domblklist wasdev Target Source ------------------------------------------------ vda /glustervol1/vm/wasdev/wasdev.reboot hdc /dev/sr0 Yet: root@wasvirt1:/etc/libvirt# lsof | grep wasdev 2235 /usr/bin/qemu-system-x86_64 /var/log/libvirt/qemu/wasdev.log 2235 /usr/bin/qemu-system-x86_64 /var/log/libvirt/qemu/wasdev.log 2235 /usr/bin/qemu-system-x86_64 /glustervol1/vm/wasdev/wasdev.qcow2 The reason is because a blockcommit --active --pivot doesn't work, I blockjob --abort, then try again, it works the second time, but then the disk is wrong. Help would be greatly appreciated. Thanks, schu
Laine Stump
2015-Apr-21 01:12 UTC
Re: [libvirt-users] QemuDomainObjEndJob called when libvirtd is started and libvirt insists qemu is using the wrong disk source.
On 04/20/2015 08:45 PM, Matthew Schumacher wrote:> List, > > I was under the impression that I could restart libvirtd without it > destroying my VMs, but am not finding that to be true.If not, then something is wrong.> When I killall > libvirtd then my VM's keep running, but then when I start libvirtd it > calls qemuDomainObjEndJob:1542 : Stopping job: modify (async=none > vm=0x7fb8cc0d8510 name=test)That message doesn't indicate that a domain is being destroyed, just that libvirt performed some operation on the domain that modified it in some way.> and my domain gets whacked.If your domain is terminated, there should be a libvirtd error log (at least one saying that it got end of file from the monitor, meaning that the qemu process itself terminated unexpectedly. In that case there should be some indication of qemu's problem at the end of /var/log/libvirt/qemu/test.log).> > Any way to disable this behavior?Whatever the bug is causing this to happen, fix it :-)> > Also, while I'm at it, due to issues with snapshotting, I ended up with > two domains where libvirt insists the disk source is incorrect which > causes many things to break like this: > > root@wasvirt1:/etc/libvirt# virsh domblklist wasdev > Target Source > ------------------------------------------------ > vda /glustervol1/vm/wasdev/wasdev.reboot > hdc /dev/sr0 > > Yet: > > root@wasvirt1:/etc/libvirt# lsof | grep wasdev > 2235 /usr/bin/qemu-system-x86_64 /var/log/libvirt/qemu/wasdev.log > 2235 /usr/bin/qemu-system-x86_64 /var/log/libvirt/qemu/wasdev.log > 2235 /usr/bin/qemu-system-x86_64 /glustervol1/vm/wasdev/wasdev.qcow2 > > The reason is because a blockcommit --active --pivot doesn't work, I > blockjob --abort, then try again, it works the second time, but then the > disk is wrong. > > Help would be greatly appreciated. > > Thanks, > schu > > _______________________________________________ > libvirt-users mailing list > libvirt-users@redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-users > >
Matthew Schumacher
2015-Apr-21 01:28 UTC
Re: [libvirt-users] QemuDomainObjEndJob called when libvirtd is started and libvirt insists qemu is using the wrong disk source.
On 04/20/2015 05:12 PM, Laine Stump wrote:> On 04/20/2015 08:45 PM, Matthew Schumacher wrote: >> List, >> >> I was under the impression that I could restart libvirtd without it >> destroying my VMs, but am not finding that to be true. > If not, then something is wrong. >Here are the logs from the vm: 2015-04-20 23:50:20.387+0000: shutting down qemu: terminating on signal 15 from pid 25298 And part of the logs from libvirt when I started it. Does this help at all? 2015-04-20 23:50:20.387+0000: 25373: error : virStorageFileBackendFileReadHeader:1443 : Failed to open file '/dev/sr0': No medium found 2015-04-20 23:50:20.387+0000: 25373: debug : virStorageFileReadHeader:2714 : read of storage header 0x7fb8cc0d75f0: ret=-1 2015-04-20 23:50:20.387+0000: 25373: debug : virStorageFileBackendFileDeinit:1369 : deinitializing FS storage file 0x7fb8cc0d75f0 (block:/dev/sr0) 2015-04-20 23:50:20.387+0000: 25373: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fb8cc0155b0 2015-04-20 23:50:20.387+0000: 25373: debug : qemuDomainObjEndJob:1542 : Stopping job: modify (async=none vm=0x7fb8cc0d8510 name=test) 2015-04-20 23:50:20.387+0000: 25373: info : virObjectRef:296 : OBJECT_REF: obj=0x7fb8cc0155b0 2015-04-20 23:50:20.387+0000: 25373: debug : qemuProcessStop:5089 : Shutting down vm=0x7fb8cc0d8510 name=test id=6 pid=25245 flags=0 2015-04-20 23:50:20.387+0000: 25373: info : virObjectRef:296 : OBJECT_REF: obj=0x7fb8cc0155b0 2015-04-20 23:50:20.387+0000: 25373: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fb8cc0155b0 2015-04-20 23:50:20.387+0000: 25373: debug : virFileClose:102 : Closed fd 19 2015-04-20 23:50:20.387+0000: 25373: info : qemuMonitorClose:904 : QEMU_MONITOR_CLOSE: mon=0x7fb8d0000bd0 refs=2 2015-04-20 23:50:20.387+0000: 25373: info : virEventPollRemoveHandle:186 : EVENT_POLL_REMOVE_HANDLE: watch=10 2015-04-20 23:50:20.387+0000: 25373: debug : virEventPollRemoveHandle:199 : mark delete 9 18 2015-04-20 23:50:20.387+0000: 25373: debug : virEventPollInterruptLocked:726 : Interrupting 2015-04-20 23:50:20.387+0000: 25373: debug : virFileClose:102 : Closed fd 18 2015-04-20 23:50:20.387+0000: 25373: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fb8d0000bd0 2015-04-20 23:50:20.387+0000: 25373: debug : qemuProcessKill:5047 : vm=0x7fb8cc0d8510 name=test pid=25245 flags=5 2015-04-20 23:50:20.387+0000: 25373: debug : virProcessKillPainfully:352 : vpid=25245 force=1 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollRunOnce:650 : Poll got 2 event(s) 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchTimeouts:432 : Dispatch 0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:478 : Dispatch 8 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=0 w=1 2015-04-20 23:50:20.387+0000: 25298: info : virEventPollDispatchHandles:506 : EVENT_POLL_DISPATCH_HANDLE: watch=1 events=1 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=1 w=2 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=4 w=5 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=5 w=6 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=7 w=8 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=8 w=9 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=9 w=10 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:495 : Skip deleted n=9 w=10 f=18 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollDispatchHandles:492 : i=10 w=11 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCleanupTimeouts:525 : Cleanup 0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCleanupTimeouts:561 : Found 0 out of 0 timeout slots used, releasing 0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCleanupHandles:574 : Cleanup 11 2015-04-20 23:50:20.387+0000: 25298: info : virEventPollCleanupHandles:587 : EVENT_POLL_PURGE_HANDLE: watch=10 2015-04-20 23:50:20.387+0000: 25298: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fb8d0000bd0 2015-04-20 23:50:20.387+0000: 25298: info : virObjectUnref:261 : OBJECT_DISPOSE: obj=0x7fb8d0000bd0 2015-04-20 23:50:20.387+0000: 25298: debug : qemuMonitorDispose:259 : mon=0x7fb8d0000bd0 2015-04-20 23:50:20.387+0000: 25298: info : virObjectUnref:259 : OBJECT_UNREF: obj=0x7fb8cc0d8510 2015-04-20 23:50:20.387+0000: 25298: debug : virEventRunDefaultImpl:305 : running default event implementation 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCleanupTimeouts:525 : Cleanup 0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCleanupTimeouts:561 : Found 0 out of 0 timeout slots used, releasing 0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCleanupHandles:574 : Cleanup 10 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=0 w=1, f=7 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=1 w=2, f=9 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=2 w=3, f=11 e=0 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=3 w=4, f=12 e=0 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=4 w=5, f=6 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=5 w=6, f=13 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=6 w=7, f=14 e=0 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=7 w=8, f=14 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=8 w=9, f=16 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollMakePollFDs:400 : Prepare n=9 w=11, f=20 e=1 d=0 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCalculateTimeout:338 : Calculate expiry of 0 timers 2015-04-20 23:50:20.387+0000: 25298: debug : virEventPollCalculateTimeout:370 : No timeout is pending 2015-04-20 23:50:20.387+0000: 25298: info : virEventPollRunOnce:640 : EVENT_POLL_RUN: nhandles=7 timeout=-1
Possibly Parallel Threads
- Re: QemuDomainObjEndJob called when libvirtd is started and libvirt insists qemu is using the wrong disk source.
- Re: QemuDomainObjEndJob called when libvirtd is started and libvirt insists qemu is using the wrong disk source.
- Re: QemuDomainObjEndJob called when libvirtd is started and libvirt insists qemu is using the wrong disk source.
- create vm timeout
- libvirt_lxc causes cpu 100% usage