> On May 16, 2017, at 3:49 PM, Martin Kletzander <mkletzan@redhat.com> wrote: > > On Tue, May 16, 2017 at 06:41:05PM +0000, Innus, Martins wrote: >> Hi, >> >> Running on: >> >> $ cat /etc/redhat-release >> CentOS Linux release 7.3.1611 (Core) >> >> And: >> >> $ rpm -qa |grep libvirt >> libvirt-daemon-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-secret-2.0.0-10.el7_3.5.x86_64 >> libvirt-client-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-network-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-storage-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-config-network-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.5.x86_64 >> libvirt-2.0.0-10.el7_3.5.x86_64 >> libvirt-python-2.0.0-2.el7.x86_64 >> libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-lxc-2.0.0-10.el7_3.5.x86_64 >> libvirt-daemon-driver-interface-2.0.0-10.el7_3.5.x86_64 >> >> I’m seeing duplicated reboot events when using the python event api. Use is simplified down to the simple test case attached. Running with that results in: >> >> $ sudo ./libvirt_events_single.py >> Reboot: Domain i-06945b37(21) >> 1494958504.72 >> Reboot: Domain i-06945b37(21) >> 1494958504.74 >> > > Can you try with newer libvirt?I’m not sure. I will have to see if I can build it, unless there is a repo somewhere with centos7 rpms. I assume by this you mean that you don’t see this with current libvirt?> Does this happen with 'virsh event > --loop --event lifecycle’?“lifecycle” shows no output, but with “reboot” or “all”: $ sudo virsh event --loop --all event 'reboot' for domain i-06945b37 event 'reboot' for domain i-06945b37> > Is there anything fishy going on when looking at the console (VN/spice) > during that reboot?Don’t see anything unusual. I can send you the full console output if you want, but I don’t see anything strange. Thanks Martins> >> This is reproducible 100% of the time by doing /sbin/reboot from within a VM started with qemu-kvm. Any ideas on whats going on? >> >> Let me know if any other information would be useful. >> >> Thanks >> >> Martins >> > >> #!/usr/bin/env python >> >> import libvirt >> import time >> >> def RBcallback(conn, dom, opaque): >> print "Reboot: Domain %s(%s)" % (dom.name(), dom.ID()) >> print time.time() >> >> if __name__ == '__main__': >> >> libvirt.virEventRegisterDefaultImpl() >> >> conn = libvirt.openReadOnly('qemu:///system') >> >> conn.domainEventRegisterAny(None, libvirt.VIR_DOMAIN_EVENT_ID_REBOOT, RBcallback, None) >> >> while True: >> libvirt.virEventRunDefaultImpl() > >> _______________________________________________ >> libvirt-users mailing list >> libvirt-users@redhat.com >> https://www.redhat.com/mailman/listinfo/libvirt-users
On Tue, May 16, 2017 at 08:11:54PM +0000, Innus, Martins wrote:> >> On May 16, 2017, at 3:49 PM, Martin Kletzander <mkletzan@redhat.com> wrote: >> >> On Tue, May 16, 2017 at 06:41:05PM +0000, Innus, Martins wrote: >>> Hi, >>> >>> Running on: >>> >>> $ cat /etc/redhat-release >>> CentOS Linux release 7.3.1611 (Core) >>> >>> And: >>> >>> $ rpm -qa |grep libvirt >>> libvirt-daemon-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-secret-2.0.0-10.el7_3.5.x86_64 >>> libvirt-client-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-network-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-storage-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-config-network-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.5.x86_64 >>> libvirt-2.0.0-10.el7_3.5.x86_64 >>> libvirt-python-2.0.0-2.el7.x86_64 >>> libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-lxc-2.0.0-10.el7_3.5.x86_64 >>> libvirt-daemon-driver-interface-2.0.0-10.el7_3.5.x86_64 >>> >>> I’m seeing duplicated reboot events when using the python event api. Use is simplified down to the simple test case attached. Running with that results in: >>> >>> $ sudo ./libvirt_events_single.py >>> Reboot: Domain i-06945b37(21) >>> 1494958504.72 >>> Reboot: Domain i-06945b37(21) >>> 1494958504.74 >>> >> >> Can you try with newer libvirt? > >I’m not sure. I will have to see if I can build it, unless there is a repo somewhere with centos7 rpms. I assume by this you mean that you don’t see this with current libvirt? > >> Does this happen with 'virsh event >> --loop --event lifecycle’? > >“lifecycle” shows no output, but with “reboot” or “all”: > >$ sudo virsh event --loop --all >event 'reboot' for domain i-06945b37 >event 'reboot' for domain i-06945b37 > >> >> Is there anything fishy going on when looking at the console (VN/spice) >> during that reboot? > > >Don’t see anything unusual. I can send you the full console output if you want, but I don’t see anything strange. >I meant the graphical terminal, but that won't help much. I tried gathering as much info. So we just need to figure out whether QEMU sends us the event multiple times. Could you either set up debug logs [1] and then look for that in them or use the qemu-monitor.stp systemtap script to show you what's happening on the monitor? If it comes from qemu two times, than there's not much we can do about it. Martin [1] http://wiki.libvirt.org/page/DebugLogs
> On May 16, 2017, at 4:28 PM, Martin Kletzander <mkletzan@redhat.com> wrote: > > On Tue, May 16, 2017 at 08:11:54PM +0000, Innus, Martins wrote: >> >>> On May 16, 2017, at 3:49 PM, Martin Kletzander <mkletzan@redhat.com> wrote: >>> >>> On Tue, May 16, 2017 at 06:41:05PM +0000, Innus, Martins wrote: >>>> Hi, >>>> >>>> Running on: >>>> >>>> $ cat /etc/redhat-release >>>> CentOS Linux release 7.3.1611 (Core) >>>> >>>> And: >>>> >>>> $ rpm -qa |grep libvirt >>>> libvirt-daemon-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-qemu-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-secret-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-client-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-network-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-storage-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-config-network-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-python-2.0.0-2.el7.x86_64 >>>> libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-lxc-2.0.0-10.el7_3.5.x86_64 >>>> libvirt-daemon-driver-interface-2.0.0-10.el7_3.5.x86_64 >>>> >>>> I’m seeing duplicated reboot events when using the python event api. Use is simplified down to the simple test case attached. Running with that results in: >>>> >>>> $ sudo ./libvirt_events_single.py >>>> Reboot: Domain i-06945b37(21) >>>> 1494958504.72 >>>> Reboot: Domain i-06945b37(21) >>>> 1494958504.74 >>>> >>> >>> Can you try with newer libvirt? >> >> I’m not sure. I will have to see if I can build it, unless there is a repo somewhere with centos7 rpms. I assume by this you mean that you don’t see this with current libvirt? >> >>> Does this happen with 'virsh event >>> --loop --event lifecycle’? >> >> “lifecycle” shows no output, but with “reboot” or “all”: >> >> $ sudo virsh event --loop --all >> event 'reboot' for domain i-06945b37 >> event 'reboot' for domain i-06945b37 >> >>> >>> Is there anything fishy going on when looking at the console (VN/spice) >>> during that reboot? >> >> >> Don’t see anything unusual. I can send you the full console output if you want, but I don’t see anything strange. >> > > I meant the graphical terminal, but that won't help much. I tried > gathering as much info. So we just need to figure out whether QEMU > sends us the event multiple times. Could you either set up debug logs > [1] and then look for that in them or use the qemu-monitor.stp systemtap > script to show you what's happening on the monitor? If it comes from > qemu two times, than there's not much we can do about it. >So I guess with: $ sudo virsh qemu-monitor-event i-06945b37 --pretty --loop event RESET at 1495030343.874769 for domain i-06945b37: <null> event RESET at 1495030343.896221 for domain i-06945b37: <null> And the following in the debug logs: 2017-05-17 14:21:03.369+0000: 31471: info : qemuMonitorJSONIOProcessLine:206 : QEMU_MONITOR_RECV_EVENT: mon=0x7f5a24000f50 event={"timestamp": {"seconds": 1495030863, "microseconds": 369485}, "event": "RESET"} 2017-05-17 14:21:03.384+0000: 31471: info : qemuMonitorJSONIOProcessLine:206 : QEMU_MONITOR_RECV_EVENT: mon=0x7f5a24000f50 event={"timestamp": {"seconds": 1495030863, "microseconds": 383626}, "event": "RESET”} That means its coming from QEMU right? Any suggestions besides bugging the QEMU folks? Thanks Martins
On May 16, 2017, at 4:28 PM, Martin Kletzander <mkletzan@redhat.com<mailto:mkletzan@redhat.com>> wrote: On Tue, May 16, 2017 at 08:11:54PM +0000, Innus, Martins wrote: On May 16, 2017, at 3:49 PM, Martin Kletzander <mkletzan@redhat.com<mailto:mkletzan@redhat.com>> wrote: On Tue, May 16, 2017 at 06:41:05PM +0000, Innus, Martins wrote: Hi, Running on: $ cat /etc/redhat-release CentOS Linux release 7.3.1611 (Core) And: $ rpm -qa |grep libvirt libvirt-daemon-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-qemu-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-secret-2.0.0-10.el7_3.5.x86_64 libvirt-client-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-network-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-storage-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-config-nwfilter-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-config-network-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-nodedev-2.0.0-10.el7_3.5.x86_64 libvirt-2.0.0-10.el7_3.5.x86_64 libvirt-python-2.0.0-2.el7.x86_64 libvirt-daemon-driver-nwfilter-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-lxc-2.0.0-10.el7_3.5.x86_64 libvirt-daemon-driver-interface-2.0.0-10.el7_3.5.x86_64 I’m seeing duplicated reboot events when using the python event api. Use is simplified down to the simple test case attached. Running with that results in: $ sudo ./libvirt_events_single.py Reboot: Domain i-06945b37(21) 1494958504.72 Reboot: Domain i-06945b37(21) 1494958504.74 Can you try with newer libvirt? I’m not sure. I will have to see if I can build it, unless there is a repo somewhere with centos7 rpms. I assume by this you mean that you don’t see this with current libvirt? Does this happen with 'virsh event --loop --event lifecycle’? “lifecycle” shows no output, but with “reboot” or “all”: $ sudo virsh event --loop --all event 'reboot' for domain i-06945b37 event 'reboot' for domain i-06945b37 Is there anything fishy going on when looking at the console (VN/spice) during that reboot? Don’t see anything unusual. I can send you the full console output if you want, but I don’t see anything strange. I meant the graphical terminal, but that won't help much. I tried gathering as much info. So we just need to figure out whether QEMU sends us the event multiple times. Could you either set up debug logs [1] and then look for that in them or use the qemu-monitor.stp systemtap script to show you what's happening on the monitor? If it comes from qemu two times, than there's not much we can do about it. Nevermind, as far as I can tell, this is a bug in QEMU and is closed as a WONTFIX: https://bugzilla.redhat.com/show_bug.cgi?id=951525 Thanks for the debugging help. Martins