libvirt 1.3.3 seems to have introduced some kind of issue on CentOS 6
(no systemd). After upgrading, libvirt would take an incredibly long
time to get to a state where it would actually respond to a simple
'virsh list' command.
Looking at verbose output showed this repeating at exactly 30s intervals
2016-04-12 19:40:57.142+0000: 14929: info :
virEventPollDispatchHandles:507 : EVENT_POLL_DISPATCH_HANDLE: watch=1
events=1
2016-04-12 19:40:57.142+0000: 14929: info : virEventPollRunOnce:641 :
EVENT_POLL_RUN: nhandles=116 timeout=-1
2016-04-12 19:40:57.143+0000: 15223: info : virObjectRef:296 :
OBJECT_REF: obj=0x7fb944181bc0
2016-04-12 19:40:57.144+0000: 15223: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb944181bc0
2016-04-12 19:40:57.157+0000: 15223: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb944253020
2016-04-12 19:40:57.157+0000: 15223: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb944309450
2016-04-12 19:40:57.157+0000: 15223: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb94400ca90
2016-04-12 19:41:27.147+0000: 15224: info : virDBusCall:1577 :
DBUS_METHOD_REPLY: 'org.freedesktop.DBus.ListActivatableNames' on
'/org/freedesktop/DBus' at 'org.freedesktop.DBus'
2016-04-12 19:41:27.148+0000: 15224: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb94400ca90
2016-04-12 19:41:27.148+0000: 15224: info : virObjectRef:296 :
OBJECT_REF: obj=0x7fb94400ca90
2016-04-12 19:41:27.148+0000: 15224: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb94400ca90
2016-04-12 19:41:27.148+0000: 15224: info : virObjectRef:296 :
OBJECT_REF: obj=0x7fb94400ca90
2016-04-12 19:41:27.151+0000: 15224: info : virObjectUnref:259 :
OBJECT_UNREF: obj=0x7fb94400ca90
2016-04-12 19:41:27.151+0000: 15224: info : virObjectRef:296 :
OBJECT_REF: obj=0x7fb860000e30
2016-04-12 19:41:27.151+0000: 15224: info : virEventPollUpdateHandle:152
: EVENT_POLL_UPDATE_HANDLE: watch=58 events=15
2016-04-12 19:41:27.151+0000: 15224: info : qemuMonitorSend:1005 :
QEMU_MONITOR_SEND_MSG: mon=0x7fb860000e30
msg={"execute":"query-status","id":"libvirt-7"}
fd=-1
Looking at the dbus src, there's a 30s timeout defined here:
src/util/virdbus.c:# define VIR_DBUS_METHOD_CALL_TIMEOUT_MILLIS 30 * 1000
So, for some reason libvirt is trying to call a dbus method before every
KVM instance start. This leads to absurdly long startup times. This
behavior was not present in 1.3.1.
My workaround was to just recompile with '--with-firewalld=no
--with-dbus=no', which seems to have worked. There's not really
anything useful for libvirt to be talking to via dbus (in our
environment), so no big loss there.
