libvirt users - Apr 2014 - libvirt and glusterfs integrate problem

Hi everyone

I have a problem when use libvirt and glusterfs. 
When I use libvirt to start a vm with glusterfs disk. the operation will block.
after I `CTL + c` the 'virsh start domainxx' comand ,
I use `virsh list` will find the vm in a strange state, kind like follow:
 Id    Name                           State
----------------------------------------------------
 20    vm10                           shut off 
and I when I start the vm again, It will promote the vm is in active.

the xml as follow (172.16.32.16 is a glusterfs node):
<domain type='kvm' id='20'>
  <name>vm10</name>
  <uuid>12208c14-3990-c134-2653-8a828ff1eb1d</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static'>4</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64'
machine='rhel6.5.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='localtime'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='network' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source protocol='gluster'
name='h2216v5/tmp/vm10.img'>
        <host name='172.16.32.16'/>
      </source>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0'>
      <alias name='usb0'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x2'/>
    </controller>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x01' function='0x1'/>
    </controller>
    <controller type='pci' index='0'
model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <interface type='bridge'>
      <mac address='00:0c:29:2f:aa:b1'/>
      <source bridge='virbr0'/>
      <target dev='vnet91'/>
      <model type='rtl8139'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x03' function='0x0'/>
    </interface>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5991' autoport='yes'
listen='172.16.32.3' keymap='en-us'>
      <listen type='address' address='172.16.32.3'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x04' function='0x0'/>
    </memballoon>
  </devices>
  <seclabel type='none'/>
</domain>


Also:
I can use `qemu-kvm` command to start vm successful with gluster://172.16.32.16
I have stop both nodes' iptables. and my libvirt and qemu version is :
Compiled against library: libvirt 1.1.4
Using library: libvirt 1.1.4
Using API: QEMU 1.1.4
Running hypervisor: QEMU 1.6.2
glusterfs-3.4.1-3.el6.x86_64			
--------------
lyz_pro
2014-04-14

On 04/13/2014 08:45 PM, lyz_pro wrote:
> Hi everyone
> 
> I have a problem when use libvirt and glusterfs. 
> When I use libvirt to start a vm with glusterfs disk. the operation will
block.
I've seen similar reports; the problem is in glusterfs and not in
libvirt.  Something in qemu's use of gluster is hanging, which in turn
prevents qemu from responding to libvirt, so libvirt is blocked waiting
for gluster to complete its action.  You'll need to debug why gluster is
not returning control to the user (either with the action completed, or
with a timely error message).
> after I `CTL + c` the 'virsh start domainxx' comand ,
> I use `virsh list` will find the vm in a strange state, kind like follow:
>  Id    Name                           State
> ----------------------------------------------------
>  20    vm10                           shut off 
> and I when I start the vm again, It will promote the vm is in active.
There may be follow0on bugs in libvirt not handling state changes
gracefully when you ctrl-c a virsh command in the middle of a start command.
> 
> Also:
> I can use `qemu-kvm` command to start vm successful with
gluster://172.16.32.16
> I have stop both nodes' iptables. and my libvirt and qemu version is :
> Compiled against library: libvirt 1.1.4
> Using library: libvirt 1.1.4
> Using API: QEMU 1.1.4
> Running hypervisor: QEMU 1.6.2
> glusterfs-3.4.1-3.el6.x86_64			
Can you repeat with libvirt 1.2.3, and/or newer qemu (qemu 1.7 is out,
qemu 2.0 is on release candidate 2 for a release "real soon now"), to
see if this is a bug that has been fixed in the meantime?

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org