libvirt users - Sep 2014 - Using custom QEMU binaries with libvirt

Hi all,

I compiled a custom version of QEMU 2.0.0 and I am having hard times to 
make it available to libvirt. Just to clarify, if I execute

/usr/local/bin/qemu-system-x86_64

it does performs good. But when I put this very same path to <emulator> 
tag in a domain configutation, when i start the domain I get

error: Failed to start domain vm1
error: internal error: process exited while connecting to monitor: 
libvirt:  error : cannot execute binary 
/usr/local/bin/qemu-system-x86_64: Permission denied

I tried setting +x permission to all the binaries in /usr/local/bin, 
disabling apparmor profile for libvirtd, creating and putting to 
complain a profile for
/usr/local/bin/qemu-system-x86_64, creating a softlink to 
/usr/bin/kvm-spice to the custom binary and leaving <emulator> as
default...

At the end of http://www.gossamer-threads.com/lists/openstack/dev/40033 
I found something about AppArmor, and  enabling bios.bin reading 
somewhere, but I got a little confused here.

The most disapointing thing here, is that using qemu 1.7 I could use my 
custom build, but apparently something changed with 2.0 (or with libvirt 
integration).

Any ideas?

Thanks!

On Thu, Sep 18, 2014 at 10:57:23AM +0200, Joaquim Barrera
wrote:
> Hi all,
> 
> I compiled a custom version of QEMU 2.0.0 and I am having hard times to
make
> it available to libvirt. Just to clarify, if I execute
> 
> /usr/local/bin/qemu-system-x86_64
> 
> it does performs good. But when I put this very same path to
<emulator> tag
> in a domain configutation, when i start the domain I get
> 
> error: Failed to start domain vm1
> error: internal error: process exited while connecting to monitor: libvirt:
> error : cannot execute binary /usr/local/bin/qemu-system-x86_64: Permission
> denied
> 
> I tried setting +x permission to all the binaries in /usr/local/bin,
> disabling apparmor profile for libvirtd, creating and putting to complain a
> profile for
> /usr/local/bin/qemu-system-x86_64, creating a softlink to
/usr/bin/kvm-spice
> to the custom binary and leaving <emulator> as default...
> 
> At the end of http://www.gossamer-threads.com/lists/openstack/dev/40033 I
> found something about AppArmor, and  enabling bios.bin reading somewhere,
> but I got a little confused here.
> 
> The most disapointing thing here, is that using qemu 1.7 I could use my
> custom build, but apparently something changed with 2.0 (or with libvirt
> integration).
>From libvirt's POV the only things that should matter are
 - Permission for 'qemu:qemu' user/group to execute the binary (and
access
   parent directories)
 - AppArmour profile support, or SELinux label (as appropriate for disto)



Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

On 18/09/14 11:06, Daniel P. Berrange wrote:
> On Thu, Sep 18, 2014 at 10:57:23AM +0200, Joaquim Barrera wrote:
>> Hi all,
>>
>> I compiled a custom version of QEMU 2.0.0 and I am having hard times to
make
>> it available to libvirt. Just to clarify, if I execute
>>
>> /usr/local/bin/qemu-system-x86_64
>>
>> it does performs good. But when I put this very same path to
<emulator> tag
>> in a domain configutation, when i start the domain I get
>>
>> error: Failed to start domain vm1
>> error: internal error: process exited while connecting to monitor:
libvirt:
>> error : cannot execute binary /usr/local/bin/qemu-system-x86_64:
Permission
>> denied
>>
>> I tried setting +x permission to all the binaries in /usr/local/bin,
>> disabling apparmor profile for libvirtd, creating and putting to
complain a
>> profile for
>> /usr/local/bin/qemu-system-x86_64, creating a softlink to
/usr/bin/kvm-spice
>> to the custom binary and leaving <emulator> as default...
>>
>> At the end of http://www.gossamer-threads.com/lists/openstack/dev/40033
I
>> found something about AppArmor, and  enabling bios.bin reading
somewhere,
>> but I got a little confused here.
>>
>> The most disapointing thing here, is that using qemu 1.7 I could use my
>> custom build, but apparently something changed with 2.0 (or with
libvirt
>> integration).
>  From libvirt's POV the only things that should matter are
>
>   - Permission for 'qemu:qemu' user/group to execute the binary
(and access
>     parent directories)
>   - AppArmour profile support, or SELinux label (as appropriate for disto)
>
>
>
> Regards,
> Daniel
Thanks Daniel. When you say qemu:qemu I understant libvirt-qemu:kvm 
user/group? Because in my system there is no qemu user nor qemu group.

Nevertheless, trying to move on, I recompiled QEMU with --prefix=/usr, 
to just override the original binaries (despite my will), and it worked 
partially. virsh start vm1 boots the VM but at the time to migrate it to 
another host:

virsh migrate --persistent --verbose --copy-storage-inc vm1 
qemu+ssh://gco-4:17179/system
error: Unable to read from monitor: Connection reset by peer

At this point, if the VM starts and I can open VNC connection) I guess 
that the answer will be that it's something QEMU related, and that 
libvirt doesn't have anything to do, but I just want to be sure about that.

Thanks a lot.