libvirt users - May 2014 - Libvirtd failed to start inside container: libvirt-qemu.so.0: cannot open shared object file: Permission denied

Hi all.

I run into trouble, while try to start libvirtd inside docker container
(actually it is LXC container).

During startup libvirtd can't load shared library libvirt-qemu.so.0 (and
strace results looks very odd).

* I run libvirtd as root.
* libvirt-bin - 0.9.8-2ubuntu17
* selinux/apparmor both disabled.
* No other security extension are used.
* No sticky bits are set.
* Required library are present in appropriate folder and have all required
permissions.
* I also successfully load it to other process (python).
* No file locks are holds.
* OS ubuntu linux 12.04 x64 ___running inside LXC container__(docker).
Container is privileged (I can run vm using kvm in it)
* On host system libvirtd starts ok, but stopped now

# uname -a
Linux 27119997ee44 3.11.0-19-generic #33-Ubuntu SMP Tue Mar 11 18:48:34 UTC
2014 x86_64 x86_64 x86_64 GNU/Linux

# dpkg -l | grep libvirt
ii  libvirt-bin
0.9.8-2ubuntu17                          programs for the libvirt library
ii  libvirt0
0.9.8-2ubuntu17                          library for interfacing with
different virtualization systems
ii  python-libvirt
0.9.8-2ubuntu17                          libvirt Python bindings


# libvirtd
libvirtd: error while loading shared libraries: libvirt-qemu.so.0: cannot
open shared object file: Permission denied

# whoami
root

# ls -l `which libvirtd`
-rwxr-xr-x 1 root root 1211712 Apr 16  2012 /usr/sbin/libvirtd

# ldd `which libvirtd`
        ......
        libvirt-qemu.so.0 => /usr/lib/libvirt-qemu.so.0
(0x00007fd6ed29c000)Environment:
        ....

# ls -l /usr/lib/libvirt-qemu.so.0
-rwxr-xr-x 1 root root 6144 May  6 21:46 /usr/lib/libvirt-qemu.so.0

# strace libvirtd
execve("/usr/sbin/libvirtd", ["libvirtd"], [/* 19 vars */])
= 0
brk(0)                                  = 0x1d74000
.... (~30 lines)

open("/usr/lib/libvirt-qemu.so.0", O_RDONLY|O_CLOEXEC) = -1 EACCES
(Permission denied) <<<<  !!!!
stat("/usr/lib", 0x7fffbd127840)        = -1 EACCES (Permission
denied)--
<<<<  !!!!


Before try to load /usr/lib/libvirt-qemu.so.0 libvirtd make only stat,
open, access and brk system calls (no change user or other security related
calls)

# stat /usr/lib
  File: `/usr/lib'
  Size: 8192            Blocks: 24         IO Block: 4096   directory
Device: 53h/83d Inode: 70          Links: 68
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2014-04-02 12:38:18.171617082 +0000
Modify: 2014-05-06 21:46:39.450449491 +0000
Change: 2014-05-06 21:46:39.450449491 +0000
 Birth: -

# selinuxenabled ; echo $?
1

# kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

on host system -

$ docker -v
Docker version 0.9.1, build 3600720

Thanks

----
Kostiantyn Danilov aka koder.ua
Principal software engineer, Mirantis

skype:koder.ua
http://koder-ua.blogspot.com/
http://mirantis.com