libvirt users - Jan 2015 - volume can't be attached after libvirt upgrade

Hi,

My nova-compute nodes was running libvirt 0.9.13 and QEMU 1.0, after
libvirt was upgraded to 1.2.2, the VMs launched by libvirt 0.9.13 can't
attach volume any more, i.e. "nova volume-attach <vm-uuid>
<volume-uuid>"
doesn't work. But before the libvirt upgrade, the same VM can attach volume
disk.

I did some debuging and find that, it failed in QEMU when try to open the
volume device, and the return errno is EPERM. QEMU instances is running as
user libvirt-qemu always, so is that possible some problem in Linux
capabilites or libvirt apparmor?



-------------------------------------
int qemu_open(const char *name, int flags, ...)
{
    int ret;
    int mode = 0;
    long euid, egid;

    if (flags & O_CREAT) {
        va_list ap;

        va_start(ap, flags);
        mode = va_arg(ap, int);
        va_end(ap);
    }

    euid = (long)geteuid();
    egid = (long)getegid();
#ifdef O_CLOEXEC
    ret = open(name, flags | O_CLOEXEC, mode);
#else
    ret = open(name, flags, mode);
    if (ret >= 0) {
        qemu_set_cloexec(ret);
    }
#endif

    return ret;
}


Regards,
CY.

I found the root cause myself.
The reason is that libvirt cgroup layout is changed after 1.0.5. So the new
volume device is not added to running QEMU instances' devices.allow.

Then new question is that is there any libvirt parameter to make the device
cgroup backwards compatible?


https://libvirt.org/cgroups.html.



On Wed, Jan 14, 2015 at 10:30 PM, Chengyuan Li <chengyuanli@gmail.com>
wrote:
> Hi,
>
> My nova-compute nodes was running libvirt 0.9.13 and QEMU 1.0, after
> libvirt was upgraded to 1.2.2, the VMs launched by libvirt 0.9.13 can't
> attach volume any more, i.e. "nova volume-attach <vm-uuid>
<volume-uuid>"
> doesn't work. But before the libvirt upgrade, the same VM can attach
volume
> disk.
>
> I did some debuging and find that, it failed in QEMU when try to open the
> volume device, and the return errno is EPERM. QEMU instances is running as
> user libvirt-qemu always, so is that possible some problem in Linux
> capabilites or libvirt apparmor?
>
>
>
> -------------------------------------
> int qemu_open(const char *name, int flags, ...)
> {
>     int ret;
>     int mode = 0;
>     long euid, egid;
>
>     if (flags & O_CREAT) {
>         va_list ap;
>
>         va_start(ap, flags);
>         mode = va_arg(ap, int);
>         va_end(ap);
>     }
>
>     euid = (long)geteuid();
>     egid = (long)getegid();
> #ifdef O_CLOEXEC
>     ret = open(name, flags | O_CLOEXEC, mode);
> #else
>     ret = open(name, flags, mode);
>     if (ret >= 0) {
>         qemu_set_cloexec(ret);
>     }
> #endif
>
>     return ret;
> }
>
>
> Regards,
> CY.
>