libvirt users - Feb 2023 - ACLs problem on /dev/kvm

Dear All,

I have started to work with libvirt a few weeks ago, but I have some problem
with starting a virtual machine. Currently, I am using an embedded arm64 device
with a Linux built with Yocto. I managed to install lbvirt 8.1.0 in the image
and I have a qemu user and qemu and kvm groups in the system. I am using KVM
hypervisor and I did the configuration in the qemu.conf and libvirtd.conf files,
enabled all the sockets and services in the system. The xml based definition of
the virtual machine is simple, but when I try to start it I get the error
message: Failed to start domain 'XYZ' and Unable to set ACLs on
/dev/kvm: Invalid argument. I cannot set ACLs on the /dev/kvm (owner is root,
group is kvm but I have also tried to set it root:root) with the setfacl
command, but I gave rwx access to user, group and others as well so everybody
can use the device. I also uncommented the relevant lines in the qemu.conf file
(cgroup_controllers = ... and cgroup_device_acl = ...) and I also found that
devices controller is already mounted at /sys/fs/cgroup/devices. Can anybody
help me with this issue? Did Anybody have similar problem? I can start a virtual
machine with qemu-system-aarch64, but I would like to use the libvirt library to
manage the machines.

Thank you in advance for an early reply.

Best regards,

M?rton S?nta


This transmission is intended solely for the addressee and contains confidential
information.
If you are not the intended recipient, please immediately inform the sender and
delete the message and any attachments from your system.
Furthermore, please do not copy the message or disclose the contents to anyone
unless agreed otherwise. To the extent permitted by law we shall in no way be
liable for any damages, whatever their nature, arising out of transmission
failures, viruses, external influence, delays and the like.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20230221/79228f81/attachment.htm>

On 2/21/23 09:47, S?nta, M?rton (ext) wrote:
> Dear All,
> 
> ?
> 
> I have started to work with libvirt a few weeks ago, but I have some
> problem with starting a virtual machine. Currently, I am using an
> *embedded arm64 device* with a *Linux* built with *Yocto*. I managed to
> install *lbvirt 8.1.0* in the image and I have a *qemu user* and *qemu
> and kvm groups *in the system. I am using *KVM hypervisor* and I did the
> configuration in the *qemu.conf* and *libvirtd.conf* files, enabled all
> the sockets and services in the system. The xml based definition of the
> virtual machine is simple, but when I try to start it I get the error
> message: *Failed to start domain ?XYZ?* and *Unable to set ACLs on
> /dev/kvm: Invalid argument*. I cannot set ACLs on the /dev/kvm (owner is
> /root/, group is /kvm/ but I have also tried to set it /root:root/) with
> the /setfacl /command, but I gave /rwx/ access to user, group and others
> as well so everybody can use the device. I also uncommented the relevant
> lines in the /qemu.conf/ file (/cgroup_controllers = ?/ and
> /cgroup_device_acl = ?/) and I also found that /devices/ controller is
> already mounted at //sys/fs/cgroup/devices/. Can anybody help me with
> this issue? Did Anybody have similar problem? I can start a virtual
> machine with *qemu-system-aarch64*, but I would like to use the libvirt
> library to manage the machines.
> 
> ?
> 
> Thank you in advance for an early reply.
> 
This is a namespace issue. Basically, when starting a guest (or domain
as we call it), libvirt creates a private /dev for it. It's using mount
namespace to create a private mount table to replace the original /dev,
hence the name of the feature. And this private /dev is populated with
only a handful of nodes (some basic ones, like /dev/zero, /dev/null, ...
and those which are configured in domain XML). Each individual node is
created as an exact copy of the original /dev, including ACL entries. If
you understand C a bit you can see the function that's responsible for
creating the nodes here [1].

Now, there used to be a bug, where libvirt tried to set ACLs even though
the corresponding file had none. It was fixed by the following commit
[2]. unfortunately, the commit is part of newer libvirt than what you
have: v8.8.0.

There is a workaround though: you can disable this namespace feature by
setting the following in /etc/libvirt/qemu.conf:

  namespaces = []


Michal

1:
https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_namespace.c#L972

2:
https://gitlab.com/libvirt/libvirt/-/commit/687374959e160dc566bd4b6d43c7bf1beb470c59