Benjamin Weik
2016-Feb-10 13:10 UTC
[libvirt-users] Unable to create raw volume on netfs storage (Operation not permitted)
Hello, we are trying to create a new setup with a centralized storage server. Currently we are running multiple "All-in-One" hosts using raw volumes stored on the local disk of the server. I tried it first with samba but it did not work so currently I have configured it as NFS. This is my current setup: For Storage: OS: Ubuntu 16.04 beta (to match vHost version), IP: 123.123.123.2 NFS: v1.2.8 Samba: v4.3.3 For vHost: OS: Ubuntu 16.04 beta (as the Package in Ubuntu 14.04 has still a bug that prevents mounting of samba shares) IP: 123.123.123.3 Libvirt: v2.21 Qemu: 2.5 Both are simply connected via switched network for testing. So on the Storage server I created a user and a group called vstorage (uid:1000,gid:500) Then created a folder /pool owned by this user, permission 777 And exported it via nfs in /etc/exports: /pool 123.123.123.3(rw,async,no_subtree_check,all_squash,anonuid=1000,anongid=500) On the vHost I defined the pool via XML: <pool type='netfs'> <name>pool</name> <source> <host name='123.123.123.2'/> <dir path='pool'/> <format type='nfs'/> </source> <target> <path>/var/lib/virt/images</path> </target> </pool> When I try to define a volume in the pool, it will always fail if raw format (default) is used. virsh vol-create-as --pool pool --name test --capacity 512M error: Failed to create vol test error: Failed to create file '/var/lib/virt/images/test': Operation not permitted In /var/log/syslog I can find the following libvirtd[889]: cannot chown '/var/lib/virt/images/test' to (0, 0): Operation not permitted libvirtd[889]: Failed to create file '/var/lib/virt/images/test': Operation not permitted When I try it with a different format, like qcow2 it is working fine: virsh vol-create-as --pool pool --name test --capacity 512M --format qcow2 Vol test created I am unable to find the reason for this problem, it also took me quite a while find out that this is a format issue as the error message is actually not really helpful. I found a small guide on setting up KVM from IBM and they are using NFS with raw images, so I'm not sure where the error is ibm.com/developerworks/library/l-using-kvm Maybe someone could shed some light on this? The same error message happens btw. with Samba. Best regards, Benjamin Weik
Michal Privoznik
2016-Feb-10 15:21 UTC
Re: [libvirt-users] Unable to create raw volume on netfs storage (Operation not permitted)
On 10.02.2016 14:10, Benjamin Weik wrote:> Hello, > > we are trying to create a new setup with a centralized storage server. Currently we are running multiple "All-in-One" hosts using raw volumes stored on the local disk of the server. > > I tried it first with samba but it did not work so currently I have configured it as NFS. > > This is my current setup: > > For Storage: > OS: Ubuntu 16.04 beta (to match vHost version), > IP: 123.123.123.2 > NFS: v1.2.8 > Samba: v4.3.3 > > For vHost: > OS: Ubuntu 16.04 beta (as the Package in Ubuntu 14.04 has still a bug that prevents mounting of samba shares) > IP: 123.123.123.3 > Libvirt: v2.21 > Qemu: 2.5 > > Both are simply connected via switched network for testing. > > So on the Storage server I created a user and a group called vstorage (uid:1000,gid:500) > Then created a folder /pool owned by this user, permission 777 > And exported it via nfs in /etc/exports: > > /pool 123.123.123.3(rw,async,no_subtree_check,all_squash,anonuid=1000,anongid=500) > > > On the vHost I defined the pool via XML: > > <pool type='netfs'> > <name>pool</name> > <source> > <host name='123.123.123.2'/> > <dir path='pool'/> > <format type='nfs'/> > </source> > <target> > <path>/var/lib/virt/images</path> > </target> > </pool> > > > When I try to define a volume in the pool, it will always fail if raw format (default) is used. > > virsh vol-create-as --pool pool --name test --capacity 512M > error: Failed to create vol test > error: Failed to create file '/var/lib/virt/images/test': Operation not permitted > > In /var/log/syslog I can find the following > libvirtd[889]: cannot chown '/var/lib/virt/images/test' to (0, 0): Operation not permitted > libvirtd[889]: Failed to create file '/var/lib/virt/images/test': Operation not permitted > > When I try it with a different format, like qcow2 it is working fine: > > virsh vol-create-as --pool pool --name test --capacity 512M --format qcow2 > Vol test createdUnfortunately, this will create the volume with the default permissions as there are none specified in vol-create XML. This vol-create-as command creates a very minimalistic volume XML and passes it to virStorageVolCreateXML() API which if no perms specified in the XML will use current euid/egid of libvirtd (0:0). Hence the error message. What you can do here is create your own volume XML with perms specified and use vol-create instead. What we can do is: 1) introduce --print-xml option to vol-crate-as like other commands constructing an XML have. 2) Make libvirt to inherit pool perms for the volume if none specified. 3) Teach vol-create-as to accept permissions. Michal