thr3ads.net - libvirt users - Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path [Apr 2016]

If this information is useful, please help other people find it:
Share via:

John Ferlan

2016-Apr-13 14:00 UTC

Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

On 04/13/2016 09:23 AM, TomK wrote:> On 4/13/2016 1:33 AM, Martin Kletzander wrote:
>> On Tue, Apr 12, 2016 at 06:24:16PM -0400, TomK wrote:
>>> On 4/12/2016 5:08 PM, John Ferlan wrote:
>>>> Having/using a root squash via an NFS pool is "easy"
(famous last
>>>> words)
>>>>
>>>> Create some pool XML (taking the example I have)
>>>>
>>>> % cat nfs.xml
>>>> <pool type='netfs'>
>>>>      <name>rootsquash</name>
>>>>      <source>
>>>>          <host name='localhost'/>
>>>>          <dir path='/home/bzs/rootsquash/nfs'/>
>>>>          <format type='nfs'/>
>>>>      </source>
>>>>      <target>
>>>>          <path>/tmp/netfs-rootsquash-pool</path>
>>>>          <permissions>
>>>>              <mode>0755</mode>
>>>>              <owner>107</owner>
>>>>              <group>107</group>
>>>>          </permissions>
>>>>      </target>
>>>> </pool>
>>>>
>>>> In this case 107:107 is qemu:qemu and I used
'localhost' as the
>>>> hostname, but that can be a fqdn or ip-addr to the NFS server.
>>>>
>>>> You've already seen my /etc/exports
>>>>
>>>> virsh pool-define nfs.xml
>>>> virsh pool-build rootsquash
>>>> virsh pool-start rootsquash
>>>> virsh vol-list rootsquash
>>>>
>>>> Now instead of
>>>>
>>>>     <disk type='file' device='disk'>
>>>>       <source
file='/var/lib/one//datastores/0/38/disk.0'/>
>>>>       <target dev='hda'/>
>>>>       <driver name='qemu' type='qcow2'
cache='none'/>
>>>>     </disk>
>>>>
>>>> Something like:
>>>>
>>>>    <disk type='volume' device='disk'>
>>>>      <driver name='qemu' type='qemu'
cache='none'/>
>>>>      <source pool='rootsquash'
volume='disk.0'/>
>>>>      <target dev='hda'/>
>>>>    </disk>
>>>>
>>>> The volume name may be off, but it's perhaps close.  I
forget how to do
>>>> the readonly bit for a pool (again, my focus is elsewhere).
>>>>
>>>> Of course you'd have to adjust the nfs.xml above to suit
your
>>>> environment and see what you see/get.  The privileges for the
pool and
>>>> volumes in the pool become the key to how libvirt decides to
"request
>>>> access" to the volume.  "disk.1" having read
access is probably not an
>>>> issue since you seem to be using it as a CDROM; however,
"disk.0" is
>>>> going to be used for read/write - thus would have to be
appropriately
>>>> configured...
>>>>
>>>
>>> Thanks John!  Appreciated again.
>>>
>>> No worries, handle what's on the plate now and earmark this for
checking
>>> once you have some free cycles.  I can temporarily hop on one leg
by
>>> using Martin Kletzander's workaround (It's a POC at the
moment).
>>>
>>> I'll have a look at your instructions further but wanted to
find out if
>>> that config nfs.xml is a one time thing correct?  I'm spinning
these up
>>> at will via the OpenNebula GUI and if I have update for each VM,
that
>>> breaks the Cloud provisioning.  I'll go over your notes again.
I'm
>>> optimistic.   :)
>>>
>>
>> The more I'm thinking about it, the more I am convinced that the
>> workaround is actually not a workaround.  The only thing you need to do
>> is having execute for others (precisely for 'nobody' on the nfs
share)
>> in the whole path on all directories.  Without that even the pool
won't
>> be usable from libvirt.  However it does not pose any security issue as
>> it only allows others to check the path.  When qemu is launched, it has
>> the proper "label", meaning uid:gid to access the file so it
will be
>> able to read/write or whatever permissions you set there.  It's
just
>> that libvirt does some checks that the path exists for example.
>>
>> Hope that's understandable and it will resolve your issue
permanently.
>>
>> Have a nice day,
>> Martin
>>
>>
>> _______________________________________________
>> libvirt-users mailing list
>> libvirt-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/libvirt-users
> 
> The only reason I said that this might be a 'workaround' is due to
John
> Farlan commenting that he'll look at this later on.  Ideally the
> opennebula community keeps the other permissions to nill and presumably
> they work on NFSv3 per the forum topic I included earlier from them. 
> But if setting the permissions on nobody to allow for the functionality,
> I would be comfortable with that.
> 
Martin and I were taking different paths...  But yes, it certainly makes
sense given your error message about canonical path and the need for
eXecute permissions... I think I started wondering about that first, but
then jumped into the NFS pool because that's what my reference point is
for root-squash. Since root squash essentially sends root requests as
"nfsnobody" (IOW: others not the user or group), then the
"o+x" approach
is the solution if you're going directly at the file.

John

TomK

2016-Apr-14 05:01 UTC

head link

Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

On 4/13/2016 10:00 AM, John Ferlan wrote:>
> On 04/13/2016 09:23 AM, TomK wrote:
>> On 4/13/2016 1:33 AM, Martin Kletzander wrote:
>>> On Tue, Apr 12, 2016 at 06:24:16PM -0400, TomK wrote:
>>>> On 4/12/2016 5:08 PM, John Ferlan wrote:
>>>>> Having/using a root squash via an NFS pool is
"easy" (famous last
>>>>> words)
>>>>>
>>>>> Create some pool XML (taking the example I have)
>>>>>
>>>>> % cat nfs.xml
>>>>> <pool type='netfs'>
>>>>>       <name>rootsquash</name>
>>>>>       <source>
>>>>>           <host name='localhost'/>
>>>>>           <dir
path='/home/bzs/rootsquash/nfs'/>
>>>>>           <format type='nfs'/>
>>>>>       </source>
>>>>>       <target>
>>>>>          
<path>/tmp/netfs-rootsquash-pool</path>
>>>>>           <permissions>
>>>>>               <mode>0755</mode>
>>>>>               <owner>107</owner>
>>>>>               <group>107</group>
>>>>>           </permissions>
>>>>>       </target>
>>>>> </pool>
>>>>>
>>>>> In this case 107:107 is qemu:qemu and I used
'localhost' as the
>>>>> hostname, but that can be a fqdn or ip-addr to the NFS
server.
>>>>>
>>>>> You've already seen my /etc/exports
>>>>>
>>>>> virsh pool-define nfs.xml
>>>>> virsh pool-build rootsquash
>>>>> virsh pool-start rootsquash
>>>>> virsh vol-list rootsquash
>>>>>
>>>>> Now instead of
>>>>>
>>>>>      <disk type='file' device='disk'>
>>>>>        <source
file='/var/lib/one//datastores/0/38/disk.0'/>
>>>>>        <target dev='hda'/>
>>>>>        <driver name='qemu' type='qcow2'
cache='none'/>
>>>>>      </disk>
>>>>>
>>>>> Something like:
>>>>>
>>>>>     <disk type='volume'
device='disk'>
>>>>>       <driver name='qemu' type='qemu'
cache='none'/>
>>>>>       <source pool='rootsquash'
volume='disk.0'/>
>>>>>       <target dev='hda'/>
>>>>>     </disk>
>>>>>
>>>>> The volume name may be off, but it's perhaps close.  I
forget how to do
>>>>> the readonly bit for a pool (again, my focus is elsewhere).
>>>>>
>>>>> Of course you'd have to adjust the nfs.xml above to
suit your
>>>>> environment and see what you see/get.  The privileges for
the pool and
>>>>> volumes in the pool become the key to how libvirt decides
to "request
>>>>> access" to the volume.  "disk.1" having read
access is probably not an
>>>>> issue since you seem to be using it as a CDROM; however,
"disk.0" is
>>>>> going to be used for read/write - thus would have to be
appropriately
>>>>> configured...
>>>>>
>>>> Thanks John!  Appreciated again.
>>>>
>>>> No worries, handle what's on the plate now and earmark this
for checking
>>>> once you have some free cycles.  I can temporarily hop on one
leg by
>>>> using Martin Kletzander's workaround (It's a POC at the
moment).
>>>>
>>>> I'll have a look at your instructions further but wanted to
find out if
>>>> that config nfs.xml is a one time thing correct?  I'm
spinning these up
>>>> at will via the OpenNebula GUI and if I have update for each
VM, that
>>>> breaks the Cloud provisioning.  I'll go over your notes
again. I'm
>>>> optimistic.   :)
>>>>
>>> The more I'm thinking about it, the more I am convinced that
the
>>> workaround is actually not a workaround.  The only thing you need
to do
>>> is having execute for others (precisely for 'nobody' on the
nfs share)
>>> in the whole path on all directories.  Without that even the pool
won't
>>> be usable from libvirt.  However it does not pose any security
issue as
>>> it only allows others to check the path.  When qemu is launched, it
has
>>> the proper "label", meaning uid:gid to access the file so
it will be
>>> able to read/write or whatever permissions you set there.  It's
just
>>> that libvirt does some checks that the path exists for example.
>>>
>>> Hope that's understandable and it will resolve your issue
permanently.
>>>
>>> Have a nice day,
>>> Martin
>>>
>>>
>>> _______________________________________________
>>> libvirt-users mailing list
>>> libvirt-users@redhat.com
>>> https://www.redhat.com/mailman/listinfo/libvirt-users
>> The only reason I said that this might be a 'workaround' is due
to John
>> Farlan commenting that he'll look at this later on.  Ideally the
>> opennebula community keeps the other permissions to nill and presumably
>> they work on NFSv3 per the forum topic I included earlier from them.
>> But if setting the permissions on nobody to allow for the
functionality,
>> I would be comfortable with that.
>>
> Martin and I were taking different paths...  But yes, it certainly makes
> sense given your error message about canonical path and the need for
> eXecute permissions... I think I started wondering about that first, but
> then jumped into the NFS pool because that's what my reference point is
> for root-squash. Since root squash essentially sends root requests as
> "nfsnobody" (IOW: others not the user or group), then the
"o+x" approach
> is the solution if you're going directly at the file.
>
> John
Yes, appears the o+x is the only way right now.  It definitely tries to 
access the share as root though, on CentOS 7 since I also tried to add 
nfsnobody and nobody to the oneadmin group and that did not work 
either.  Seems OpenNebula doesn't have this issue with NFSv3 running on 
Ubuntu:

[root@mdskvm-p01 ~]# rmdir /tmp/netfs-rootsquash-pool
[root@mdskvm-p01 ~]# cat nfs.xml
<pool type='netfs'>
      <name>rootsquash</name>
      <source>
          <host name='opennebula01'/>
          <dir path='/var/lib/one'/>
          <format type='nfs'/>
      </source>
      <target>
          <path>/tmp/netfs-rootsquash-pool</path>
          <permissions>
              <mode>0755</mode>
              <owner>9869</owner>
              <group>9869</group>
          </permissions>
      </target>
</pool>
[root@mdskvm-p01 ~]#
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# virsh pool-define nfs.xml
Pool rootsquash defined from nfs.xml

[root@mdskvm-p01 ~]# virsh pool-build rootsquash
Pool rootsquash built

[root@mdskvm-p01 ~]# virsh pool-start rootsquash
error: Failed to start pool rootsquash
error: cannot open path '/tmp/netfs-rootsquash-pool': Permission denied

[root@mdskvm-p01 ~]# virsh vol-list rootsquash
error: Failed to list volumes
error: Requested operation is not valid: storage pool 'rootsquash' is 
not active

[root@mdskvm-p01 ~]# ls -altri /tmp/netfs-rootsquash-pool
total 4
      133 drwxrwxrwt. 14 root     root     4096 Apr 14 00:05 ..
68785924 drwxr-xr-x   2 oneadmin oneadmin    6 Apr 14 00:05 .
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# id oneadmin
uid=9869(oneadmin) gid=9869(oneadmin) 
groups=9869(oneadmin),992(libvirt),36(kvm)
[root@mdskvm-p01 ~]# id nobody
uid=99(nobody) gid=99(nobody) groups=99(nobody),9869(oneadmin)
[root@mdskvm-p01 ~]# id nfsnobody
uid=65534(nfsnobody) gid=65534(nfsnobody) 
groups=65534(nfsnobody),9869(oneadmin)
[root@mdskvm-p01 ~]# id root
uid=0(root) gid=0(root) groups=0(root)
[root@mdskvm-p01 ~]#

[root@mdskvm-p01 ~]# ps -ef|grep -i libvirtd
root       352 31058  0 00:31 pts/1    00:00:00 grep --color=auto -i 
libvirtd
root      1459     1  0 Apr11 ?        00:07:40 /usr/sbin/libvirtd 
--listen --config /etc/libvirt/libvirtd.conf
[root@mdskvm-p01 ~]#



[root@mdskvm-p01 ~]# umount /var/lib/one
[root@mdskvm-p01 ~]# mount --no-canonicalize /var/lib/one
[root@mdskvm-p01 ~]# umount /var/lib/one
[root@mdskvm-p01 ~]# mount /var/lib/one
[root@mdskvm-p01 ~]# mount|tail -n 1
192.168.0.70:/var/lib/one on /var/lib/one type nfs4 
(rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.60,local_lock=none,addr=192.168.0.70)
[root@mdskvm-p01 ~]# umount /var/lib/one
[root@mdskvm-p01 ~]# mount --no-canonicalize /var/lib/one
[root@mdskvm-p01 ~]# mount|tail -n 1
192.168.0.70:/var/lib/one on /var/lib/one type nfs4 
(rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.60,local_lock=none,addr=192.168.0.70)
[root@mdskvm-p01 ~]# su - oneadmin
Last login: Thu Apr 14 00:27:59 EDT 2016 on pts/0
[oneadmin@mdskvm-p01 ~]$ virsh -d 1 --connect qemu:///system create 
/var/lib/one//datastores/0/47/deployment.0
create: file(optdata): /var/lib/one//datastores/0/47/deployment.0
error: Failed to create domain from 
/var/lib/one//datastores/0/47/deployment.0
error: can't canonicalize path
'/var/lib/one//datastores/0/47/disk.1':
Permission denied
[oneadmin@mdskvm-p01 ~]$




CONTROLLER ( NFS Server )

[oneadmin@opennebula01 one]$ ls -ld 
/var{,/lib{,/one{,/datastores{,/0{,/47{,/disk.1}}}}}}
drwxr-xr-x. 19 root     root       4096 Apr  4 21:26 /var
drwxr-xr-x. 28 root     root       4096 Apr 13 03:30 /var/lib
drwxr-x---. 12 oneadmin oneadmin   4096 Apr 14 00:40 /var/lib/one
drwxrwxr-x   6 oneadmin oneadmin     46 Mar 31 02:44 /var/lib/one/datastores
drwxrwxr-x   8 oneadmin oneadmin     60 Apr 13 23:31 
/var/lib/one/datastores/0
drwxrwxr-x   2 oneadmin oneadmin     68 Apr 13 23:32 
/var/lib/one/datastores/0/47
-rw-r--r--   1 oneadmin oneadmin 372736 Apr 13 23:32 
/var/lib/one/datastores/0/47/disk.1
[oneadmin@opennebula01 one]$



NODE ( NFS Client )

[oneadmin@mdskvm-p01 ~]$ ls -ld 
/var{,/lib{,/one{,/datastores{,/0{,/47{,/disk.1}}}}}}
drwxr-xr-x. 21 root     root       4096 Apr 11 07:10 /var
drwxr-xr-x. 45 root     root       4096 Apr 13 04:11 /var/lib
drwxr-x---  12 oneadmin oneadmin   4096 Apr 14 00:39 /var/lib/one
drwxrwxr-x   6 oneadmin oneadmin     46 Mar 31 02:44 /var/lib/one/datastores
drwxrwxr-x   8 oneadmin oneadmin     60 Apr 13 23:31 
/var/lib/one/datastores/0
drwxrwxr-x   2 oneadmin oneadmin     68 Apr 13 23:32 
/var/lib/one/datastores/0/47
-rw-r--r--   1 oneadmin oneadmin 372736 Apr 13 23:32 
/var/lib/one/datastores/0/47/disk.1
[oneadmin@mdskvm-p01 ~]$



Cheers,
Tom K.
-------------------------------------------------------------------------------------

Living on earth is expensive, but it includes a free trip around the sun.

TomK

2016-Apr-14 05:24 UTC

head link

Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

On 4/14/2016 1:01 AM, TomK wrote:> On 4/13/2016 10:00 AM, John Ferlan wrote:
>>
>> On 04/13/2016 09:23 AM, TomK wrote:
>>> On 4/13/2016 1:33 AM, Martin Kletzander wrote:
>>>> On Tue, Apr 12, 2016 at 06:24:16PM -0400, TomK wrote:
>>>>> On 4/12/2016 5:08 PM, John Ferlan wrote:
>>>>>> Having/using a root squash via an NFS pool is
"easy" (famous last
>>>>>> words)
>>>>>>
>>>>>> Create some pool XML (taking the example I have)
>>>>>>
>>>>>> % cat nfs.xml
>>>>>> <pool type='netfs'>
>>>>>>       <name>rootsquash</name>
>>>>>>       <source>
>>>>>>           <host name='localhost'/>
>>>>>>           <dir
path='/home/bzs/rootsquash/nfs'/>
>>>>>>           <format type='nfs'/>
>>>>>>       </source>
>>>>>>       <target>
>>>>>> <path>/tmp/netfs-rootsquash-pool</path>
>>>>>>           <permissions>
>>>>>>               <mode>0755</mode>
>>>>>>               <owner>107</owner>
>>>>>>               <group>107</group>
>>>>>>           </permissions>
>>>>>>       </target>
>>>>>> </pool>
>>>>>>
>>>>>> In this case 107:107 is qemu:qemu and I used
'localhost' as the
>>>>>> hostname, but that can be a fqdn or ip-addr to the NFS
server.
>>>>>>
>>>>>> You've already seen my /etc/exports
>>>>>>
>>>>>> virsh pool-define nfs.xml
>>>>>> virsh pool-build rootsquash
>>>>>> virsh pool-start rootsquash
>>>>>> virsh vol-list rootsquash
>>>>>>
>>>>>> Now instead of
>>>>>>
>>>>>>      <disk type='file'
device='disk'>
>>>>>>        <source
file='/var/lib/one//datastores/0/38/disk.0'/>
>>>>>>        <target dev='hda'/>
>>>>>>        <driver name='qemu'
type='qcow2' cache='none'/>
>>>>>>      </disk>
>>>>>>
>>>>>> Something like:
>>>>>>
>>>>>>     <disk type='volume'
device='disk'>
>>>>>>       <driver name='qemu'
type='qemu' cache='none'/>
>>>>>>       <source pool='rootsquash'
volume='disk.0'/>
>>>>>>       <target dev='hda'/>
>>>>>>     </disk>
>>>>>>
>>>>>> The volume name may be off, but it's perhaps close.
I forget how
>>>>>> to do
>>>>>> the readonly bit for a pool (again, my focus is
elsewhere).
>>>>>>
>>>>>> Of course you'd have to adjust the nfs.xml above to
suit your
>>>>>> environment and see what you see/get.  The privileges
for the
>>>>>> pool and
>>>>>> volumes in the pool become the key to how libvirt
decides to
>>>>>> "request
>>>>>> access" to the volume.  "disk.1" having
read access is probably
>>>>>> not an
>>>>>> issue since you seem to be using it as a CDROM;
however, "disk.0" is
>>>>>> going to be used for read/write - thus would have to be
>>>>>> appropriately
>>>>>> configured...
>>>>>>
>>>>> Thanks John!  Appreciated again.
>>>>>
>>>>> No worries, handle what's on the plate now and earmark
this for
>>>>> checking
>>>>> once you have some free cycles.  I can temporarily hop on
one leg by
>>>>> using Martin Kletzander's workaround (It's a POC at
the moment).
>>>>>
>>>>> I'll have a look at your instructions further but
wanted to find
>>>>> out if
>>>>> that config nfs.xml is a one time thing correct?  I'm
spinning
>>>>> these up
>>>>> at will via the OpenNebula GUI and if I have update for
each VM, that
>>>>> breaks the Cloud provisioning.  I'll go over your notes
again. I'm
>>>>> optimistic.   :)
>>>>>
>>>> The more I'm thinking about it, the more I am convinced
that the
>>>> workaround is actually not a workaround.  The only thing you
need
>>>> to do
>>>> is having execute for others (precisely for 'nobody' on
the nfs share)
>>>> in the whole path on all directories.  Without that even the
pool
>>>> won't
>>>> be usable from libvirt.  However it does not pose any security 
>>>> issue as
>>>> it only allows others to check the path.  When qemu is
launched, it
>>>> has
>>>> the proper "label", meaning uid:gid to access the
file so it will be
>>>> able to read/write or whatever permissions you set there.
It's just
>>>> that libvirt does some checks that the path exists for example.
>>>>
>>>> Hope that's understandable and it will resolve your issue
permanently.
>>>>
>>>> Have a nice day,
>>>> Martin
>>>>
>>>>
>>>> _______________________________________________
>>>> libvirt-users mailing list
>>>> libvirt-users@redhat.com
>>>> https://www.redhat.com/mailman/listinfo/libvirt-users
>>> The only reason I said that this might be a 'workaround' is
due to John
>>> Farlan commenting that he'll look at this later on.  Ideally
the
>>> opennebula community keeps the other permissions to nill and
presumably
>>> they work on NFSv3 per the forum topic I included earlier from
them.
>>> But if setting the permissions on nobody to allow for the 
>>> functionality,
>>> I would be comfortable with that.
>>>
>> Martin and I were taking different paths...  But yes, it certainly
makes
>> sense given your error message about canonical path and the need for
>> eXecute permissions... I think I started wondering about that first,
but
>> then jumped into the NFS pool because that's what my reference
point is
>> for root-squash. Since root squash essentially sends root requests as
>> "nfsnobody" (IOW: others not the user or group), then the
"o+x" approach
>> is the solution if you're going directly at the file.
>>
>> John
>
> Yes, appears the o+x is the only way right now.  It definitely tries 
> to access the share as root though, on CentOS 7 since I also tried to 
> add nfsnobody and nobody to the oneadmin group and that did not work 
> either.  Seems OpenNebula doesn't have this issue with NFSv3 running 
> on Ubuntu:
>
> [root@mdskvm-p01 ~]# rmdir /tmp/netfs-rootsquash-pool
> [root@mdskvm-p01 ~]# cat nfs.xml
> <pool type='netfs'>
>      <name>rootsquash</name>
>      <source>
>          <host name='opennebula01'/>
>          <dir path='/var/lib/one'/>
>          <format type='nfs'/>
>      </source>
>      <target>
>          <path>/tmp/netfs-rootsquash-pool</path>
>          <permissions>
>              <mode>0755</mode>
>              <owner>9869</owner>
>              <group>9869</group>
>          </permissions>
>      </target>
> </pool>
> [root@mdskvm-p01 ~]#
> [root@mdskvm-p01 ~]#
>
> [root@mdskvm-p01 ~]# virsh pool-define nfs.xml
> Pool rootsquash defined from nfs.xml
>
> [root@mdskvm-p01 ~]# virsh pool-build rootsquash
> Pool rootsquash built
>
> [root@mdskvm-p01 ~]# virsh pool-start rootsquash
> error: Failed to start pool rootsquash
> error: cannot open path '/tmp/netfs-rootsquash-pool': Permission
denied
>
> [root@mdskvm-p01 ~]# virsh vol-list rootsquash
> error: Failed to list volumes
> error: Requested operation is not valid: storage pool 'rootsquash'
is
> not active
>
> [root@mdskvm-p01 ~]# ls -altri /tmp/netfs-rootsquash-pool
> total 4
>      133 drwxrwxrwt. 14 root     root     4096 Apr 14 00:05 ..
> 68785924 drwxr-xr-x   2 oneadmin oneadmin    6 Apr 14 00:05 .
> [root@mdskvm-p01 ~]#
>
> [root@mdskvm-p01 ~]# id oneadmin
> uid=9869(oneadmin) gid=9869(oneadmin) 
> groups=9869(oneadmin),992(libvirt),36(kvm)
> [root@mdskvm-p01 ~]# id nobody
> uid=99(nobody) gid=99(nobody) groups=99(nobody),9869(oneadmin)
> [root@mdskvm-p01 ~]# id nfsnobody
> uid=65534(nfsnobody) gid=65534(nfsnobody) 
> groups=65534(nfsnobody),9869(oneadmin)
> [root@mdskvm-p01 ~]# id root
> uid=0(root) gid=0(root) groups=0(root)
> [root@mdskvm-p01 ~]#
>
> [root@mdskvm-p01 ~]# ps -ef|grep -i libvirtd
> root       352 31058  0 00:31 pts/1    00:00:00 grep --color=auto -i 
> libvirtd
> root      1459     1  0 Apr11 ?        00:07:40 /usr/sbin/libvirtd 
> --listen --config /etc/libvirt/libvirtd.conf
> [root@mdskvm-p01 ~]#
>
>
>
> [root@mdskvm-p01 ~]# umount /var/lib/one
> [root@mdskvm-p01 ~]# mount --no-canonicalize /var/lib/one
> [root@mdskvm-p01 ~]# umount /var/lib/one
> [root@mdskvm-p01 ~]# mount /var/lib/one
> [root@mdskvm-p01 ~]# mount|tail -n 1
> 192.168.0.70:/var/lib/one on /var/lib/one type nfs4 
>
(rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.60,local_lock=none,addr=192.168.0.70)
> [root@mdskvm-p01 ~]# umount /var/lib/one
> [root@mdskvm-p01 ~]# mount --no-canonicalize /var/lib/one
> [root@mdskvm-p01 ~]# mount|tail -n 1
> 192.168.0.70:/var/lib/one on /var/lib/one type nfs4 
>
(rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.60,local_lock=none,addr=192.168.0.70)
> [root@mdskvm-p01 ~]# su - oneadmin
> Last login: Thu Apr 14 00:27:59 EDT 2016 on pts/0
> [oneadmin@mdskvm-p01 ~]$ virsh -d 1 --connect qemu:///system create 
> /var/lib/one//datastores/0/47/deployment.0
> create: file(optdata): /var/lib/one//datastores/0/47/deployment.0
> error: Failed to create domain from 
> /var/lib/one//datastores/0/47/deployment.0
> error: can't canonicalize path
'/var/lib/one//datastores/0/47/disk.1':
> Permission denied
> [oneadmin@mdskvm-p01 ~]$
>
>
>
>
> CONTROLLER ( NFS Server )
>
> [oneadmin@opennebula01 one]$ ls -ld 
> /var{,/lib{,/one{,/datastores{,/0{,/47{,/disk.1}}}}}}
> drwxr-xr-x. 19 root     root       4096 Apr  4 21:26 /var
> drwxr-xr-x. 28 root     root       4096 Apr 13 03:30 /var/lib
> drwxr-x---. 12 oneadmin oneadmin   4096 Apr 14 00:40 /var/lib/one
> drwxrwxr-x   6 oneadmin oneadmin     46 Mar 31 02:44 
> /var/lib/one/datastores
> drwxrwxr-x   8 oneadmin oneadmin     60 Apr 13 23:31 
> /var/lib/one/datastores/0
> drwxrwxr-x   2 oneadmin oneadmin     68 Apr 13 23:32 
> /var/lib/one/datastores/0/47
> -rw-r--r--   1 oneadmin oneadmin 372736 Apr 13 23:32 
> /var/lib/one/datastores/0/47/disk.1
> [oneadmin@opennebula01 one]$
>
>
>
> NODE ( NFS Client )
>
> [oneadmin@mdskvm-p01 ~]$ ls -ld 
> /var{,/lib{,/one{,/datastores{,/0{,/47{,/disk.1}}}}}}
> drwxr-xr-x. 21 root     root       4096 Apr 11 07:10 /var
> drwxr-xr-x. 45 root     root       4096 Apr 13 04:11 /var/lib
> drwxr-x---  12 oneadmin oneadmin   4096 Apr 14 00:39 /var/lib/one
> drwxrwxr-x   6 oneadmin oneadmin     46 Mar 31 02:44 
> /var/lib/one/datastores
> drwxrwxr-x   8 oneadmin oneadmin     60 Apr 13 23:31 
> /var/lib/one/datastores/0
> drwxrwxr-x   2 oneadmin oneadmin     68 Apr 13 23:32 
> /var/lib/one/datastores/0/47
> -rw-r--r--   1 oneadmin oneadmin 372736 Apr 13 23:32 
> /var/lib/one/datastores/0/47/disk.1
> [oneadmin@mdskvm-p01 ~]$
>
>
>
> Cheers,
> Tom K.
>
-------------------------------------------------------------------------------------
>
> Living on earth is expensive, but it includes a free trip around the sun.
>
> _______________________________________________
> libvirt-users mailing list
> libvirt-users@redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
+ opennebula runs this as oneadmin:

Wed Apr 13 23:32:40 2016 [Z0][VMM][D]: Message received: LOG I 47 + echo 
'Running as user oneadmin'
Wed Apr 13 23:32:40 2016 [Z0][VMM][D]: Message received: LOG I 47 ++ 
virsh --connect qemu:///system create 
/var/lib/one//datastores/0/47/deployment.0
Wed Apr 13 23:32:40 2016 [Z0][VMM][D]: Message received: LOG I 47 error: 
Failed to create domain from /var/lib/one//datastores/0/47/deployment.0
Wed Apr 13 23:32:40 2016 [Z0][VMM][D]: Message received: LOG I 47 error: 
can't canonicalize path '/var/lib/one//datastores/0/47/disk.1': 
Permission denied

Cheers,
TK

Reasonably Related Threads

Search for more apparently analagous threads

libvirt users - Apr 2016 - Re: [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

Re: [libvirt-users] [libvirt] Libvirtd running as root tries to access oneadmin (OpenNebula) NFS mount but throws: error: can’t canonicalize path

Reasonably Related Threads