tizo
2022-Mar-21 17:21 UTC
[Gluster-users] NFSv4 permissions issues with an exported glusterfs
I have posted this problem exactly in Server Fault and in Linux NFS, but it has not been answered yet. Maybe you can help me. I have a situation with kernel NFS server. I have two exports with exactly the same ACLs, with full permissions for the informatica at adtest.xx.xx.xx group. One is /exports/directo_informatica/, which is the mount point for an LV with XFS, and the other is /exports/gv0_inf/, which is the mount point for a glusterfs. The first export works right when mounting it remotely with NFS, and accessing it with a user of the group informatica at adtest.xx.xx.xx. The second one doesn't: it can be mounted correctly, but when trying to access it with the same user it gives "Permission denied". If I access directly to the NFS server (ssh) with the same user of the previous tests, I can access both directories inside /exports/ without problems. More details at following: OS: Rocky Linux release 8.5 (Green Obsidian) fstab for the exported directories: /dev/mapper/vg_kvm_sistema-lv_directo_informatica /exports/directo_informatica xfs defaults 0 0 glustersrv02.xx.xx.xx:/gv0_inf /exports/gv0_inf/ glusterfs defaults,acl 0 0 Mount for the exported directories: /dev/mapper/vg_kvm_sistema-lv_directo_informatica on /exports/directo_informatica type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota) glustersrv02.xx.xx.xx:/gv0_inf on /exports/gv0_inf type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,allow_other,max_read=131072) exports file: /exports *(sec=krb5p,secure,rw,sync,no_wdelay,no_subtree_check,root_squash,fsid=0) /exports/directo_informatica *(sec=krb5p,secure,rw,sync,no_wdelay,no_subtree_check,root_squash,mountpoint) /exports/gv0_inf *(sec=krb5p,secure,rw,sync,no_wdelay,no_subtree_check,root_squash,mountpoint,fsid=2) Exported directories ACLs: # getfacl /exports/directo_informatica/ getfacl: Removing leading '/' from absolute path names # file: exports/directo_informatica/ # owner: root # group: root user::rwx user:root:rwx group::r-x group:root:r-x group:informatica at adtest.xx.xx.xx:rwx mask::rwx other::--- default:user::rwx default:user:root:rwx default:group::r-x default:group:root:r-x default:group:informatica at adtest.xx.xx.xx:rwx default:mask::rwx default:other::--- # getfacl /exports/gv0_inf/ getfacl: Removing leading '/' from absolute path names # file: exports/gv0_inf/ # owner: root # group: root user::rwx user:root:rwx group::r-x group:root:r-x group:informatica at adtest.xx.xx.xx:rwx mask::rwx other::--- default:user::rwx default:user:root:rwx default:group::r-x default:group:root:r-x default:group:informatica at adtest.xx.xx.xx:rwx default:mask::rwx default:other::--- Directories mounted remoteley: gluster02.adtest.xx.xx.xx:/directo_informatica on /prueba2 type nfs4 (rw,relatime,vers=4.2,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5p,clientaddr=10.2.100.8,local_lock=none,addr=10.2.100.8) gluster02.adtest.xx.xx.xx:/gv0_inf on /prueba type nfs4 (rw,relatime,vers=4.2,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5p,clientaddr=10.2.100.8,local_lock=none,addr=10.2.100.8) NFSv4 ACLs remotely: $ nfs4_getfacl /prueba2 # file: /prueba2 A::OWNER@:rwaDxtTcCy A::root at idmpru.fnr.gub.uy:rwaDxtcy A::GROUP@:rxtcy A:g:root at idmpru.fnr.gub.uy:rxtcy A:g:informatica at adtest.xx.xx.xx@idmpru.xx.xx.xx:rwaDxtcy A::EVERYONE@:tcy A:fdi:OWNER@:rwaDxtTcCy A:fdi:root at idmpru.xx.xx.xx:rwaDxtcy A:fdi:GROUP@:rxtcy A:fdig:root at idmpru.xx.xx.xx:rxtcy A:fdig:informatica at adtest.xx.xx.xx@idmpru.xx.xx.xx:rwaDxtcy A:fdi:EVERYONE@:tcy $ nfs4_getfacl /prueba # file: /prueba A::OWNER@:rwaDxtTcCy A::GROUP@:rwaDxtcy A::EVERYONE@:tcy The only additional question for this list, is if you think that this problem could be avoided with NFS Ganesha. Any help is appreciated. Thanks very much.
Strahil Nikolov
2022-Mar-28 13:56 UTC
[Gluster-users] NFSv4 permissions issues with an exported glusterfs
When you mount the gluster with 'mount -t glusterfs -o acl gluster_volume_fileserver1:/volume /mnt' and then when you execute 'getfacl /mnt' , what is the output ?I assume (based on the kerberos) that both systems have the same uid/gids . Can you reproduce it , if you remove the krb5 mount options ? Any reason not to use the FUSE client ? (BSD/Unix systems ?) It's far more tested to use NFS Ganesha and Gluster has some scripts to configure HA setup for the NFS.Another not very common (but working ) setup is to recompile the Gluster aource with the gNFS enabled , so you can use the built-in NFS server. Best Regards,Strahil Nikolov On Mon, Mar 28, 2022 at 12:36, tizo<tizone at gmail.com> wrote: I have posted this problem exactly in Server Fault and in Linux NFS, but it has not been answered yet. Maybe you can help me. I have a situation with kernel NFS server. I have two exports with exactly the same ACLs, with full permissions for the informatica at adtest.xx.xx.xx group. One is /exports/directo_informatica/, which is the mount point for an LV with XFS, and the other is /exports/gv0_inf/, which is the mount point for a glusterfs. The first export works right when mounting it remotely with NFS, and accessing it with a user of the group informatica at adtest.xx.xx.xx. The second one doesn't: it can be mounted correctly, but when trying to access it with the same user it gives "Permission denied". If I access directly to the NFS server (ssh) with the same user of the previous tests, I can access both directories inside /exports/ without problems. More details at following: OS: Rocky Linux release 8.5 (Green Obsidian) fstab for the exported directories: /dev/mapper/vg_kvm_sistema-lv_directo_informatica /exports/directo_informatica xfs defaults 0 0 glustersrv02.xx.xx.xx:/gv0_inf /exports/gv0_inf/ glusterfs defaults,acl 0 0 Mount for the exported directories: /dev/mapper/vg_kvm_sistema-lv_directo_informatica on /exports/directo_informatica type xfs (rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota) glustersrv02.xx.xx.xx:/gv0_inf on /exports/gv0_inf type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,allow_other,max_read=131072) exports file: /exports *(sec=krb5p,secure,rw,sync,no_wdelay,no_subtree_check,root_squash,fsid=0) /exports/directo_informatica *(sec=krb5p,secure,rw,sync,no_wdelay,no_subtree_check,root_squash,mountpoint) /exports/gv0_inf *(sec=krb5p,secure,rw,sync,no_wdelay,no_subtree_check,root_squash,mountpoint,fsid=2) Exported directories ACLs: # getfacl /exports/directo_informatica/ getfacl: Removing leading '/' from absolute path names # file: exports/directo_informatica/ # owner: root # group: root user::rwx user:root:rwx group::r-x group:root:r-x group:informatica at adtest.xx.xx.xx:rwx mask::rwx other::--- default:user::rwx default:user:root:rwx default:group::r-x default:group:root:r-x default:group:informatica at adtest.xx.xx.xx:rwx default:mask::rwx default:other::--- # getfacl /exports/gv0_inf/ getfacl: Removing leading '/' from absolute path names # file: exports/gv0_inf/ # owner: root # group: root user::rwx user:root:rwx group::r-x group:root:r-x group:informatica at adtest.xx.xx.xx:rwx mask::rwx other::--- default:user::rwx default:user:root:rwx default:group::r-x default:group:root:r-x default:group:informatica at adtest.xx.xx.xx:rwx default:mask::rwx default:other::--- Directories mounted remoteley: gluster02.adtest.xx.xx.xx:/directo_informatica on /prueba2 type nfs4 (rw,relatime,vers=4.2,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5p,clientaddr=10.2.100.8,local_lock=none,addr=10.2.100.8) gluster02.adtest.xx.xx.xx:/gv0_inf on /prueba type nfs4 (rw,relatime,vers=4.2,rsize=131072,wsize=131072,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=krb5p,clientaddr=10.2.100.8,local_lock=none,addr=10.2.100.8) NFSv4 ACLs remotely: $ nfs4_getfacl /prueba2 # file: /prueba2 A::OWNER@:rwaDxtTcCy A::root at idmpru.fnr.gub.uy:rwaDxtcy A::GROUP@:rxtcy A:g:root at idmpru.fnr.gub.uy:rxtcy A:g:informatica at adtest.xx.xx.xx@idmpru.xx.xx.xx:rwaDxtcy A::EVERYONE@:tcy A:fdi:OWNER@:rwaDxtTcCy A:fdi:root at idmpru.xx.xx.xx:rwaDxtcy A:fdi:GROUP@:rxtcy A:fdig:root at idmpru.xx.xx.xx:rxtcy A:fdig:informatica at adtest.xx.xx.xx@idmpru.xx.xx.xx:rwaDxtcy A:fdi:EVERYONE@:tcy $ nfs4_getfacl /prueba # file: /prueba A::OWNER@:rwaDxtTcCy A::GROUP@:rwaDxtcy A::EVERYONE@:tcy The only additional question for this list, is if you think that this problem could be avoided with NFS Ganesha. Any help is appreciated. Thanks very much. ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users at gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20220328/234d3def/attachment.html>