New to Solairs/ZFS and having a difficult time getting ZFS, NFS and ACLs
all working together, properly. Trying access/use zfs shared
filesystems on a linux client. When I access the dir/files on the linux
client, my permissions do not carry over, nor do the newly created
files, and I can not create new files/dirs. The permissions/owner on
the zfs share are set so the owner (mfitzpat) is allowed to do
everything, but permissions are not carrying over via NFS to the linux
client. I have googled/read and can not get it right. I think this
has something to do with NSF4, but I can not figure it out.
Any help appreciated
Mary Ellen
Running Solaris10 5/09 (u7) on a SunFire x4540 (hecate) with ZFS and zfs
shares automounted to Centos5 client (nona-man).
Running NIS on nona-man(Centos5) and hecate (zfs) is a client. All
works well.
I have created the following zfs filesystems to share and have sharenfs=on
hecate:/zp-ext/spartans/umass> zfs get sharenfs
zp-ext/spartans/umass sharenfs on inherited from
zp-ext/spartans
zp-ext/spartans/umass/mfitzpat sharenfs on inherited from
zp-ext/spartans
set up inheritance:
hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough
zp-ext/spartans/umass
hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough
zp-ext/spartans/umass/mfitzpat
hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough
zp-ext/spartans/umass
hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough
zp-ext/spartans/umass/mfitzpat
Set owner:group:
hecate:/zp-ext/spartans/umass> chown mfitzpat:umass mfitzpat
hecate:/zp-ext/spartans/umass> ls -l
total 5
drwxr-xr-x 2 mfitzpat umass 2 Apr 28 13:18 mfitzpat
Permissions:
hecate:/zp-ext/spartans/umass> ls -dv mfitzpat
drwxr-xr-x 2 mfitzpat umass 2 Apr 28 14:06 mfitzpat
0:owner@::deny
1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory
/append_data/write_xattr/execute/write_attributes/write_acl
/write_owner:allow
2:group@:add_file/write_data/add_subdirectory/append_data:deny
3:group@:list_directory/read_data/execute:allow
4:everyone@:add_file/write_data/add_subdirectory/append_data/write_xattr
/write_attributes/write_acl/write_owner:deny
5:everyone@:list_directory/read_data/read_xattr/execute/read_attributes
/read_acl/synchronize:allow
I can access, create/delete files/dirs on the zfs system and permissions
hold.
[mfitzpat at hecate mfitzpat]$ touch foo
[mfitzpat at hecate mfitzpat]$ ls -l
total 1
-rw-r--r-- 1 mfitzpat umass 0 Apr 28 14:18 foo
When I try to access the dir/files on the linux client, my permissions
do no carry over, nor do the newly created files, and I can not create
new files/dirs.
[mfitzpat at nona-man umass]$ ls -l
drwxr-xr-x+ 2 root root 2 Apr 28 13:18 mfitzpat
[mfitzpat at nona-man mfitzpat]$ pwd
/fs/umass/mfitzpat
[mfitzpat at nona-man mfitzpat]$ ls
[mfitzpat at nona-man mfitzpat]$
--
Thanks
Mary Ellen
Mary Ellen FitzPatrick
Systems Analyst
Bioinformatics
Boston University
24 Cummington St.
Boston, MA 02215
office 617-358-2771
cell 617-797-7856
mfitzpat at bu.edu
Hi Mary Ellen, We were looking at this problem and are unsure what the problem is... To rule out NFS as the root cause, could you create and share a test ZFS file system without any ACLs to see if you can access the data from the Linux client? Let us know the result of your test. Thanks, Cindy On 04/28/10 12:54, Mary Ellen Fitzpatrick wrote:> New to Solairs/ZFS and having a difficult time getting ZFS, NFS and ACLs > all working together, properly. Trying access/use zfs shared > filesystems on a linux client. When I access the dir/files on the linux > client, my permissions do not carry over, nor do the newly created > files, and I can not create new files/dirs. The permissions/owner on > the zfs share are set so the owner (mfitzpat) is allowed to do > everything, but permissions are not carrying over via NFS to the linux > client. I have googled/read and can not get it right. I think this > has something to do with NSF4, but I can not figure it out. > > Any help appreciated > Mary Ellen > > Running Solaris10 5/09 (u7) on a SunFire x4540 (hecate) with ZFS and zfs > shares automounted to Centos5 client (nona-man). > Running NIS on nona-man(Centos5) and hecate (zfs) is a client. All > works well. > > I have created the following zfs filesystems to share and have sharenfs=on > hecate:/zp-ext/spartans/umass> zfs get sharenfs > zp-ext/spartans/umass sharenfs on inherited from > zp-ext/spartans > zp-ext/spartans/umass/mfitzpat sharenfs on inherited from > zp-ext/spartans > > set up inheritance: > hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough > zp-ext/spartans/umass > hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough > zp-ext/spartans/umass/mfitzpat > hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough > zp-ext/spartans/umass > hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough > zp-ext/spartans/umass/mfitzpat > > Set owner:group: > hecate:/zp-ext/spartans/umass> chown mfitzpat:umass mfitzpat > hecate:/zp-ext/spartans/umass> ls -l > total 5 > drwxr-xr-x 2 mfitzpat umass 2 Apr 28 13:18 mfitzpat > > Permissions: > hecate:/zp-ext/spartans/umass> ls -dv mfitzpat > drwxr-xr-x 2 mfitzpat umass 2 Apr 28 14:06 mfitzpat > 0:owner@::deny > 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory > /append_data/write_xattr/execute/write_attributes/write_acl > /write_owner:allow > 2:group@:add_file/write_data/add_subdirectory/append_data:deny > 3:group@:list_directory/read_data/execute:allow > > 4:everyone@:add_file/write_data/add_subdirectory/append_data/write_xattr > /write_attributes/write_acl/write_owner:deny > 5:everyone@:list_directory/read_data/read_xattr/execute/read_attributes > /read_acl/synchronize:allow > > I can access, create/delete files/dirs on the zfs system and permissions > hold. > [mfitzpat at hecate mfitzpat]$ touch foo > [mfitzpat at hecate mfitzpat]$ ls -l > total 1 > -rw-r--r-- 1 mfitzpat umass 0 Apr 28 14:18 foo > > When I try to access the dir/files on the linux client, my permissions > do no carry over, nor do the newly created files, and I can not create > new files/dirs. > [mfitzpat at nona-man umass]$ ls -l > drwxr-xr-x+ 2 root root 2 Apr 28 13:18 mfitzpat > > [mfitzpat at nona-man mfitzpat]$ pwd > /fs/umass/mfitzpat > [mfitzpat at nona-man mfitzpat]$ ls > [mfitzpat at nona-man mfitzpat]$ > >
I setup the share and mounted on linux client, permissions did not carry over from zfs share. hecate:~> zfs create zp-ext/test/mfitzpat hecate:/zp-ext/test> zfs get sharenfs zp-ext/test/mfitzpat NAME PROPERTY VALUE SOURCE zp-ext/test/mfitzpat sharenfs on inherited from zp-ext hecate:/zp-ext/test> chown -R mfitzpat:umass mfitzpat updated auto.home on linux client(nona-man) test -rw,hard,intr hecate:/zp-ext/test nona-man:/# cd /fs/test nona-man:/fs/test# ls -l total 3 drwxr-xr-x+ 2 root root 2 Apr 29 11:15 mfitzpat Permissions did not carry over from zfs share. Willing test/try next step. Mary Ellen Cindy Swearingen wrote:> Hi Mary Ellen, > > We were looking at this problem and are unsure what the problem is... > > To rule out NFS as the root cause, could you create and share a test ZFS > file system without any ACLs to see if you can access the data from the > Linux client? > > Let us know the result of your test. > > Thanks, > > Cindy > On 04/28/10 12:54, Mary Ellen Fitzpatrick wrote: > >> New to Solairs/ZFS and having a difficult time getting ZFS, NFS and ACLs >> all working together, properly. Trying access/use zfs shared >> filesystems on a linux client. When I access the dir/files on the linux >> client, my permissions do not carry over, nor do the newly created >> files, and I can not create new files/dirs. The permissions/owner on >> the zfs share are set so the owner (mfitzpat) is allowed to do >> everything, but permissions are not carrying over via NFS to the linux >> client. I have googled/read and can not get it right. I think this >> has something to do with NSF4, but I can not figure it out. >> >> Any help appreciated >> Mary Ellen >> >> Running Solaris10 5/09 (u7) on a SunFire x4540 (hecate) with ZFS and zfs >> shares automounted to Centos5 client (nona-man). >> Running NIS on nona-man(Centos5) and hecate (zfs) is a client. All >> works well. >> >> I have created the following zfs filesystems to share and have sharenfs=on >> hecate:/zp-ext/spartans/umass> zfs get sharenfs >> zp-ext/spartans/umass sharenfs on inherited from >> zp-ext/spartans >> zp-ext/spartans/umass/mfitzpat sharenfs on inherited from >> zp-ext/spartans >> >> set up inheritance: >> hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough >> zp-ext/spartans/umass >> hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough >> zp-ext/spartans/umass/mfitzpat >> hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough >> zp-ext/spartans/umass >> hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough >> zp-ext/spartans/umass/mfitzpat >> >> Set owner:group: >> hecate:/zp-ext/spartans/umass> chown mfitzpat:umass mfitzpat >> hecate:/zp-ext/spartans/umass> ls -l >> total 5 >> drwxr-xr-x 2 mfitzpat umass 2 Apr 28 13:18 mfitzpat >> >> Permissions: >> hecate:/zp-ext/spartans/umass> ls -dv mfitzpat >> drwxr-xr-x 2 mfitzpat umass 2 Apr 28 14:06 mfitzpat >> 0:owner@::deny >> 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory >> /append_data/write_xattr/execute/write_attributes/write_acl >> /write_owner:allow >> 2:group@:add_file/write_data/add_subdirectory/append_data:deny >> 3:group@:list_directory/read_data/execute:allow >> >> 4:everyone@:add_file/write_data/add_subdirectory/append_data/write_xattr >> /write_attributes/write_acl/write_owner:deny >> 5:everyone@:list_directory/read_data/read_xattr/execute/read_attributes >> /read_acl/synchronize:allow >> >> I can access, create/delete files/dirs on the zfs system and permissions >> hold. >> [mfitzpat at hecate mfitzpat]$ touch foo >> [mfitzpat at hecate mfitzpat]$ ls -l >> total 1 >> -rw-r--r-- 1 mfitzpat umass 0 Apr 28 14:18 foo >> >> When I try to access the dir/files on the linux client, my permissions >> do no carry over, nor do the newly created files, and I can not create >> new files/dirs. >> [mfitzpat at nona-man umass]$ ls -l >> drwxr-xr-x+ 2 root root 2 Apr 28 13:18 mfitzpat >> >> [mfitzpat at nona-man mfitzpat]$ pwd >> /fs/umass/mfitzpat >> [mfitzpat at nona-man mfitzpat]$ ls >> [mfitzpat at nona-man mfitzpat]$ >> >> >>-- Thanks Mary Ellen Mary Ellen FitzPatrick Systems Analyst Bioinformatics Boston University 24 Cummington St. Boston, MA 02215 office 617-358-2771 cell 617-797-7856 mfitzpat at bu.edu
Hi Mary Ellen, I''m not really qualified to help you troubleshoot this problem. Other community members on this list have wrestled with similar problems and I hope they will comment... Your Linux client doesn''t seem to be suffering from the nobody problem because you see mfitzpat on nona-man so UID/GIDs are translated correctly. This issue has come up often enough that I will start tracking this in our troubleshooting wiki as soon as we get more feedback. Thanks, Cindy On 04/29/10 09:23, Mary Ellen Fitzpatrick wrote:> I setup the share and mounted on linux client, permissions did not carry > over from zfs share. > > > hecate:~> zfs create zp-ext/test/mfitzpat > hecate:/zp-ext/test> zfs get sharenfs zp-ext/test/mfitzpat > NAME PROPERTY VALUE SOURCE > zp-ext/test/mfitzpat sharenfs on inherited from zp-ext > hecate:/zp-ext/test> chown -R mfitzpat:umass mfitzpat > > updated auto.home on linux client(nona-man) > test -rw,hard,intr hecate:/zp-ext/test > > nona-man:/# cd /fs/test > nona-man:/fs/test# ls -l > total 3 > drwxr-xr-x+ 2 root root 2 Apr 29 11:15 mfitzpat > > Permissions did not carry over from zfs share. > Willing test/try next step. > > Mary Ellen > > > > > Cindy Swearingen wrote: >> Hi Mary Ellen, >> >> We were looking at this problem and are unsure what the problem is... >> >> To rule out NFS as the root cause, could you create and share a test >> ZFS file system without any ACLs to see if you can access the data >> from the >> Linux client? >> >> Let us know the result of your test. >> >> Thanks, >> >> Cindy >> On 04/28/10 12:54, Mary Ellen Fitzpatrick wrote: >> >>> New to Solairs/ZFS and having a difficult time getting ZFS, NFS and >>> ACLs all working together, properly. Trying access/use zfs shared >>> filesystems on a linux client. When I access the dir/files on the >>> linux client, my permissions do not carry over, nor do the newly >>> created files, and I can not create new files/dirs. The >>> permissions/owner on the zfs share are set so the owner (mfitzpat) is >>> allowed to do everything, but permissions are not carrying over via >>> NFS to the linux client. I have googled/read and can not get it >>> right. I think this has something to do with NSF4, but I can not >>> figure it out. >>> >>> Any help appreciated >>> Mary Ellen >>> >>> Running Solaris10 5/09 (u7) on a SunFire x4540 (hecate) with ZFS and >>> zfs shares automounted to Centos5 client (nona-man). >>> Running NIS on nona-man(Centos5) and hecate (zfs) is a client. All >>> works well. >>> >>> I have created the following zfs filesystems to share and have >>> sharenfs=on >>> hecate:/zp-ext/spartans/umass> zfs get sharenfs >>> zp-ext/spartans/umass sharenfs on inherited >>> from zp-ext/spartans >>> zp-ext/spartans/umass/mfitzpat sharenfs on inherited >>> from zp-ext/spartans >>> >>> set up inheritance: >>> hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough >>> zp-ext/spartans/umass >>> hecate:/zp-ext/spartans/umass> zfs set aclinherit=passthrough >>> zp-ext/spartans/umass/mfitzpat >>> hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough >>> zp-ext/spartans/umass >>> hecate:/zp-ext/spartans/umass> zfs set aclmode=passthrough >>> zp-ext/spartans/umass/mfitzpat >>> >>> Set owner:group: >>> hecate:/zp-ext/spartans/umass> chown mfitzpat:umass mfitzpat >>> hecate:/zp-ext/spartans/umass> ls -l >>> total 5 >>> drwxr-xr-x 2 mfitzpat umass 2 Apr 28 13:18 mfitzpat >>> >>> Permissions: >>> hecate:/zp-ext/spartans/umass> ls -dv mfitzpat >>> drwxr-xr-x 2 mfitzpat umass 2 Apr 28 14:06 mfitzpat >>> 0:owner@::deny >>> >>> 1:owner@:list_directory/read_data/add_file/write_data/add_subdirectory >>> /append_data/write_xattr/execute/write_attributes/write_acl >>> /write_owner:allow >>> 2:group@:add_file/write_data/add_subdirectory/append_data:deny >>> 3:group@:list_directory/read_data/execute:allow >>> >>> 4:everyone@:add_file/write_data/add_subdirectory/append_data/write_xattr >>> /write_attributes/write_acl/write_owner:deny >>> >>> 5:everyone@:list_directory/read_data/read_xattr/execute/read_attributes >>> /read_acl/synchronize:allow >>> >>> I can access, create/delete files/dirs on the zfs system and >>> permissions hold. >>> [mfitzpat at hecate mfitzpat]$ touch foo >>> [mfitzpat at hecate mfitzpat]$ ls -l >>> total 1 >>> -rw-r--r-- 1 mfitzpat umass 0 Apr 28 14:18 foo >>> >>> When I try to access the dir/files on the linux client, my >>> permissions do no carry over, nor do the newly created files, and I >>> can not create new files/dirs. >>> [mfitzpat at nona-man umass]$ ls -l >>> drwxr-xr-x+ 2 root root 2 Apr 28 13:18 mfitzpat >>> >>> [mfitzpat at nona-man mfitzpat]$ pwd >>> /fs/umass/mfitzpat >>> [mfitzpat at nona-man mfitzpat]$ ls >>> [mfitzpat at nona-man mfitzpat]$ >>> >>> >>> >
On Thu, 29 Apr 2010, Mary Ellen Fitzpatrick wrote:> hecate:/zp-ext/test> zfs get sharenfs zp-ext/test/mfitzpat[...]> hecate:/zp-ext/test> chown -R mfitzpat:umass mfitzpat[...]> test -rw,hard,intr hecate:/zp-ext/test[...]> drwxr-xr-x+ 2 root root 2 Apr 29 11:15 mfitzpatUnless I''m missing something, you chown''d the filesystem zp-ext/test/mfitzpat but you mounted the filesystem zp-ext/test; hence you''re seeing the mount point for the mfitzpat filesystem in the zp-ext/test filesystem over NFS, not the actual zp-ext/test/mfitzpat filesystem. Pending the availability of mirror mounts (http://hub.opensolaris.org/bin/download/Project+nfs-namespace/files/mm-PRS-open.html) you need to mount each ZFS filesystem you''re exporting via NFS separately. -- Paul B. Henson | (909) 979-6361 | http://www.csupomona.edu/~henson/ Operating Systems and Network Analyst | henson at csupomona.edu California State Polytechnic University | Pomona CA 91768
Yes, you were correct, I chown zp-ext/test/mfitzpat and tried to mount /zp-ext/test When I mounted /zp-ext/test/mfitzpat on the remote linux system, as /fs/mfitzpat I get the correct permissions and I can create new files/dirs. [mfitzpat at nona-man test]$ ls -l total 4 drwxr-xr-x+ 2 mfitzpat umass 2 May 3 12:42 bar -rw-r--r--+ 1 mfitzpat umass 0 May 3 12:41 foo -rw-r--r--+ 1 mfitzpat umass 0 May 3 13:10 foo.1 I want to use autofs on the remote clients, as I have many dirs that need to be exported from /zp-ext/test/* Here is the auto.home on the client, as setup for the user mfitzpat. I really do not want to edit the auto.home file for each user. mfitzpat -rw,hard,intr hecate:/zp-ext/test/mfitzpat Is there a way to set permissions so that the /etc/auto.home file on the clients does not list every exported dir/mount point? Thanks for your help Mary Ellen Paul B. Henson wrote:> On Thu, 29 Apr 2010, Mary Ellen Fitzpatrick wrote: > > >> hecate:/zp-ext/test> zfs get sharenfs zp-ext/test/mfitzpat >> > [...] > >> hecate:/zp-ext/test> chown -R mfitzpat:umass mfitzpat >> > [...] > >> test -rw,hard,intr hecate:/zp-ext/test >> > [...] > >> drwxr-xr-x+ 2 root root 2 Apr 29 11:15 mfitzpat >> > > Unless I''m missing something, you chown''d the filesystem > zp-ext/test/mfitzpat but you mounted the filesystem zp-ext/test; hence > you''re seeing the mount point for the mfitzpat filesystem in the > zp-ext/test filesystem over NFS, not the actual zp-ext/test/mfitzpat > filesystem. > > Pending the availability of mirror mounts > > (http://hub.opensolaris.org/bin/download/Project+nfs-namespace/files/mm-PRS-open.html) > > you need to mount each ZFS filesystem you''re exporting via NFS separately. > > >-- Thanks Mary Ellen Mary Ellen FitzPatrick Systems Analyst Bioinformatics Boston University 24 Cummington St. Boston, MA 02215 office 617-358-2771 cell 617-797-7856 mfitzpat at bu.edu
>>>>> "mef" == Mary Ellen Fitzpatrick <mfitzpat at bu.edu> writes:mef> Is there a way to set permissions so that the /etc/auto.home mef> file on the clients does not list every exported dir/mount mef> point? If I understand the question right, then, no. These maps are very traditional from the earliest days of NIS and need to be managed centrally, and they should match the structure of filesystems exported from servers exactly. The new scheme of ``mirror mounts'''' and ``referrals'''' which does away with the global automount map and sprinkles bits of the map onto individual servers, is oft-discussed but seldom implemented, and it''s not at all traditional so it''s unclear to me that it''s ever going to become The Way even though everyone talks about it like it''s _fait accompli_. I only mention it to poison the well: if someone tries to discuss this with you, you should immediately close your ears because they are only dreaming for now, and none of it works yet. Unfortunately, autofs implementations'' quality varies widely. I think Linux is on their...fourth? rewrite of the whole automount framework, and Mac OS X on at least their second if not third. I found Mac OS X''s one is poor at handling nested mounts like what you''re doing compared to the solaris one. The apple people sneakily altered the automounter documentation to remove all examples showing nested mounts, without actually documenting frankly the limitation which surely prompted them to alter the examples. slimey fucks. You can work around their fail using the ''net'' option but this prevents assembling subtrees from several different servers. Each of your nested subtrees must be from the same server when using the ''net'' option workaround because you lose the right to choose where they''re mounted: http://web.ivy.net/~carton/rant/macos-automounter.html#9050149 The linux one will do nested subtrees, but I think you need to express the entire subtree as a single automount record, with a single trigger. This is different from Mac OS X with-workaround which will (provided you use ''net'') miraculously assemble a view of the entire subtree from several dscl records which in theory could even be on LDAP. so, Linux will automount and unmount everything together, while Mac OS X will not. You might reasonably wish to have the mountpoints within the automounted filesystem turn into triggers themselves so that parts of the subtree are only mounted just as deeply as and only along the branch needed to satisfy the trigger---that way a subtree could be assembled from many servers, and if the map for a deep corner of the subtree were changed and pushed, clients could start obeying the changes sooner. but I think on Linux this won''t work. not sure it works anywhere though. I guess it sort of works on Mac OS X with heavy caveat, but not sure about Solaris. carton -hard,intr,noacl / cash:/export/home/& \ /VDI cash:/export/home/&/VDI but although Linux beats the Mac here, the linux one is shit at handling direct mounts: if you give it a subdirectory in which it owns all of the fake files, like /home, it works ok. but if you want it to for example automount /arrchive (just the one filesystem onto /arrchive from one share on one server) I found it hardly works at all. so I have /arrchive automounted on my Solaris boxes, and /remedial-automount/arrchive with symlink /archive -> remedial-automount/arrchive on Linux boxes. so much for one traditional centrally-managed map. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100503/7ad56efb/attachment.bin>
On Mon, 3 May 2010, Mary Ellen Fitzpatrick wrote:> I want to use autofs on the remote clients, as I have many dirs that > need to be exported from /zp-ext/test/* > > Here is the auto.home on the client, as setup for the user mfitzpat. I > really do not want to edit the auto.home file for each user. > mfitzpat -rw,hard,intr hecate:/zp-ext/test/mfitzpat > > Is there a way to set permissions so that the /etc/auto.home file on the > clients does not list every exported dir/mount point?If all of the directories to be mounted are on the same server in the same place, you can use a wildcard entry: * -rw,hard,intr hecate:/zp-ext/test/& -- Paul B. Henson | (909) 979-6361 | http://www.csupomona.edu/~henson/ Operating Systems and Network Analyst | henson at csupomona.edu California State Polytechnic University | Pomona CA 91768