zfs discuss - Oct 2009 - NFS sgid directory interoperability with Linux

We''re running Solaris 10 with ZFS to provide home and group directory
file
space over NFSv4. We''ve run into an interoperability issue between the
Solaris NFS server and the Linux NFS client regarding the sgid bit on
directories and assigning appropriate group ownership on newly created
subdirectories.

If a directory exists with the sgid bit set owned by a group other than
the user''s primary group, new directories created in that directory are
owned by the primary group rather than by the group of the parent
directory.

Evidently, the Solaris NFS server assumes the client will specify the
correct owner of the directory, whereas the Linux NFS client assumes the
server is in charge of implementing the sgid functionality and will assign
the right group itself. As such, with a Solaris server and a Linux client
the functionality is simply broken :(.

This poses a considerable security issue, as the GROUP@ inherited ACL now
provides access to the primary group of the user rather than the intended
group, which as you might imagine is somewhat problematic.

Ideally, it seems that the server should be responsible for this, rather
than the client voluntarily enforcing it. Is this functionality strictly
defined anywhere, or is it implementation dependent? You''d think
something like this would have turned up in an interoperability bake-off at
some point.

Thanks for any information...


-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

Paul B. Henson wrote:
> We''re running Solaris 10 with ZFS to provide home and group
directory file
> space over NFSv4. We''ve run into an interoperability issue between
the
> Solaris NFS server and the Linux NFS client regarding the sgid bit on
> directories and assigning appropriate group ownership on newly created
> subdirectories.
> 
> If a directory exists with the sgid bit set owned by a group other than
> the user''s primary group, new directories created in that
directory are
> owned by the primary group rather than by the group of the parent
> directory.
> 
> Evidently, the Solaris NFS server assumes the client will specify the
> correct owner of the directory, whereas the Linux NFS client assumes the
> server is in charge of implementing the sgid functionality and will assign
> the right group itself. As such, with a Solaris server and a Linux client
> the functionality is simply broken :(.
> 
> This poses a considerable security issue, as the GROUP@ inherited ACL now
> provides access to the primary group of the user rather than the intended
> group, which as you might imagine is somewhat problematic.
> 
> Ideally, it seems that the server should be responsible for this, rather
> than the client voluntarily enforcing it. Is this functionality strictly
> defined anywhere, or is it implementation dependent? You''d think
> something like this would have turned up in an interoperability bake-off at
> some point.
> 
> Thanks for any information...
> 
> 
Does it only fail under NFS or does it only fail when inheriting an ACL?

I just tried it locally and it appears to work.

# ls -ld test.dir
drwsr-sr-x   2 marks    storage        4 Oct 12 16:45 test.dir

my primary group is "staff"

$ touch file
$ ls -l file
-rw-r--r--   1 marks    storage        0 Oct 12 16:49 file


    -Mark

On Mon, 12 Oct 2009, Mark Shellenbaum wrote:
> Does it only fail under NFS or does it only fail when inheriting an ACL?
It only fails over NFS from a Linux client, locally it works fine, and from
a Solaris client it works fine. It also only seems to fail on directories,
files receive the correct group ownership:

$ uname -a
Linux damien 2.6.27-gentoo-r8 #7 SMP Tue May 26 13:15:08 PDT 2009 x86_64
Dual Core AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux

$ id
uid=1005(henson) gid=1012(csupomona)

$ mount | grep henson
kyle.unx.csupomona.edu:/export/user/henson on /user/henson type nfs4
(rw,sec=krb5p,clientaddr=134.71.247.8,sloppy,addr=134.71.247.14)

$ ls -ld .
drwx--s--x 3 henson iit 4 Oct 12 15:58 .

$ touch foo
$ mkdir bar
$ ls -l

total 1
drwxr-sr-x 2 henson csupomona 2 Oct 12 15:58 bar
-rw-r--r-- 1 henson iit       0 Oct 12 15:58 foo

New directory group ownership is wrong whether the containing directory has
an inheritable ACL or not.

I only have ZFS filesystems exported right now, but I assume it would
behave the same for ufs. The underlying issue seems to be the Sun NFS
server expects the NFS client to apply the sgid bit itself and create the
new directory with the parent directory''s group, while the Linux NFS
client
expects the server to enforce the sgid bit.


-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

>I only have ZFS filesystems exported right now, but I assume it would
>behave the same for ufs. The underlying issue seems to be the Sun NFS
>server expects the NFS client to apply the sgid bit itself and create the
>new directory with the parent directory''s group, while the Linux
NFS client
>expects the server to enforce the sgid bit.

If you look at the code in ufs and zfs, you''ll see that they both
create
the mode correctly and the same code is used through NFS.

There''s another scenario: the Linux client updates the attributes
after creating the file/directory/

Casper

"Paul B. Henson" <henson at acm.org> wrote:
>
> We''re running Solaris 10 with ZFS to provide home and group
directory file
> space over NFSv4. We''ve run into an interoperability issue between
the
> Solaris NFS server and the Linux NFS client regarding the sgid bit on
> directories and assigning appropriate group ownership on newly created
> subdirectories.
The correct behavior would be to assign the group ownership of the parent 
directory to a new directory (instead of using the current process 
credentials) in case that the sgid bit is set in the parent directory. 
Is this your problem?

J?rg

-- 
 EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin
       js at cs.tu-berlin.de                (uni)  
       joerg.schilling at fokus.fraunhofer.de (work) Blog:
http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

On Tue, 13 Oct 2009 Casper.Dik at Sun.COM wrote:
> If you look at the code in ufs and zfs, you''ll see that they both
create
> the mode correctly and the same code is used through NFS.
>
> There''s another scenario: the Linux client updates the attributes
after
> creating the file/directory/
I don''t think that is the case. My colleague Brian captured the network
traffic and analyzed it, and if I understood him correctly the Linux client
issues the mkdir op with no group specified, which per RFC indicates the
server should set the appropriate group. On the Solaris client, the nfs
mkdir op explicitly specifies the group.

Brian is going to follow up shortly with more technical detail.

Thanks...

-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

On Tue, 13 Oct 2009, Joerg Schilling wrote:
> The correct behavior would be to assign the group ownership of the parent
> directory to a new directory (instead of using the current process
> credentials) in case that the sgid bit is set in the parent directory.
> Is this your problem?
Yes, that is exactly our problem -- when a Linux NFSv4 client creates a
directory on a Solaris NFSv4 server when the parent directory has the sgid
bit set and a different group owner then the user''s primary group, the
new
directory is incorrectly created with the primary group as group owner
rather than the parent directory group.

-- 
Paul B. Henson  |  (909) 979-6361  |  http://www.csupomona.edu/~henson/
Operating Systems and Network Analyst  |  henson at csupomona.edu
California State Polytechnic University  |  Pomona CA 91768

On 10/12/2009 04:38 PM, Paul B. Henson wrote:
> I only have ZFS filesystems exported right now, but I assume it would
> behave the same for ufs. The underlying issue seems to be the Sun NFS
> server expects the NFS client to apply the sgid bit itself and create the
> new directory with the parent directory''s group, while the Linux
NFS client
> expects the server to enforce the sgid bit.
> 
When the clients send the opcode CREATE, the Solaris client specifies the parent
directory''s group in attr_vals whereas the Linux client
doesn''t specify a group.
  There appears to be a disparity between the servers in what to do in an SGID 
directory when attr_vals does not specify a group.  On Solaris, this leads the 
server to use the process'' group, but on Linux, the SGID is enforced
and it
takes the group of the parent directory.  The problem arises when the Linux 
client expects the Linux server''s behavior, leading it to not send the
group to
a Solaris server, leading the Solaris server to assume the client wanted to 
ignore the SGID bit.

This issue has been frustrating because there didn''t appear to be any
official
word on which client was right.  However, I did find this in the RFC which may 
indicate that the Solaris server might be at fault.  In 14.2.4, for the opcode 
CREATE, it says this about situations where the group isn''t specified:


Similarly, if createattrs includes neither the group attribute nor a group ACE, 
and if the server''s filesystem both supports and requires the notion of
a group
attribute (or group ACE), the server MUST derive the group attribute (or the 
corresponding owner ACE) for the file. This could be from the RPC
call''s
credentials, such as the group principal if the credentials include it (such as 
with AUTH_SYS), from the group identifier associated with the principal in the 
credentials (for e.g., POSIX systems have a passwd database that has the group 
identifier for every user identifier), inherited from directory the object is 
created in, or whatever else the server''s operating environment or
filesystem
semantics dictate. This applies to the OPEN operation too.


The important phrase being "inherited from directory the object is created
in",
which says to me that the server should enforce the SGID bit if no group is 
specified.  However, reading this closer makes me wonder if this sentence is too
open-ended.  It appears that the Solaris server uses a group principle or group 
identifier and the Linux server inherits from the parent directory.  Both of 
these are valid choices from the list...they just happen to make incompatible 
implementations.