Gluster users - Sep 2017 - Restrict root clients / experimental patch

Hi All,

I would like to use glusterfs in an environment where storage servers are
managed by an IT service - myself :) - and several users in the
organization can mount the distributed fs. The users are root on their
machines.
As far as I know about glusterfs, a root client user may impersonate any
uid/gid since it provides its uid/gid itself when it talks to the bricks
(like nfsv3).
The thing is, we want to enforce permissions, i.e. user X may only access
files shared with him even if he's root on his machine.
I found a draft spec about glusterfs+kerberos
<https://github.com/gluster/glusterfs-specs/blob/master/under_review/Kerberos.md>
but not much more so I think it's not possible with glusterfs right now,
correct?
(Also I feel that kerberos would be a bit heavy to manage)

---

An simple hack that I found is to add custom uid/gid fields in clients' ssl
certificates. The bricks use the client's uid/gid specified in its
certificate rather than using one specified by the user. This solution has
no effect on performance and there's no need for a central authentication.
The thing that changes is the way client certificates are generated and
glusterfsd needs a small patch.

I did an experimental implementation
<https://github.com/eshard/glusterfs/commit/768bf63154fdc59ba67d5788743adab8679ec5ab>
of this idea. Custom fields "1.2.3.4.5.6.7" and
"1.2.3.4.5.6.8" are used
for uid and gid.
I tried it with custom CA trusted by all bricks and I issued a few client
certificates.
No server configuration is needed when a new client is added, when a client
is revoked the a CRL
<https://en.wikipedia.org/wiki/Certificate_revocation_list> must updated
and pushed to all servers.
By the way I didn't get glusterfs servers to accept my CRLs, do some people
use it?

Notes:
 * groups are not handled right now and since users may change groups
regularly I don't think it would be a great idea to freeze them in a
certificate. The bricks could possibly do an ldap lookup in order to
retrieve and cache the groups for an uid.
 * Clients obviously can't modify their certificates because they are
signed by CA

What do you think of this implementation, is it safe?
Do all client operations use auth_glusterfs_v2_authenticate or did I miss
other codepaths?

Thanks!

Pierre Carru
eshard

PS: By the way I found the source code very clean and well organized, nice
job :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170921/4fdc349c/attachment.html>

Hi,

On 09/21/2017 07:32 PM, Pierre C wrote:
> Hi All,
> 
> I would like to use glusterfs in an environment where storage servers 
> are managed by an IT service - myself :) - and several users in the 
> organization can mount the distributed fs. The users are root on their 
> machines.
> As far as I know about glusterfs, a root client user may impersonate any 
> uid/gid since it provides its uid/gid itself when it talks to the bricks 
> (like nfsv3).
> The thing is, we want to enforce permissions, i.e. user X may only 
> access files shared with him even if he's root on his machine.
> I found a draft spec about glusterfs+kerberos 
>
<https://github.com/gluster/glusterfs-specs/blob/master/under_review/Kerberos.md>
> but not much more so I think it's not possible with glusterfs right
now,
> correct?
> (Also I feel that kerberos would be a bit heavy to manage)
I am not much familiar with how ssl is handled. But from what I 
understand from your statement above, you need to restrict root users. 
Isn't root-squash option enough for it?

Option: server.root-squash
Default Value: off
Description: Map requests from uid/gid 0 to the anonymous uid/gid. Note 
that this does not apply to any other uids or gids that might be equally 
sensitive, such as user bin or group staff.

Option: server.anonuid
Default Value: 65534
Description: value of the uid used for the anonymous user/nfsnobody when 
root-squash is enabled.

Option: server.anongid
Default Value: 65534
Description: value of the gid used for the anonymous user/nfsnobody when 
root-squash is enabled.


Thanks,
Soumya
> 
> ---
> 
> An simple hack that I found is to add custom uid/gid fields in clients'
> ssl certificates. The bricks use the client's uid/gid specified in its 
> certificate rather than using one specified by the user. This solution 
> has no effect on performance and there's no need for a central 
> authentication.
> The thing that changes is the way client certificates are generated and 
> glusterfsd needs a small patch.
> 
> I did an experimental implementation 
>
<https://github.com/eshard/glusterfs/commit/768bf63154fdc59ba67d5788743adab8679ec5ab>
> of this idea. Custom fields "1.2.3.4.5.6.7" and
"1.2.3.4.5.6.8" are used
> for uid and gid.
> I tried it with custom CA trusted by all bricks and I issued a few 
> client certificates.
> No server configuration is needed when a new client is added, when a 
> client is revoked the a CRL 
> <https://en.wikipedia.org/wiki/Certificate_revocation_list> must
updated
> and pushed to all servers.
> By the way I didn't get glusterfs servers to accept my CRLs, do some 
> people use it?
> 
> Notes:
>   * groups are not handled right now and since users may change groups 
> regularly I don't think it would be a great idea to freeze them in a 
> certificate. The bricks could possibly do an ldap lookup in order to 
> retrieve and cache the groups for an uid.
>   * Clients obviously can't modify their certificates because they are 
> signed by CA
> 
> What do you think of this implementation, is it safe?
> Do all client operationsuse auth_glusterfs_v2_authenticate or did I miss 
> other codepaths?
> 
> Thanks!
> 
> Pierre Carru
> eshard
> 
> PS: By the way I found the source code very clean and well organized, 
> nice job :)
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>

On Fri, Sep 22, 2017 at 10:37 AM, Soumya Koduri <skoduri at redhat.com>
wrote:
> Hi,
>
> On 09/21/2017 07:32 PM, Pierre C wrote:
>
>> Hi All,
>>
>> I would like to use glusterfs in an environment where storage servers
are
>> managed by an IT service - myself :) - and several users in the
>> organization can mount the distributed fs. The users are root on their
>> machines.
>> As far as I know about glusterfs, a root client user may impersonate
any
>> uid/gid since it provides its uid/gid itself when it talks to the
bricks
>> (like nfsv3).
>> The thing is, we want to enforce permissions, i.e. user X may only
access
>> files shared with him even if he's root on his machine.
>> I found a draft spec about glusterfs+kerberos <
>> https://github.com/gluster/glusterfs-specs/blob/master/unde
>> r_review/Kerberos.md> but not much more so I think it's not
possible
>> with glusterfs right now, correct?
>> (Also I feel that kerberos would be a bit heavy to manage)
>>
>
> I am not much familiar with how ssl is handled. But from what I understand
> from your statement above, you need to restrict root users. Isn't
> root-squash option enough for it?
>
> Option: server.root-squash
> Default Value: off
> Description: Map requests from uid/gid 0 to the anonymous uid/gid. Note
> that this does not apply to any other uids or gids that might be equally
> sensitive, such as user bin or group staff.
>
> Option: server.anonuid
> Default Value: 65534
> Description: value of the uid used for the anonymous user/nfsnobody when
> root-squash is enabled.
>
> Option: server.anongid
> Default Value: 65534
> Description: value of the gid used for the anonymous user/nfsnobody when
> root-squash is enabled.
>
>
Hi Soumya,

This is not sufficient because the glusterfs protocol lets the client
choose its uid/gid/groups when the client connects to a server. (done there
https://github.com/gluster/glusterfs/blob/master/rpc/rpc-lib/src/auth-glusterfs.c#L177
)
So the current implementation relies on the client being trustworthy. A
slightly modified client can pretend to be any user.

Do you see what I mean?

Cheers,
Pierre
eshard


>
> Thanks,
> Soumya
>
>
>> ---
>>
>> An simple hack that I found is to add custom uid/gid fields in
clients'
>> ssl certificates. The bricks use the client's uid/gid specified in
its
>> certificate rather than using one specified by the user. This solution
has
>> no effect on performance and there's no need for a central
authentication.
>> The thing that changes is the way client certificates are generated and
>> glusterfsd needs a small patch.
>>
>> I did an experimental implementation <https://github.com/eshard/glu
>> sterfs/commit/768bf63154fdc59ba67d5788743adab8679ec5ab> of this
idea.
>> Custom fields "1.2.3.4.5.6.7" and "1.2.3.4.5.6.8"
are used for uid and gid.
>> I tried it with custom CA trusted by all bricks and I issued a few
client
>> certificates.
>> No server configuration is needed when a new client is added, when a
>> client is revoked the a CRL <https://en.wikipedia.org/wiki
>> /Certificate_revocation_list> must updated and pushed to all
servers.
>> By the way I didn't get glusterfs servers to accept my CRLs, do
some
>> people use it?
>>
>> Notes:
>>   * groups are not handled right now and since users may change groups
>> regularly I don't think it would be a great idea to freeze them in
a
>> certificate. The bricks could possibly do an ldap lookup in order to
>> retrieve and cache the groups for an uid.
>>   * Clients obviously can't modify their certificates because they
are
>> signed by CA
>>
>> What do you think of this implementation, is it safe?
>> Do all client operationsuse auth_glusterfs_v2_authenticate or did I
miss
>> other codepaths?
>>
>> Thanks!
>>
>> Pierre Carru
>> eshard
>>
>> PS: By the way I found the source code very clean and well organized,
>> nice job :)
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170922/a8858747/attachment.html>