Hi All, I would like to use glusterfs in an environment where storage servers are managed by an IT service - myself :) - and several users in the organization can mount the distributed fs. The users are root on their machines. As far as I know about glusterfs, a root client user may impersonate any uid/gid since it provides its uid/gid itself when it talks to the bricks (like nfsv3). The thing is, we want to enforce permissions, i.e. user X may only access files shared with him even if he's root on his machine. I found a draft spec about glusterfs+kerberos <https://github.com/gluster/glusterfs-specs/blob/master/under_review/Kerberos.md> but not much more so I think it's not possible with glusterfs right now, correct? (Also I feel that kerberos would be a bit heavy to manage) --- An simple hack that I found is to add custom uid/gid fields in clients' ssl certificates. The bricks use the client's uid/gid specified in its certificate rather than using one specified by the user. This solution has no effect on performance and there's no need for a central authentication. The thing that changes is the way client certificates are generated and glusterfsd needs a small patch. I did an experimental implementation <https://github.com/eshard/glusterfs/commit/768bf63154fdc59ba67d5788743adab8679ec5ab> of this idea. Custom fields "1.2.3.4.5.6.7" and "1.2.3.4.5.6.8" are used for uid and gid. I tried it with custom CA trusted by all bricks and I issued a few client certificates. No server configuration is needed when a new client is added, when a client is revoked the a CRL <https://en.wikipedia.org/wiki/Certificate_revocation_list> must updated and pushed to all servers. By the way I didn't get glusterfs servers to accept my CRLs, do some people use it? Notes: * groups are not handled right now and since users may change groups regularly I don't think it would be a great idea to freeze them in a certificate. The bricks could possibly do an ldap lookup in order to retrieve and cache the groups for an uid. * Clients obviously can't modify their certificates because they are signed by CA What do you think of this implementation, is it safe? Do all client operations use auth_glusterfs_v2_authenticate or did I miss other codepaths? Thanks! Pierre Carru eshard PS: By the way I found the source code very clean and well organized, nice job :) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170921/4fdc349c/attachment.html>
Soumya Koduri
2017-Sep-22 08:37 UTC
[Gluster-users] Restrict root clients / experimental patch
Hi, On 09/21/2017 07:32 PM, Pierre C wrote:> Hi All, > > I would like to use glusterfs in an environment where storage servers > are managed by an IT service - myself :) - and several users in the > organization can mount the distributed fs. The users are root on their > machines. > As far as I know about glusterfs, a root client user may impersonate any > uid/gid since it provides its uid/gid itself when it talks to the bricks > (like nfsv3). > The thing is, we want to enforce permissions, i.e. user X may only > access files shared with him even if he's root on his machine. > I found a draft spec about glusterfs+kerberos > <https://github.com/gluster/glusterfs-specs/blob/master/under_review/Kerberos.md> > but not much more so I think it's not possible with glusterfs right now, > correct? > (Also I feel that kerberos would be a bit heavy to manage)I am not much familiar with how ssl is handled. But from what I understand from your statement above, you need to restrict root users. Isn't root-squash option enough for it? Option: server.root-squash Default Value: off Description: Map requests from uid/gid 0 to the anonymous uid/gid. Note that this does not apply to any other uids or gids that might be equally sensitive, such as user bin or group staff. Option: server.anonuid Default Value: 65534 Description: value of the uid used for the anonymous user/nfsnobody when root-squash is enabled. Option: server.anongid Default Value: 65534 Description: value of the gid used for the anonymous user/nfsnobody when root-squash is enabled. Thanks, Soumya> > --- > > An simple hack that I found is to add custom uid/gid fields in clients' > ssl certificates. The bricks use the client's uid/gid specified in its > certificate rather than using one specified by the user. This solution > has no effect on performance and there's no need for a central > authentication. > The thing that changes is the way client certificates are generated and > glusterfsd needs a small patch. > > I did an experimental implementation > <https://github.com/eshard/glusterfs/commit/768bf63154fdc59ba67d5788743adab8679ec5ab> > of this idea. Custom fields "1.2.3.4.5.6.7" and "1.2.3.4.5.6.8" are used > for uid and gid. > I tried it with custom CA trusted by all bricks and I issued a few > client certificates. > No server configuration is needed when a new client is added, when a > client is revoked the a CRL > <https://en.wikipedia.org/wiki/Certificate_revocation_list> must updated > and pushed to all servers. > By the way I didn't get glusterfs servers to accept my CRLs, do some > people use it? > > Notes: > * groups are not handled right now and since users may change groups > regularly I don't think it would be a great idea to freeze them in a > certificate. The bricks could possibly do an ldap lookup in order to > retrieve and cache the groups for an uid. > * Clients obviously can't modify their certificates because they are > signed by CA > > What do you think of this implementation, is it safe? > Do all client operationsuse auth_glusterfs_v2_authenticate or did I miss > other codepaths? > > Thanks! > > Pierre Carru > eshard > > PS: By the way I found the source code very clean and well organized, > nice job :) > > > _______________________________________________ > Gluster-users mailing list > Gluster-users at gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users >
On Fri, Sep 22, 2017 at 10:37 AM, Soumya Koduri <skoduri at redhat.com> wrote:> Hi, > > On 09/21/2017 07:32 PM, Pierre C wrote: > >> Hi All, >> >> I would like to use glusterfs in an environment where storage servers are >> managed by an IT service - myself :) - and several users in the >> organization can mount the distributed fs. The users are root on their >> machines. >> As far as I know about glusterfs, a root client user may impersonate any >> uid/gid since it provides its uid/gid itself when it talks to the bricks >> (like nfsv3). >> The thing is, we want to enforce permissions, i.e. user X may only access >> files shared with him even if he's root on his machine. >> I found a draft spec about glusterfs+kerberos < >> https://github.com/gluster/glusterfs-specs/blob/master/unde >> r_review/Kerberos.md> but not much more so I think it's not possible >> with glusterfs right now, correct? >> (Also I feel that kerberos would be a bit heavy to manage) >> > > I am not much familiar with how ssl is handled. But from what I understand > from your statement above, you need to restrict root users. Isn't > root-squash option enough for it? > > Option: server.root-squash > Default Value: off > Description: Map requests from uid/gid 0 to the anonymous uid/gid. Note > that this does not apply to any other uids or gids that might be equally > sensitive, such as user bin or group staff. > > Option: server.anonuid > Default Value: 65534 > Description: value of the uid used for the anonymous user/nfsnobody when > root-squash is enabled. > > Option: server.anongid > Default Value: 65534 > Description: value of the gid used for the anonymous user/nfsnobody when > root-squash is enabled. > >Hi Soumya, This is not sufficient because the glusterfs protocol lets the client choose its uid/gid/groups when the client connects to a server. (done there https://github.com/gluster/glusterfs/blob/master/rpc/rpc-lib/src/auth-glusterfs.c#L177 ) So the current implementation relies on the client being trustworthy. A slightly modified client can pretend to be any user. Do you see what I mean? Cheers, Pierre eshard> > Thanks, > Soumya > > >> --- >> >> An simple hack that I found is to add custom uid/gid fields in clients' >> ssl certificates. The bricks use the client's uid/gid specified in its >> certificate rather than using one specified by the user. This solution has >> no effect on performance and there's no need for a central authentication. >> The thing that changes is the way client certificates are generated and >> glusterfsd needs a small patch. >> >> I did an experimental implementation <https://github.com/eshard/glu >> sterfs/commit/768bf63154fdc59ba67d5788743adab8679ec5ab> of this idea. >> Custom fields "1.2.3.4.5.6.7" and "1.2.3.4.5.6.8" are used for uid and gid. >> I tried it with custom CA trusted by all bricks and I issued a few client >> certificates. >> No server configuration is needed when a new client is added, when a >> client is revoked the a CRL <https://en.wikipedia.org/wiki >> /Certificate_revocation_list> must updated and pushed to all servers. >> By the way I didn't get glusterfs servers to accept my CRLs, do some >> people use it? >> >> Notes: >> * groups are not handled right now and since users may change groups >> regularly I don't think it would be a great idea to freeze them in a >> certificate. The bricks could possibly do an ldap lookup in order to >> retrieve and cache the groups for an uid. >> * Clients obviously can't modify their certificates because they are >> signed by CA >> >> What do you think of this implementation, is it safe? >> Do all client operationsuse auth_glusterfs_v2_authenticate or did I miss >> other codepaths? >> >> Thanks! >> >> Pierre Carru >> eshard >> >> PS: By the way I found the source code very clean and well organized, >> nice job :) >> >> >> _______________________________________________ >> Gluster-users mailing list >> Gluster-users at gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-users >> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20170922/a8858747/attachment.html>