CentOS - Nov 2018 - Tools/mechanisms for the management of access permissions in big filebased datasets

Hello,

we are currently managing access permissions through classical 
user-group-others permissions on a multi-petabyte directory tree with 
partially very deep and broad directories.  Projects are represented by 
directory trees and mapped through GIDs.  Lately we had lots of 
"singular" permission request (one single user needs access to a
single
dataset but should not be able to see all other datasets belonging to 
the same project).  We realized, that the UGO model doesn't scale and is 
becoming more and more unmanageable.

Can you recommend tools/mechanisms/technologies to overcome the 
drawbacks of the UGO model?  We are thinking about some purely ACL based 
mechanism (but are open to other ideas).  All filesystems in question 
are mounted via NFSv4 and the clients are (almost) completely CentOS 7.x 
hsots.  Ideally the tool would have some web UI and some kind of 
(REST)API which allows us to modify permissions from our inhouse data 
management application (which does /not/ manage permissions, just the 
structure of the data).  Additionally it should be able to 
visualize/report permissions in directory.

I wasn't very successful in googling possible candidates, hence the 
question to the list.

Cheers
frank

Well, there are extended ACLs if they're available in CentOS, when I first
worked with them (long ago) they were new (and on a different Distro).  I hope
support for them has improved.  They allow multiple users/groups to be assigned
permissions to a file/directory.  The problem then was that chmod (and other
programs) were not extended-ACL-aware and could over-ride extended ACLs.  There
was a mechanism to recover from the situation but what it basically came down to
was eternal vigilance - the system administrators had to understand (and agree
about) extended ACLs and be careful/diligent in applying them.  There are hacks
which could possibly help (rename chmod and replace it with a script warning
about extended ACLs) but, in the final analysis, it's not a decision to be
undertaken lightly (unless the situation has changed dramatically).


Leroy Tennison
Network Information/Cyber Security Specialist
E: leroy at datavoiceint.com
2220 Bush Dr
McKinney, Texas
75070
www.datavoiceint.com
TThis message has been sent on behalf
of a company that is part of the Harris Operating Group of
Constellation Software Inc. These companies are listed
here
.
If you prefer not to be contacted by Harris
Operating Group
please notify us
.
This message is intended exclusively for the
individual or entity to which it is addressed. This communication
may contain information that is proprietary, privileged or
confidential or otherwise legally exempt from disclosure. If you are
not the named addressee, you are not authorized to read, print,
retain, copy or disseminate this message or any part of it. If you
have received this message in error, please notify the sender
immediately by e-mail and delete all copies of the
message.

________________________________________
From: CentOS <centos-bounces at centos.org> on behalf of Frank Thommen
<list.centos at drosera.ch>
Sent: Tuesday, November 27, 2018 7:25 AM
To: CentOS mailing list
Subject: [EXTERNAL] [CentOS] Tools/mechanisms for the management of access
permissions in big filebased datasets

Hello,

we are currently managing access permissions through classical
user-group-others permissions on a multi-petabyte directory tree with
partially very deep and broad directories.  Projects are represented by
directory trees and mapped through GIDs.  Lately we had lots of
"singular" permission request (one single user needs access to a
single
dataset but should not be able to see all other datasets belonging to
the same project).  We realized, that the UGO model doesn't scale and is
becoming more and more unmanageable.

Can you recommend tools/mechanisms/technologies to overcome the
drawbacks of the UGO model?  We are thinking about some purely ACL based
mechanism (but are open to other ideas).  All filesystems in question
are mounted via NFSv4 and the clients are (almost) completely CentOS 7.x
hsots.  Ideally the tool would have some web UI and some kind of
(REST)API which allows us to modify permissions from our inhouse data
management application (which does /not/ manage permissions, just the
structure of the data).  Additionally it should be able to
visualize/report permissions in directory.

I wasn't very successful in googling possible candidates, hence the
question to the list.

Cheers
frank


_______________________________________________
CentOS mailing list
CentOS at centos.org
https://lists.centos.org/mailman/listinfo/centos

Thank you.  Basically our problem are not the ACLs or their support per 
se, but that we have to manage a huge number of individual ACLS (several 
hundred users in more than hundred projects) in multi-petabyte 
filesystem and still have to keep overview and control.  Our problem is 
more the management side.  Effectively we are looking for a tool that 
helps us manage these permissions and we would accept whatever 
permissions mechanism this tool uses (UGO/ACLs).

Cheers
frank


On 11/27/2018 03:06 PM, Leroy Tennison wrote:
> Well, there are extended ACLs if they're available in CentOS, when I
first worked with them (long ago) they were new (and on a different Distro).  I
hope support for them has improved.  They allow multiple users/groups to be
assigned permissions to a file/directory.  The problem then was that chmod (and
other programs) were not extended-ACL-aware and could over-ride extended ACLs. 
There was a mechanism to recover from the situation but what it basically came
down to was eternal vigilance - the system administrators had to understand (and
agree about) extended ACLs and be careful/diligent in applying them.  There are
hacks which could possibly help (rename chmod and replace it with a script
warning about extended ACLs) but, in the final analysis, it's not a decision
to be undertaken lightly (unless the situation has changed dramatically).
> 
> 
> Leroy Tennison
> Network Information/Cyber Security Specialist
> E: leroy at datavoiceint.com
> 2220 Bush Dr
> McKinney, Texas
> 75070
> www.datavoiceint.com
> TThis message has been sent on behalf
> of a company that is part of the Harris Operating Group of
> Constellation Software Inc. These companies are listed
> here
> .
> If you prefer not to be contacted by Harris
> Operating Group
> please notify us
> .
> This message is intended exclusively for the
> individual or entity to which it is addressed. This communication
> may contain information that is proprietary, privileged or
> confidential or otherwise legally exempt from disclosure. If you are
> not the named addressee, you are not authorized to read, print,
> retain, copy or disseminate this message or any part of it. If you
> have received this message in error, please notify the sender
> immediately by e-mail and delete all copies of the
> message.
> 
> ________________________________________
> From: CentOS <centos-bounces at centos.org> on behalf of Frank
Thommen <list.centos at drosera.ch>
> Sent: Tuesday, November 27, 2018 7:25 AM
> To: CentOS mailing list
> Subject: [EXTERNAL] [CentOS] Tools/mechanisms for the management of access
permissions in big filebased datasets
> 
> Hello,
> 
> we are currently managing access permissions through classical
> user-group-others permissions on a multi-petabyte directory tree with
> partially very deep and broad directories.  Projects are represented by
> directory trees and mapped through GIDs.  Lately we had lots of
> "singular" permission request (one single user needs access to a
single
> dataset but should not be able to see all other datasets belonging to
> the same project).  We realized, that the UGO model doesn't scale and
is
> becoming more and more unmanageable.
> 
> Can you recommend tools/mechanisms/technologies to overcome the
> drawbacks of the UGO model?  We are thinking about some purely ACL based
> mechanism (but are open to other ideas).  All filesystems in question
> are mounted via NFSv4 and the clients are (almost) completely CentOS 7.x
> hsots.  Ideally the tool would have some web UI and some kind of
> (REST)API which allows us to modify permissions from our inhouse data
> management application (which does /not/ manage permissions, just the
> structure of the data).  Additionally it should be able to
> visualize/report permissions in directory.
> 
> I wasn't very successful in googling possible candidates, hence the
> question to the list.
> 
> Cheers
> frank
> 
> 
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
>