Hi, Currently we use different security transformation: on RPC message we use gssapi/kerberos; on bulk data we use separate hash + cipher mechanism. And to some extend the 2 parts can be configured independently. For example, you can use "krb5p" on RPC message while use some light-weight hash/cipher on bulk data for performance or whatever reasons. Now we''re thinking about using gssapi for both RPC and bulk data. But this way we won''t be able to configure them separately. What we''re not sure is: is it necessary to be able to set different security level for RPC message and bulk data? Thanks! -- Eric
Hi Eric - can you add this detail to the presentation about the security architecture? On 8/5/08 12:12 PM, "Eric Mei" <Eric.Mei at Sun.COM> wrote:> Hi, > > Currently we use different security transformation: on RPC message we > use gssapi/kerberos; on bulk data we use separate hash + cipher > mechanism. And to some extend the 2 parts can be configured > independently. For example, you can use "krb5p" on RPC message while use > some light-weight hash/cipher on bulk data for performance or whatever > reasons.We should not use KRB/GSS for bulk data encryption, see below.> > Now we''re thinking about using gssapi for both RPC and bulk data. But > this way we won''t be able to configure them separately. What we''re not > sure is: is it necessary to be able to set different security level for > RPC message and bulk data?Yes two policies are good to have. Eventually file systems will store data encrypted. It would not be a good idea to have encrypt twice - we want to ask the client to encrypt the data, send it over the wire and store on disk without decrypting.> > Thanks!
Peter, Peter Braam wrote:> Eric - can you add this detail to the presentation about the security > architecture?OK.> On 8/5/08 12:12 PM, "Eric Mei" <Eric.Mei at Sun.COM> wrote: > >> Hi, >> >> Currently we use different security transformation: on RPC message we >> use gssapi/kerberos; on bulk data we use separate hash + cipher >> mechanism. And to some extend the 2 parts can be configured >> independently. For example, you can use "krb5p" on RPC message while use >> some light-weight hash/cipher on bulk data for performance or whatever >> reasons. > > We should not use KRB/GSS for bulk data encryption, see below. > >> Now we''re thinking about using gssapi for both RPC and bulk data. But >> this way we won''t be able to configure them separately. What we''re not >> sure is: is it necessary to be able to set different security level for >> RPC message and bulk data? > > Yes two policies are good to have. Eventually file systems will store data > encrypted. It would not be a good idea to have encrypt twice - we want to > ask the client to encrypt the data, send it over the wire and store on disk > without decrypting.I see. Suppose even if we use gss/krb5 for bulk i/o, we can still turn it off independently. That is to say: suppose we''re using gssapi for RPC message, for bulk data we can choose either use same gssapi to do protection, or don''t do protection at all. Is this approach sounds good compare to current using completely different policy? I''m asking this because Darren and Jim pointed out the current bulk i/o policy is ad-hoc. There''s 2 choices: 1) simply using standard mechanism (e.g. hmac+cipher, CCM/GCM, etc.), still keep it independent of RPC message policy; 2) suggested by Eric Barton, using the same gssapi for bulk data as well. Thanks -- Eric
...snip>> We should not use KRB/GSS for bulk data encryption, see below. >> >>> Now we''re thinking about using gssapi for both RPC and bulk data. But >>> this way we won''t be able to configure them separately. What we''re not >>> sure is: is it necessary to be able to set different security level for >>> RPC message and bulk data? >> >> Yes two policies are good to have. Eventually file systems will store data >> encrypted. It would not be a good idea to have encrypt twice - we want to >> ask the client to encrypt the data, send it over the wire and store on disk >> without decrypting. > > I see. > > Suppose even if we use gss/krb5 for bulk i/o, we can still turn it off > independently. That is to say: suppose we''re using gssapi for RPC > message, for bulk data we can choose either use same gssapi to do > protection, or don''t do protection at all. Is this approach sounds good > compare to current using completely different policy?It doesn''t sound so good. What you want is authorization (which is included in the capabilities), integrity (which should be the integrity of the encrypted data stored on disk for the data, to avoid re-computing it) and encryption (which should be the same encryption as what is used on the disk to avoid re-encrypting). GSS doesn''t have the right keys and signatures.> > I''m asking this because Darren and Jim pointed out the current bulk i/o > policy is ad-hoc. There''s 2 choices: 1) simply using standard mechanism > (e.g. hmac+cipher, CCM/GCM, etc.), still keep it independent of RPC > message policy; 2) suggested by Eric Barton, using the same gssapi for > bulk data as well.GSS is a protocol to set up communications with a certain level of security based on authentication. It is explicitly not an authorization protocol. I think this means that you can use it if you can authenticate the client to the OSS server. That however is hairy, as the identity you could use is the identity of the "page cache" in the client, not really of any particular user. Combined with the above mismatch of the crypto and integrity, I don''t believe this is right. Of course the CMU research about NASD concluded the same 10-15 years ago - you need a different protocol here, calling it ad-hoc is not so positive, calling it the NASD protocol sounds rather nice. Peter> > Thanks
On Wed, 2008-08-06 at 16:48 -0600, Peter Braam wrote:> Of course the CMU research about NASD concluded the same 10-15 years > ago - > you need a different protocol here, calling it ad-hoc is not so > positive, > calling it the NASD protocol sounds rather nice.I am not following the thread here. The existing implementation leverages off of kerberos in the client machine and does not leverage the NASD style ticket granting with versioning that the CMU papers advocated. Going to the NASD protocol is OK, but that means either abandoning Kerberos or adding Kerberos ticket honoring to the NASD versioned ticketing. This is doable, but not trivial. Can you send pointers to the relevant NASD security papers so that we can have a firm set of terminology to discuss this with? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20080808/484084ff/attachment.html
On 8/8/08 8:25 AM, "James Hughes" <James.Hughes at Sun.COM> wrote:> > > On Wed, 2008-08-06 at 16:48 -0600, Peter Braam wrote: >> Of course the CMU research about NASD concluded the same 10-15 years ago - >> you need a different protocol here, calling it ad-hoc is not so positive, >> calling it the NASD protocol sounds rather nice. > > I am not following the thread here. > > The existing implementation leverages off of kerberos in the client machine > and does not leverage the NASD style ticket granting with versioning that the > CMU papers advocated. Going to the NASD protocol is OK, but that means either > abandoning Kerberos or adding Kerberos ticket honoring to the NASD versioned > ticketing. This is doable, but not trivial. >Not true. There are capabilities generated by the MDS, with something similar to versioning. GSS between clients and OSS nodes is merely used to send the capabilities encrypted.> > > Can you send pointers to the relevant NASD security papers so that we can have > a firm set of terminology to discuss this with?http://www.pdl.cmu.edu/ - click on NASD. Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20080808/1236fad9/attachment-0001.html
1. Securing bulk data.
It seems to me that it is appropriate to use the GSSAPI to secure the
transfer of bulk data between client and server since it''s effectively
just
another message. I can see (at least naively) that it would be good to
avoid double encryption in the case where file contents are actually stored
encrypted on disk. But even in this case, don''t we still have to sign
the
(encrypted) bulk so that the receiver can be sure it arrived intact?
2. Securing Capabilities.
If we want to be sure that a Capability given to client A cannot be
snooped and used by client B we either (a) have to make the Capability
secret (i.e. never passed in cleartext) or (b) have to make the Capability
identify which client it is valid for.
It seems to me that (b) is preferrable since it ensures that a malicious
client cannot leak Capabilities to a 3rd party. The downside is that this
multiplies the number of unique Capabilities by the number of clients,
potentially increasing CPU load when 1000s of clients all open the same
shared file and each require unique Capabilities to access the stripe objects.
Do we have a feel for how bad this could be?
Cheers,
Eric
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-devel/attachments/20080808/7c229165/attachment.html
On 8/8/08 11:03 AM, "Eric Barton" <eeb at sun.com> wrote:> 1. Securing bulk data. > > It seems to me that it is appropriate to use the GSSAPI to secure the > transfer of bulk data between client and server since it''s effectively just > another message. I can see (at least naively) that it would be good to > avoid double encryption in the case where file contents are actually stored > encrypted on disk.You are not telling me that we are going through a lot of re-design, that we are encrypting data and that then we are not storing it encrypted on disk? Come on, adding an EA with a key to decrypt is not so hard and one gets lots of value from it.> > But even in this case, don''t we still have to sign the > (encrypted) bulk so that the receiver can be sure it arrived intact? >Well, yes, but as I indicated you can sign the hash that is stored on (ZFS) disk for this. That avoids generating the hash twice. So I am really not convinced yet.>The issue is not the message mechanism, but is what identity to use for GSS to authenticate and how to manage and revoke that etc.> > 2. Securing Capabilities. > > If we want to be sure that a Capability given to client A cannot be > snooped and used by client B we either (a) have to make the Capability > secret (i.e. never passed in cleartext) or (b) have to make the Capability > identify which client it is valid for. > > It seems to me that (b) is preferrable since it ensures that a malicious > client cannot leak Capabilities to a 3rd party. The downside is that this > multiplies the number of unique Capabilities by the number of clients, > potentially increasing CPU load when 1000s of clients all open the same > shared file and each require unique Capabilities to access the stripe objects. > Do we have a feel for how bad this could be? >Yes, very bad, and it is absolutely necessary to have an option that avoids this (also 1000s is out of date ? it could be 100x worse). That option could be to simply not have security on the compute cluster if customers agree with this. We also need to discuss your proposals with a review committee from LLNL and Sandia, as we did during the PF discussions. Peter> > > Cheers, > Eric > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20080808/45c1a7ad/attachment.html
Peter Braam wrote:> On 8/8/08 11:03 AM, "Eric Barton" <eeb at sun.com> wrote: > > 1. Securing bulk data. > > It seems to me that it _is_ appropriate to use the GSSAPI to secure the > transfer of bulk data between client and server since it''s > effectively just > another message. I can see (at least naively) that it would be good to > avoid double encryption in the case where file contents are actually > stored > encrypted on disk. > > > You are not telling me that we are going through a lot of re-design, > that we are encrypting data and that then we are not storing it > encrypted on disk? Come on, adding an EA with a key to decrypt is not > so hard and one gets lots of value from it. > > > But even in this case, don''t we still have to sign the > (encrypted) bulk so that the receiver can be sure it arrived intact? > > Well, yes, but as I indicated you can sign the hash that is stored on > (ZFS) disk for this. That avoids generating the hash twice. So I am > really not convinced yet.Peter, are you saying that except using NASD-style protocol, we don''t need to encrypt/sign bulk data at all?> The issue is not the message mechanism, but is what identity to use for > GSS to authenticate and how to manage and revoke that etc.Here we only want to protect on-wire data, the gss authentication is only for the "node", not particular user, as you pointed out previously.> 2. Securing Capabilities. > > If we want to be sure that a Capability given to client A cannot be > snooped and used by client B we either (a) have to make the Capability > secret (i.e. never passed in cleartext) or (b) have to make the > Capability > identify which client it is valid for. > > It seems to me that (b) is preferrable since it ensures that a malicious > client cannot leak Capabilities to a 3rd party. The downside is > that this > multiplies the number of unique Capabilities by the number of clients, > potentially increasing CPU load when 1000s of clients all open the same > shared file and each require unique Capabilities to access the > stripe objects. > Do we have a feel for how bad this could be? > > Yes, very bad, and it is absolutely necessary to have an option that > avoids this (also 1000s is out of date ? it could be 100x worse). That > option could be to simply not have security on the compute cluster if > customers agree with this. > > We also need to discuss your proposals with a review committee from LLNL > and Sandia, as we did during the PF discussions.We''re trying to figure out a way to generate only one capability for each MD object, but somehow mingled with per-export data to generate client-unique capability, but till now we haven''t found a good solution. The other thought is using some kind of light-weight, but still reasonably secure hash algorithm. By changing the KEY frequently enough (e.g. every 2 hours) we can still be secure. But we''v no idea what hash algorithm could fit our needs. -- Eric
Hi - On 8/8/08 11:44 AM, "Eric Mei" <Eric.Mei at Sun.COM> wrote:> Peter Braam wrote: >> On 8/8/08 11:03 AM, "Eric Barton" <eeb at sun.com> wrote: >> >> 1. Securing bulk data. >> >> It seems to me that it _is_ appropriate to use the GSSAPI to secure the >> transfer of bulk data between client and server since it''s >> effectively just >> another message. I can see (at least naively) that it would be good to >> avoid double encryption in the case where file contents are actually >> stored >> encrypted on disk. >> >> >> You are not telling me that we are going through a lot of re-design, >> that we are encrypting data and that then we are not storing it >> encrypted on disk? Come on, adding an EA with a key to decrypt is not >> so hard and one gets lots of value from it. >> >> >> But even in this case, don''t we still have to sign the >> (encrypted) bulk so that the receiver can be sure it arrived intact? >> >> Well, yes, but as I indicated you can sign the hash that is stored on >> (ZFS) disk for this. That avoids generating the hash twice. So I am >> really not convinced yet. > > Peter, are you saying that except using NASD-style protocol, we don''t > need to encrypt/sign bulk data at all?You do need to sign it and encrypt it - for multiple purposes, to secure the wire transaction and for storage on the server.> >> The issue is not the message mechanism, but is what identity to use for >> GSS to authenticate and how to manage and revoke that etc. > > Here we only want to protect on-wire data, the gss authentication is > only for the "node", not particular user, as you pointed out previously.Yes, and how is this managed? This is not so trivial.> >> 2. Securing Capabilities. >> >> If we want to be sure that a Capability given to client A cannot be >> snooped and used by client B we either (a) have to make the Capability >> secret (i.e. never passed in cleartext) or (b) have to make the >> Capability >> identify which client it is valid for. >> >> It seems to me that (b) is preferrable since it ensures that a malicious >> client cannot leak Capabilities to a 3rd party. The downside is >> that this >> multiplies the number of unique Capabilities by the number of clients, >> potentially increasing CPU load when 1000s of clients all open the same >> shared file and each require unique Capabilities to access the >> stripe objects. >> Do we have a feel for how bad this could be? >> >> Yes, very bad, and it is absolutely necessary to have an option that >> avoids this (also 1000s is out of date ? it could be 100x worse). That >> option could be to simply not have security on the compute cluster if >> customers agree with this. >> >> We also need to discuss your proposals with a review committee from LLNL >> and Sandia, as we did during the PF discussions. > > We''re trying to figure out a way to generate only one capability for > each MD object, but somehow mingled with per-export data to generate > client-unique capability, but till now we haven''t found a good solution. > > The other thought is using some kind of light-weight, but still > reasonably secure hash algorithm. By changing the KEY frequently enough > (e.g. every 2 hours) we can still be secure. But we''v no idea what hash > algorithm could fit our needs.
On Aug 8, 2008, at 9:12 PM, Peter Braam wrote:>> 2. Securing Capabilities. >> >> If we want to be sure that a Capability given to client A cannot be >> snooped and used by client B we either (a) have to make the >> Capability >> secret (i.e. never passed in cleartext) or (b) have to make the >> Capability >> identify which client it is valid for. >> >> It seems to me that (b) is preferrable since it ensures that a >> malicious >> client cannot leak Capabilities to a 3rd party. The downside is >> that this >> multiplies the number of unique Capabilities by the number of >> clients, >> potentially increasing CPU load when 1000s of clients all open the >> same >> shared file and each require unique Capabilities to access the >> stripe objects. >> Do we have a feel for how bad this could be? >> > Yes, very bad, and it is absolutely necessary to have an option that > avoids this (also 1000s is out of date ? it could be 100x worse). > That option could be to simply not have security on the compute > cluster if customers agree with this.this kind of problem also exists for OST capabilities even without making them unique per client. As objects do not know the mds fid initially, we can check the operation is intended for this particular object by objid/gr only. The proposed solution was to generate on mds an objid-based capa for each object instead of 1 common fid-based capa (i.e. a pair objid/group is signed instead of mds fid), and therefore we give out default-stripe-count OST capas instead of 1 every time. it could be optimized later, by switching to fid-based capas with time when mds gets known all the objects know the mds fid. however, the mechanism is not clear yet. -- Vitaly
Peter Braam wrote:> Hi - > > > On 8/8/08 11:44 AM, "Eric Mei" <Eric.Mei at Sun.COM> wrote: > >> Peter Braam wrote: >>> On 8/8/08 11:03 AM, "Eric Barton" <eeb at sun.com> wrote: >>> >>> 1. Securing bulk data. >>> >>> It seems to me that it _is_ appropriate to use the GSSAPI to secure the >>> transfer of bulk data between client and server since it''s >>> effectively just >>> another message. I can see (at least naively) that it would be good to >>> avoid double encryption in the case where file contents are actually >>> stored >>> encrypted on disk. >>> >>> >>> You are not telling me that we are going through a lot of re-design, >>> that we are encrypting data and that then we are not storing it >>> encrypted on disk? Come on, adding an EA with a key to decrypt is not >>> so hard and one gets lots of value from it. >>> >>> >>> But even in this case, don''t we still have to sign the >>> (encrypted) bulk so that the receiver can be sure it arrived intact? >>> >>> Well, yes, but as I indicated you can sign the hash that is stored on >>> (ZFS) disk for this. That avoids generating the hash twice. So I am >>> really not convinced yet. >> Peter, are you saying that except using NASD-style protocol, we don''t >> need to encrypt/sign bulk data at all? > > You do need to sign it and encrypt it - for multiple purposes, to secure the > wire transaction and for storage on the server.Sorry I''m still a little confused. To be exactly clear, do you mean: In the future we''ll use NASD-style protocol to secure the bulk data''s wire transfer & storage on server; and for now we can simply leave the bulk data unprotected? -- Eric
>> >> You do need to sign it and encrypt it - for multiple purposes, to secure the >> wire transaction and for storage on the server. > > Sorry I''m still a little confused. To be exactly clear, do you mean: In > the future we''ll use NASD-style protocol to secure the bulk data''s wire > transfer & storage on server; and for now we can simply leave the bulk > data unprotected?No you need to be able to encrypt it. I''m just stating that if you the current solution (which gives privacy using GSS between client and OSS iirc) is going to be changed, then please change it in the correct way to accommodate re-using checksums and re-using the encrypted data for storage. Doing either of these things twice is pointless. Probably there should be options not to do so as part of the configuration. Peter
Peter Braam wrote:>>> You do need to sign it and encrypt it - for multiple purposes, to secure the >>> wire transaction and for storage on the server. >> Sorry I''m still a little confused. To be exactly clear, do you mean: In >> the future we''ll use NASD-style protocol to secure the bulk data''s wire >> transfer & storage on server; and for now we can simply leave the bulk >> data unprotected? > > No you need to be able to encrypt it. > > I''m just stating that if you the current solution (which gives privacy using > GSS between client and OSS iirc) is going to be changed, then please change > it in the correct way to accommodate re-using checksums and re-using the > encrypted data for storage. Doing either of these things twice is > pointless. > > Probably there should be options not to do so as part of the configuration.Peter, Thanks for clarifying this! Yes that''s what we''re thinking. -- Eric
On Mon, 2008-08-11 at 13:33 -0600, Peter Braam wrote:> >> > >> You do need to sign it and encrypt it - for multiple purposes, to > secure the > >> wire transaction and for storage on the server. > > > > Sorry I''m still a little confused. To be exactly clear, do you mean: > In > > the future we''ll use NASD-style protocol to secure the bulk data''s > wire > > transfer & storage on server; and for now we can simply leave the > bulk > > data unprotected? > > No you need to be able to encrypt it. > > I''m just stating that if you the current solution (which gives privacy > using > GSS between client and OSS iirc) is going to be changed, then please > change > it in the correct way to accommodate re-using checksums and re-using > the > encrypted data for storage. Doing either of these things twice is > pointless.The NASD protocols do not allow for the encryption of the communications and the stored information to be the same. The key management for communications is well understood, but the key management for the stored information as defined by NASD requires the encryption of the stored information to be performed at the target.> Probably there should be options not to do so as part of the > configuration. > > Peter > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20080813/88e59ca4/attachment.html
On 8/13/08 12:40 PM, "James Hughes" <James.Hughes at Sun.COM> wrote:> > > On Mon, 2008-08-11 at 13:33 -0600, Peter Braam wrote: >>>> >> >>>> >> You do need to sign it and encrypt it - for multiple purposes, to >>>> secure the >>>> >> wire transaction and for storage on the server. >>> > >>> > Sorry I''m still a little confused. To be exactly clear, do you mean: In >>> > the future we''ll use NASD-style protocol to secure the bulk data''s wire >>> > transfer & storage on server; and for now we can simply leave the bulk >>> > data unprotected? >> >> No you need to be able to encrypt it. >> >> I''m just stating that if you the current solution (which gives privacy using >> GSS between client and OSS iirc) is going to be changed, then please change >> it in the correct way to accommodate re-using checksums and re-using the >> encrypted data for storage. Doing either of these things twice is >> pointless. > The NASD protocols do not allow for the encryption of the communications and > the stored information to be the same. The key management for communications > is well understood, but the key management for the stored information as > defined by NASD requires the encryption of the stored information to be > performed at the target. >Why? I don?t see a reason not to pursue this, even though it would not be what NASD does. Using EAs there are easy key management systems (and more complex ones), and we discussed an basic key server running inside the MDS?s with the tri-labs who reacted quite positively. Peter> > >> Probably there should be options not to do so as part of the configuration. >> >> Peter >> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20080813/6e9fb6f4/attachment.html