Hello everyone, My understanding is that the ZFS crypto framework will not release until 2010. In light of that, I''m wondering if the following approach to encryption could make sense for some subset of users: The idea is to use the compression framework to do both compression and encryption in one pass. This would be done by defining a new compression type, which might be called "compress-encrypt" or something like that. There could be two levels, one that does both compress and encrypt and another that does encrypt only. I see the following issues with this approach: 1. ZFS compression framework presently takes compressed data only if there was at least 12.5% reduction. For data that didn''t compress, you would wind up storing it unencrypted, even if encryption was on. 2. Meta-data would not be encrypted. I.e., even if you don''t have the key, you will be able to do directory listings and see file names, etc. 3. There is no key management framework. I would deal with these as follows: Issue #1 can be solved by changing ZFS code such that it always accepts the "compressed" data. I guess this is an easy change. Issue #2 may be a limitation to some and feature to others. May be OK. Issue #3 can be solved using encryption hardware (which my company happens to make). The keys are stored in hardware and can be used directly from that. Of course, this means that the solution will be specific to our hardware, but that''s fine by me. The idea is that we would do this project on our own and supply this modified ZFS with our compression/encryption hardware to our customers. We may submit the patch for inclusion in some future version of OS, if the developers are amenable to that. Does anyone see any problems with this? There are probably various gotchas here that I haven''t thought of. If you can think of any, please let me know. Thanks, Monish ---- Monish Shah CEO, Indra Networks, Inc. www.indranetworks.com
Monish Shah wrote:> Hello everyone, > > My understanding is that the ZFS crypto framework will not release until > 2010.That is incorrect information, where did you get that from ? > In light of that, I''m wondering if the following approach to> encryption could make sense for some subset of users: > > The idea is to use the compression framework to do both compression and > encryption in one pass. This would be done by defining a new > compression type, which might be called "compress-encrypt" or something > like that. There could be two levels, one that does both compress and > encrypt and another that does encrypt only. > > I see the following issues with this approach: > > 1. ZFS compression framework presently takes compressed data only if > there was at least 12.5% reduction. For data that didn''t compress, you > would wind up storing it unencrypted, even if encryption was on. > > 2. Meta-data would not be encrypted. I.e., even if you don''t have the > key, you will be able to do directory listings and see file names, etc. > > 3. There is no key management framework.That is impossible there has to be key management somewhere.> I would deal with these as follows: > > Issue #1 can be solved by changing ZFS code such that it always accepts > the "compressed" data. I guess this is an easy change. > > Issue #2 may be a limitation to some and feature to others. May be OK. > > Issue #3 can be solved using encryption hardware (which my company > happens to make). The keys are stored in hardware and can be used > directly from that. Of course, this means that the solution will be > specific to our hardware, but that''s fine by me. > > The idea is that we would do this project on our own and supply this > modified ZFS with our compression/encryption hardware to our customers. > We may submit the patch for inclusion in some future version of OS, if > the developers are amenable to that.If it is specific to your companies hardware I doubt it would ever get integrated into OpenSolaris particularly given the existing zfs-crypto project has no hardware dependencies at all. The better way to use your encryption hardware is to get it plugged into the OpenSolaris cryptographic framework (see the crypto project on OpenSolaris.org)> Does anyone see any problems with this? There are probably various > gotchas here that I haven''t thought of. If you can think of any, please > let me know.The various gotchas are the things that have been taking me and the rest of the ZFS team a large part of the zfs-crypto project to resolve. It really isn''t as simple as you think it is - if it were then the zfs-crypto project would be done by now! If you really want to help get encryption for ZFS then please come and join the already existing project rather than starting another one from scratch. -- Darren J Moffat
Hello Darren,> Monish Shah wrote: >> Hello everyone, >> >> My understanding is that the ZFS crypto framework will not release until >> 2010. > > That is incorrect information, where did you get that from ?It was in Mike Shapiro''s presentation at the Open Solaris Storage Summit that took place a couple of weeks ago. Perhaps I mis-read the slide, but I''m pretty sure it listed encryption as a feature for 2010. ...>> 3. There is no key management framework. > > That is impossible there has to be key management somewhere.What I meant was, the compression framework does not have key management framework. Using our hardware (which I mentioned later in my mail), the key management would come with the hardware, since we store keys in the hardware. We provide a utility to manage the keys stored in the hardware. ...> If it is specific to your companies hardware I doubt it would ever get > integrated into OpenSolaris particularly given the existing zfs-crypto > project has no hardware dependencies at all. > > The better way to use your encryption hardware is to get it plugged into > the OpenSolaris cryptographic framework (see the crypto project on > OpenSolaris.org)That was precisely what I want thinking originally. However, if it is out in 2010, there is temptation to do our own project, which I thought could be done in a couple of months. (In light of your comment below, my estimate may have been wildly optimistic, but the foregoing is merely an explanation of what I was thinking.)>> Does anyone see any problems with this? There are probably various >> gotchas here that I haven''t thought of. If you can think of any, please >> let me know. > > The various gotchas are the things that have been taking me and the rest > of the ZFS team a large part of the zfs-crypto project to resolve. It > really isn''t as simple as you think it is - if it were then the zfs-crypto > project would be done by now! > > If you really want to help get encryption for ZFS then please come and > join the already existing project rather than starting another one from > scratch.If the schedule is much sooner than 2010, I would definitely do so. What is your present schedule estimate?> -- > Darren J Moffat >Monish
Monish Shah wrote:> Hello Darren, > > >> Monish Shah wrote: >>> Hello everyone, >>> >>> My understanding is that the ZFS crypto framework will not release >>> until 2010. >> >> That is incorrect information, where did you get that from ? > > It was in Mike Shapiro''s presentation at the Open Solaris Storage Summit > that took place a couple of weeks ago. Perhaps I mis-read the slide, > but I''m pretty sure it listed encryption as a feature for 2010.That is for its availablity in the S7000 appliance. It will be in OpenSolaris before that (it has to be because the S7000 is based on an OpenSolaris build).> If the schedule is much sooner than 2010, I would definitely do so. > What is your present schedule estimate?I can''t commit to this yet but I expect somewhere around August 2009. Note that the code in hg.opensolaris.org/hg/zfs-crypto/gate actually works today and encrypts more than what your proposal work. It is just that we are making some design changes to simplify the model and ensure that encryption integrates with other ZFS features coming along. There will be a design update posted to zfs-crypto-discuss@ later this month. -- Darren J Moffat