zfs discuss - Jan 2007 - ZFS inode equivalent

Which structure in ZFS stores file property info such as permissions, owner etc?
What is its relationship with uberblock, block pointer or metadnode etc? I
thought it would be dnode. However, I don''t know which structure in
dnode is used to store such info. Thx for ur help

dnode:
uint8_tdn_type;
uint8_tdn_indblkshift;
uint8_tdn_nlevels;
uint8_tdn_nblkptr;
uint8_tdn_bonustype;
uint8_tdn_checksum;
uint8_tdn_compress;
uint8_tdn_pad[1];
uint16_tdn_datablkszsec;
uint16_tdn_bonuslen;
uint8_tdn_pad2[4];
uint64_t dn_maxblkid;
uint64_tdn_secphys;
uint64_t dn_pad3[4];
blkptr_tdn_blkptr[N];
uint8_tdn_bonus[BONUSLEN]
 
 
This message posted from opensolaris.org

On Wed, Jan 31, 2007 at 01:11:52PM -0800, Brian Gao
wrote:
> Which structure in ZFS stores file property info such as permissions, owner
etc? What is its relationship with uberblock, block pointer or metadnode etc? I
thought it would be dnode. However, I don''t know which structure in
dnode is used to store such info. Thx for ur help
Look at znode_phys_t and zfs_znode_acl_t.

Or look at pages 46-50 of the ZFS on-disk format document:

http://opensolaris.org/os/community/zfs/docs/ondiskformatfinal.pdf

Nico
--

> Or look at pages 46-50 of the ZFS on-disk format document:
>
> http://opensolaris.org/os/community/zfs/docs/ondiskformatfinal.pdf
There''s an final version?  That link appears to be broken (and the
lastest version linked from the ZFS docs area
http://opensolaris.org/os/community/zfs/docs/ is dated 0822).

James

On Wed, Jan 31, 2007 at 09:31:34PM +0000, James Blackburn
wrote:
> >Or look at pages 46-50 of the ZFS on-disk format document:
> >
> >http://opensolaris.org/os/community/zfs/docs/ondiskformatfinal.pdf
> 
> There''s an final version?  That link appears to be broken (and the
> lastest version linked from the ZFS docs area
> http://opensolaris.org/os/community/zfs/docs/ is dated 0822).
Ah, yes, I cut-n-pasted the wrong link.  The former came from Tabriz''
blog post about the on-disk format.

Thank nico. I''ll read the doc.
 
 
This message posted from opensolaris.org

Final for the first draft. :-)

Use the .../community/zfs/docs link to get to this doc link at the
bottom of the page. The current version is indeed 0822.

More updates are needed, but the dnode description is still applicable. 
Someone will correct if I''m wrong.

cs

James Blackburn wrote:
>> Or look at pages 46-50 of the ZFS on-disk format document:
>>
>> http://opensolaris.org/os/community/zfs/docs/ondiskformatfinal.pdf
> 
> 
> There''s an final version?  That link appears to be broken (and the
> lastest version linked from the ZFS docs area
> http://opensolaris.org/os/community/zfs/docs/ is dated 0822).
> 
> James
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

No it''s not the final version or even the latest!
The current on disk format version is 3. However, it hasn''t
diverged much and the znode/acl stuff hasn''t changed.


Neil.

James Blackburn wrote On 01/31/07 14:31,:
>> Or look at pages 46-50 of the ZFS on-disk format document:
>>
>> http://opensolaris.org/os/community/zfs/docs/ondiskformatfinal.pdf
> 
> 
> There''s an final version?  That link appears to be broken (and the
> lastest version linked from the ZFS docs area
> http://opensolaris.org/os/community/zfs/docs/ is dated 0822).
> 
> James
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Neil Perrin wrote:
> No it''s not the final version or even the latest!
> The current on disk format version is 3. However, it hasn''t
> diverged much and the znode/acl stuff hasn''t changed.
and it will get updated as part of zfs-crypto, I just haven''t done so 
yet because I''m not finished designing yet.

-- 
Darren J Moffat

On Thu, Feb 01, 2007 at 11:00:07AM +0000, Darren J Moffat
wrote:
> Neil Perrin wrote:
> >No it''s not the final version or even the latest!
> >The current on disk format version is 3. However, it hasn''t
> >diverged much and the znode/acl stuff hasn''t changed.
> 
> and it will get updated as part of zfs-crypto, I just haven''t done
so yet because I''m not finished designing yet.
Do you consider adding a new property type (next to readonly and
inherit) - a "oneway" property? Such propery could be only set if the
dataset has no children, no snapshots and no data, and once set can''t
be
modified. "oneway" would be the type of the "encryption"
property.
On the other hand you may still want to support encryption algorithm
change and most likely key change.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070202/f85b6acf/attachment.bin>

Pawel Jakub Dawidek wrote:
> On Thu, Feb 01, 2007 at 11:00:07AM +0000, Darren J Moffat wrote:
>> Neil Perrin wrote:
>>> No it''s not the final version or even the latest!
>>> The current on disk format version is 3. However, it
hasn''t
>>> diverged much and the znode/acl stuff hasn''t changed.
>> and it will get updated as part of zfs-crypto, I just haven''t
done so yet because I''m not finished designing yet.
> 
> Do you consider adding a new property type (next to readonly and
> inherit) - a "oneway" property? Such propery could be only set if
the
> dataset has no children, no snapshots and no data, and once set
can''t be
> modified. "oneway" would be the type of the
"encryption" property.
> On the other hand you may still want to support encryption algorithm
> change and most likely key change.
I''m not sure I understand what you are asking for.

My current plan is that once set the encryption property that describes 
which algorithm (mechanism actually: algorithm, key length and mode, eg 
aes-128-ccm) can not be changed, it would be inherited by any clones. 
Creating new child file systems "rooted" in an encrypted filesystem
you
would be allowed to turn if off (I''d like to have a policy like the acl
one here) but by default it would be inherited.

Key change is a very difficult problem because in some cases it can mean 
rewritting all previous data, in other cases it just means start using 
the new key now but keep the old one.   Which is correct depends on why 
you are doing a key change.  Key change for data at rest is a very 
different problem space from rekey in a network protocol.

In theory the algorithm could be different per dnode_phys_t just like 
checksum/compression are today, however having aes-128 on one dnode and 
aes-256 on another causes a problem because you also need different keys 
for them, it gets even more complex if you consider the algorithm mode 
and if you choose completely different algorithms.  Having a different 
algorithm and key length will certainly be possible for different 
filesystems though (eg root with aes-128 and home with aes-256).

Which I think is a long way of saying "yes".

-- 
Darren J Moffat

On Fri, Feb 02, 2007 at 08:46:34AM +0000, Darren J Moffat wrote:

Hi,

[..]
> Key change is a very difficult problem because in some cases it can mean 
> rewritting all previous data, in other cases it just means start using 
probably a dumb question: What about autogenerating a key from
/dev/random and store it encrypted by a user key in a filesystem header?
The FS-data is encrypted with the autogenerated key and if the user
changes the his user key, only the specific filesystem header have to be
rewritten.

Don''t know how good this source is, but
http://clemens.endorphin.org/cryptography talks about ''two level
hierachy of cryptographic keys''.
> the new key now but keep the old one.   Which is correct depends on why 
> you are doing a key change.  Key change for data at rest is a very 
> different problem space from rekey in a network protocol.
[..]

Best regards
Georg Sauthoff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20070202/b6aefda7/attachment.bin>

On Fri, Feb 02, 2007 at 08:46:34AM +0000, Darren J Moffat
wrote:
> Pawel Jakub Dawidek wrote:
> >On Thu, Feb 01, 2007 at 11:00:07AM +0000, Darren J Moffat wrote:
> >>Neil Perrin wrote:
> >>>No it''s not the final version or even the latest!
> >>>The current on disk format version is 3. However, it
hasn''t
> >>>diverged much and the znode/acl stuff hasn''t changed.
> >>and it will get updated as part of zfs-crypto, I just
haven''t done so yet because I''m not finished designing yet.
> >Do you consider adding a new property type (next to readonly and
> >inherit) - a "oneway" property? Such propery could be only
set if the
> >dataset has no children, no snapshots and no data, and once set
can''t be
> >modified. "oneway" would be the type of the
"encryption" property.
> >On the other hand you may still want to support encryption algorithm
> >change and most likely key change.
> 
> I''m not sure I understand what you are asking for.
I''m sorry it seems I started my explanations from too deep. I started
to
play with encryption on my own by creating a "crypto" compression
algorithm.  Currently there are few types of property (readonly,
inherited, etc.), but non of them seems to be suitable for encryption.
When you enable encryption there should be no data, or you know that
existing data is going to be encrypted and plaintext data securely
removed automatically. Of course the later is much more complex to
implement.
> My current plan is that once set the encryption property that describes
which algorithm (mechanism actually: algorithm, key length and mode, eg
aes-128-ccm) can not be
> changed, it would be inherited by any clones. Creating new child file
systems "rooted" in an encrypted filesystem you would be allowed to
turn if off (I''d like to have a
> policy like the acl one here) but by default it would be inherited.
Right. I forget that a dataset created under another dataset doesn''t
share data with the parent.
> Key change is a very difficult problem because in some cases it can mean
rewritting all previous data, in other cases it just means start using the new
key now but keep the
> old one.   Which is correct depends on why you are doing a key change.  Key
change for data at rest is a very different problem space from rekey in a
network protocol.
Key change is nice and algorithm change possibility is also nice in case
the one you use become broken.
What I''m doing in geli (my disk encryption software for FreeBSD) is to
use random, strong master key, which is encrypted by user''s passphrase,
keyfiles, etc. This is nice because changing user''s passphrase
doesn''t
affect the master key, thus doesn''t cost any I/O operations.
Another nice thing about it is that you can have many copies of the
master key protected by different passphrases. For example two persons
can decrypt your data: you and security officer in your company.

On the other hand, changing the master key should also be possible.

A good starting point IMHO will be to support user''s passphrase
(keyfile, etc.) change (without touching the master key) and document
changing the master key, algorithm, key length, etc. via eg. local zfs
send/recv.
> In theory the algorithm could be different per dnode_phys_t just like
checksum/compression are today, however having aes-128 on one dnode and aes-256
on another causes a
> problem because you also need different keys for them, it gets even more
complex if you consider the algorithm mode and if you choose completely
different algorithms.
> Having a different algorithm and key length will certainly be possible for
different filesystems though (eg root with aes-128 and home with aes-256).
Maybe keys should be pool''s properties? You add new key to the pool and
then assign selected key to the given datasets. You can then "unlock"
the key using zpool(1M) or you''ll be asked to "unlock" all
keys used by
a dataset when you want to mount/attach it (file system or zvol). Once
the key is "unlocked", the remaining datasets that use the same key
can
be mounted/attached automatically. Just a thought...

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070202/bcce6a9d/attachment.bin>

Georg Sauthoff wrote:
> On Fri, Feb 02, 2007 at 08:46:34AM +0000, Darren J Moffat wrote:
> 
> Hi,
> 
> [..]
>> Key change is a very difficult problem because in some cases it can
mean
>> rewritting all previous data, in other cases it just means start using 
> 
> probably a dumb question: What about autogenerating a key from
> /dev/random and store it encrypted by a user key in a filesystem header?
> The FS-data is encrypted with the autogenerated key and if the user
> changes the his user key, only the specific filesystem header have to be
> rewritten.
That is actually one of the things I have considered.  However that 
doesn''t necessarily address what some people meas when they say key
roll
over or key change.  What do you do if it is the actual encryption key 
that is compromised not the wrapping key (what you called the user key)?

What you have described actually has some good properties for key escrow 
(which wither people like it or not is very important for some 
customers, and not just government ones!).

There are many reasons why key change/rollover are recommended.  In 
network protocols there are often recommendations to change keys based 
on the amount of data encrypted and/or the time the key has been in use. 
  Those don''t apply the same way for data at rest, because the risks 
aren''t the same.

-- 
Darren J Moffat

[ note I''ve dropped zfs-discuss now, just zfs-crypto-discuss is
cc''d ]

Pawel Jakub Dawidek wrote:
> I''m sorry it seems I started my explanations from too deep. I
started to
> play with encryption on my own by creating a "crypto" compression
> algorithm.  
Please join us in the zfs-crypto project and share your experiences and 
ideas.
> Currently there are few types of property (readonly,
> inherited, etc.), but non of them seems to be suitable for encryption.
Agreed.
> When you enable encryption there should be no data, or you know that
> existing data is going to be encrypted and plaintext data securely
> removed automatically. Of course the later is much more complex to
> implement.
Agreed and this is why the current design of the zfs-crypto project 
states that encryption will be a data set creation time only property. 
COW makes the later case very difficult to deal with, clones make it 
more so, but even without COW clones/snapshots it still isn''t trivial.
>> My current plan is that once set the encryption property that describes
which algorithm (mechanism actually: algorithm, key length and mode, eg
aes-128-ccm) can not be
>> changed, it would be inherited by any clones. Creating new child file
systems "rooted" in an encrypted filesystem you would be allowed to
turn if off (I''d like to have a
>> policy like the acl one here) but by default it would be inherited.
> 
> Right. I forget that a dataset created under another dataset
doesn''t
> share data with the parent.
But it might be desirable to share policy.
>> Key change is a very difficult problem because in some cases it can
mean rewritting all previous data, in other cases it just means start using the
new key now but keep the
>> old one.   Which is correct depends on why you are doing a key change. 
Key change for data at rest is a very different problem space from rekey in a
network protocol.
> 
> Key change is nice and algorithm change possibility is also nice in case
> the one you use become broken.
Nice ?  What it it takes 3 months to rewrite all your data ?  Is that 
nice ?  What is happening to your production performance while this is 
happening ? What gets priority key change or "real" work ?
> What I''m doing in geli (my disk encryption software for FreeBSD)
is to
> use random, strong master key, which is encrypted by user''s
passphrase,
> keyfiles, etc. This is nice because changing user''s passphrase
doesn''t
> affect the master key, thus doesn''t cost any I/O operations.
> Another nice thing about it is that you can have many copies of the
> master key protected by different passphrases. For example two persons
> can decrypt your data: you and security officer in your company.
Indeed and this is one of the reasons why this is already being 
considered as one of the many key management options for ZFS crypto.
> On the other hand, changing the master key should also be possible.
At what expense though ?
What risk are you protecting against ?
For a small amount of data this probably isn''t too bad but consider 
doing a key change for a terrabyte of data that is already in very 
active use.

This is a very interesting problem area but quite a difficult one.
> A good starting point IMHO will be to support user''s passphrase
> (keyfile, etc.) change (without touching the master key) and document
> changing the master key, algorithm, key length, etc. via eg. local zfs
> send/recv.
I''m glad you brought up send/recv because that is one of the areas that
will need some work.

I think we need to have the ability for both clear and cipher text 
send/recv.
>> In theory the algorithm could be different per dnode_phys_t just like
checksum/compression are today, however having aes-128 on one dnode and aes-256
on another causes a
>> problem because you also need different keys for them, it gets even
more complex if you consider the algorithm mode and if you choose completely
different algorithms.
>> Having a different algorithm and key length will certainly be possible
for different filesystems though (eg root with aes-128 and home with aes-256).
> 
> Maybe keys should be pool''s properties? You add new key to the
pool and
> then assign selected key to the given datasets. You can then
"unlock"
> the key using zpool(1M) or you''ll be asked to "unlock"
all keys used by
> a dataset when you want to mount/attach it (file system or zvol). Once
> the key is "unlocked", the remaining datasets that use the same
key can
> be mounted/attached automatically. Just a thought...
I''ve thought about exactly that as well.  I think this might still be 
useful, particularly in the case of very large pools with large numbers 
of datasets and the reason for doing crypto is either "untrusted path to 
  disk (eg SAN, iSCSI)" or "physical theft of the storage".

Thank you so much for sharing your ideas.

If you haven''t done so already please have a read over the documents in
the zfs-crypto project files area.

On Fri, Feb 02, 2007 at 08:46:34AM +0000, Darren J Moffat
wrote:
> My current plan is that once set the encryption property that describes 
> which algorithm (mechanism actually: algorithm, key length and mode, eg 
> aes-128-ccm) can not be changed, it would be inherited by any clones. 
> Creating new child file systems "rooted" in an encrypted
filesystem you
> would be allowed to turn if off (I''d like to have a policy like
the acl
> one here) but by default it would be inherited.
> 
> Key change is a very difficult problem because in some cases it can mean 
> rewritting all previous data, in other cases it just means start using 
> the new key now but keep the old one.   Which is correct depends on why 
> you are doing a key change.  Key change for data at rest is a very 
> different problem space from rekey in a network protocol.
Re-keying and algorithm change should be seen as related.

And encryption off -> null encryption algorithm.
> In theory the algorithm could be different per dnode_phys_t just like 
> checksum/compression are today, however having aes-128 on one dnode and 
> aes-256 on another causes a problem because you also need different keys 
> for them, it gets even more complex if you consider the algorithm mode 
> and if you choose completely different algorithms.  Having a different 
> algorithm and key length will certainly be possible for different 
> filesystems though (eg root with aes-128 and home with aes-256).
I don''t see why having a different key per-dnode is so difficult,
particularly if the per-dnode key isn''t randomly generated, but derived
from the filesystem''s master key and the dnode number (then you
don''t
have to store it anywhere).  And you might have to have per-dnode keys
because of the birthday bound on key usage.  Just make sure that the
master key is always randomly generated, rather than derived from a
passphrase.

Now, the cipher modes you''ll use matter, as you could, over the life of
a file, end up using the same block more than once with the same key.  A
simple counter mode with the counter being based on the block address
would leak whether the same data has been written encrypted in the same
key; if the counter is based on the file offset and you have per-dnode
keys then the leak would be whether the same data has been written at
the same file offset in the same file.

Can you replace the ZFS checksum function with the MAC from an
authenticated encryption cipher mode?

If not then not using authenticated encryption would mean not having to
worry about where to store a MAC, and the existing ZFS checksum
functionality (using a cryptographic hash function) in combination with
encryption should buy you integrity protection semantics very close to
what you''d get with an authenticated cipher mode.

Nico
--

On Fri, Feb 02, 2007 at 12:25:04AM +0100, Pawel Jakub Dawidek
wrote:
> On Thu, Feb 01, 2007 at 11:00:07AM +0000, Darren J Moffat wrote:
> > Neil Perrin wrote:
> > >No it''s not the final version or even the latest!
> > >The current on disk format version is 3. However, it
hasn''t
> > >diverged much and the znode/acl stuff hasn''t changed.
> > 
> > and it will get updated as part of zfs-crypto, I just haven''t
done so yet because I''m not finished designing yet.
> 
> Do you consider adding a new property type (next to readonly and
> inherit) - a "oneway" property? Such propery could be only set if
the
> dataset has no children, no snapshots and no data, and once set
can''t be
> modified. "oneway" would be the type of the
"encryption" property.
> On the other hand you may still want to support encryption algorithm
> change and most likely key change.
I think you asking about some sort of magic property behaviour.  I
imagine magic properties like these:

 - all_ascii

   True if all filesystem object names are US-ASCII only (i.e., there
   are no bytes with the high bit set in such names).


 - all_utf8

   True if all filesystem object names are UTF-8 only (in terms of
   encoding; the filesystem couldn''t tell if there''s any
   codeset/encoding aliasing going on, of course).


 - all_encrypted

   True if all contents is encrypted.


 - encryption_start

   Timestamp from which all new content is encrypted.


and so on.  These are all magical in that you can''t set them, or if you
can you can only set them to a subset of their possible values.

Nico
--

On Fri, Feb 02, 2007 at 10:57:27AM +0000, Darren J Moffat
wrote:
> >Key change is nice and algorithm change possibility is also nice in
case
> >the one you use become broken.
> 
> Nice ?  What it it takes 3 months to rewrite all your data ?  Is that 
> nice ?  What is happening to your production performance while this is 
> happening ? What gets priority key change or "real" work ?
It all depends, doesn''t it?

I could see having a master key rollover where changing the master key
affects new contents only.  Plus an optional filesystem scrub to
re-write existing content and re-key/compress/whatever in the process (a
notion of priority would be useful here).

Do you want to have per-dnode keys?  Derived from the master key?  Or
randomly generated (and stored in a dnode header encrypted in the master
key).
> >On the other hand, changing the master key should also be possible.
> 
> At what expense though ?
> What risk are you protecting against ?
If your keys have been compromised (because your passphrase has been, or
because your token has been, and the attacker had access to your volume)
then you have a problem.

It''s not clear that re-keying saves you, because if the attacker had
access to your keys and data, well, horse, barn, door.  But then again,
if you have terabytes of data then chances are the attacker can''t get
all of it without being detected, so re-keying that which the attacker
did not get could be important.

If the attacker never had access to the master keys, just your
passphrase, then changing the passphrase should suffice.
> >A good starting point IMHO will be to support user''s
passphrase
> >(keyfile, etc.) change (without touching the master key) and document
> >changing the master key, algorithm, key length, etc. via eg. local zfs
> >send/recv.
> 
> I''m glad you brought up send/recv because that is one of the areas
that
> will need some work.
> 
> I think we need to have the ability for both clear and cipher text 
> send/recv.
Yes.
> >Maybe keys should be pool''s properties? You add new key to the
pool and
> >then assign selected key to the given datasets. You can then
"unlock"
> >the key using zpool(1M) or you''ll be asked to
"unlock" all keys used by
> >a dataset when you want to mount/attach it (file system or zvol). Once
> >the key is "unlocked", the remaining datasets that use the
same key can
> >be mounted/attached automatically. Just a thought...
> 
> I''ve thought about exactly that as well.  I think this might still
be
> useful, particularly in the case of very large pools with large numbers 
> of datasets and the reason for doing crypto is either "untrusted path
to
>  disk (eg SAN, iSCSI)" or "physical theft of the storage".
Does this help though?  If each dataset has its own master keys but
several have their master keys encrypted in the same passphrase(s) then
don''t you get the same semantics?

Nico
--

On Fri, Feb 02, 2007 at 10:57:27AM +0000, Darren J Moffat
wrote:
> >>Key change is a very difficult problem because in some cases it can
mean rewritting all previous data, in other cases it just means start using the
new key now but keep
> >>the old one.   Which is correct depends on why you are doing a key
change.  Key change for data at rest is a very different problem space from
rekey in a network
> >>protocol.
> >Key change is nice and algorithm change possibility is also nice in
case
> >the one you use become broken.
> 
> Nice ?  What it it takes 3 months to rewrite all your data ?  Is that nice
?  What is happening to your production performance while this is happening ?
What gets priority
> key change or "real" work ?
''scrub'' isn''t that bad and also have to checksum all
the data...
And remember ''make tools, not policies'':) Allow the user to
decide if
danger for his data is high enough that master key or algorithm change
is worth it, but provide tools to do so.
> >On the other hand, changing the master key should also be possible.
> 
> At what expense though ?
> What risk are you protecting against ?
> For a small amount of data this probably isn''t too bad but
consider doing a key change for a terrabyte of data that is already in very
active use.
> 
> This is a very interesting problem area but quite a difficult one.
I don''t suggest any way of how this can be done. This may be done by
implementing it or by documenting how to do it using other functionality
(like local zfs send/receive).
> >A good starting point IMHO will be to support user''s
passphrase
> >(keyfile, etc.) change (without touching the master key) and document
> >changing the master key, algorithm, key length, etc. via eg. local zfs
> >send/recv.
> 
> I''m glad you brought up send/recv because that is one of the areas
that will need some work.
> 
> I think we need to have the ability for both clear and cipher text
send/recv.
Yeah, but this may be tricky. If I recall correctly someone is working
on ''zfs send -r'', which bascially sends entire pool, not one,
selected
dataset. I think this is good starting point, ie. send plaintext data
with ''zfs send'' and encrypted data with ''zfs send
-r''.

In FreeBSD there is a GEOM layer, which allows to stack
"transformations" freely. For example you have two disks (/dev/da0 and
/dev/da1), you configure RAID1 on top of them, which gives you
/dev/mirror/foo, then you encrypt this device and get
/dev/mirror/foo.eli.  Now you can still access /dev/da0, /dev/da1,
/dev/mirror/foo and /dev/mirror/foo.eli where:

/dev/da0, /dev/da1 - halfs of the mirror
/dev/mirror/foo - encrypted data
/dev/mirror/foo.eli - unencrypted data
If you now do block-level backup, you can backup plaintext data by
reading from /dev/mirror/foo.eli or encrypted data by reading from
/dev/mirror/foo, /dev/da0 or /dev/da1.

In other words, will be very helpful if encrypted blocks are exported
somehow, but I''m not sure how this could look like (a coresponding zvol
for every encrypted zfs file system?)...
> >Maybe keys should be pool''s properties? You add new key to the
pool and
> >then assign selected key to the given datasets. You can then
"unlock"
> >the key using zpool(1M) or you''ll be asked to
"unlock" all keys used by
> >a dataset when you want to mount/attach it (file system or zvol). Once
> >the key is "unlocked", the remaining datasets that use the
same key can
> >be mounted/attached automatically. Just a thought...
> 
> I''ve thought about exactly that as well.  I think this might still
be useful, particularly in the case of very large pools with large numbers of
datasets and the reason for
> doing crypto is either "untrusted path to  disk (eg SAN, iSCSI)"
or "physical theft of the storage".
Hmm, it would be nice to separate user-derived keys (passphrase, random
bits from a USB-stick, smartcards, etc.), policies (algorithm, key
length, master key) and datasets and freely make connections between
them. For example:

	User-derived keys	Policies	Datasets

	pjd			strong		tank
	joe			fast		tank/pjd
	secofficer		very_strong	tank/joe
						tank/company
						tank/company/topsecret

And we make connections:

	pjd        ---> fast        ---> tank/pjd
	secofficer ---> fast        ---> tank/pjd
	joe        ---> fast        ---> tank/joe
	secofficer ---> fast        ---> tank/joe
	pjd        ---> strong      ---> tank/company
	secofficer ---> strong      ---> tank/company
	secofficer ---> very_strong ---> tank/company/topsecret

Unfortunately, this will probably introduce too huge administrative
overhead:)
> If you haven''t done so already please have a read over the
documents in the zfs-crypto project files area.
I read them already some time ago, will get back to them in case I
repeat something that was already said there.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20070202/5b1a050d/attachment.bin>

On Fri, Feb 02, 2007 at 06:42:56PM +0100, Pawel Jakub Dawidek
wrote:
> On Fri, Feb 02, 2007 at 10:57:27AM +0000, Darren J Moffat wrote:
> > I''m glad you brought up send/recv because that is one of the
areas
> > that will need some work.
> > 
> > I think we need to have the ability for both clear and cipher text
> > send/recv.
> 
> Yeah, but this may be tricky. If I recall correctly someone is working
> on ''zfs send -r'', which bascially sends entire pool, not
one, selected
> dataset. I think this is good starting point, ie. send plaintext data
> with ''zfs send'' and encrypted data with ''zfs
send -r''.
I think -r is orthogonal to this.  Because parts of the metadata itself
might/should be encrypted you could not expect to be able to zfs send -r
an entire pool unless you''ve unlocked ALL encrypted datasets therein.

[...]
> In other words, will be very helpful if encrypted blocks are exported
> somehow, but I''m not sure how this could look like (a coresponding
> zvol for every encrypted zfs file system?)...
If you wanted block-level backup for ZFS you''d be backing up entire
pools, not datasets.  I.e., you''d access the vdevs.  Now, ZFS crypto
will not be at the block device level (that''s what lofi crypto is for),
so it makes no sense to do block-level backups of encrypted ZFS in
plaintext.
> > I''ve thought about exactly that as well.  I think this might
still
> > be useful, particularly in the case of very large pools with large
> > numbers of datasets and the reason for doing crypto is either
> > "untrusted path to  disk (eg SAN, iSCSI)" or "physical
theft of the
> > storage".
> 
> Hmm, it would be nice to separate user-derived keys (passphrase,
> random bits from a USB-stick, smartcards, etc.), policies (algorithm,
> key length, master key) and datasets and freely make connections
> between them. For example:
> 
> 	User-derived keys	Policies	Datasets
> 
> 	pjd			strong		tank
> 	joe			fast		tank/pjd
> 	secofficer		very_strong	tank/joe
> 						tank/company
> 						tank/company/topsecret
> 
> And we make connections:
> 
> 	pjd        ---> fast        ---> tank/pjd
> 	secofficer ---> fast        ---> tank/pjd
> 	joe        ---> fast        ---> tank/joe
> 	secofficer ---> fast        ---> tank/joe
> 	pjd        ---> strong      ---> tank/company
> 	secofficer ---> strong      ---> tank/company
> 	secofficer ---> very_strong ---> tank/company/topsecret
> 
> Unfortunately, this will probably introduce too huge administrative
> overhead:)
I don''t follow.  I think each dataset should have its own
"locks": the
master key encrypted in one or more passphrases (one per-user, say).
Then locks would be related only by whether a they use the same
passphrase.  Policy here would be attached to each dataset, not to each
user, but the system could still impose per-user policies at a higher
level.

Nico
--

On Fri, Feb 02, 2007 at 10:48:56AM +0000, Darren J Moffat
wrote:
> Georg Sauthoff wrote:
> >On Fri, Feb 02, 2007 at 08:46:34AM +0000, Darren J Moffat wrote:
> >>Key change is a very difficult problem because in some cases it can
mean
> >>rewritting all previous data, in other cases it just means start
using
> >probably a dumb question: What about autogenerating a key from
> >/dev/random and store it encrypted by a user key in a filesystem
header?
> >The FS-data is encrypted with the autogenerated key and if the user
> >changes the his user key, only the specific filesystem header have to
be
> >rewritten.
 
> That is actually one of the things I have considered.  However that 
That is good to hear/read.
> doesn''t necessarily address what some people meas when they say
key roll
> over or key change.  What do you do if it is the actual encryption key 
> that is compromised not the wrapping key (what you called the user key)?
Well, that depends ;) If the actual encryption key leaked somehow
(somebody got root, modified the kernel etc.) then the administrator
could wipe the storage with his favourite disk-wiping tool[1], setup new
filesystems and restore the backup.

Ok, like discussed in an adjacent sub-thread it could take some time to
restore the data - which means offline time - thus in some usecases one
could need something like online-real-key-exchange.
 
> What you have described actually has some good properties for key escrow 
> (which wither people like it or not is very important for some 
> customers, and not just government ones!).
Yes, or you can use multiple wrapping keys, like Pawel D. is
implementing it and the endorphin.org guy did implement it. This
scales better if the security admin has some policy like ''change master
key every month'', I guess.

In the end it is nice to have both features: multiple wrapping keys and
real-key-exchange (offline or online).

Best regards
Georg Sauthoff

[1] or if the admin is really paranoid he would burn the storage
hardware and buy new one ;)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20070202/29565671/attachment.bin>

On Fri, Feb 02, 2007 at 12:02:45PM -0600, Nicolas Williams
wrote:
> On Fri, Feb 02, 2007 at 06:42:56PM +0100, Pawel Jakub Dawidek wrote:
> > On Fri, Feb 02, 2007 at 10:57:27AM +0000, Darren J Moffat wrote:
> > > I''m glad you brought up send/recv because that is one of
the areas
> > > that will need some work.
> > > 
> > > I think we need to have the ability for both clear and cipher
text
> > > send/recv.
> > 
> > Yeah, but this may be tricky. If I recall correctly someone is working
> > on ''zfs send -r'', which bascially sends entire pool,
not one, selected
> > dataset. I think this is good starting point, ie. send plaintext data
> > with ''zfs send'' and encrypted data with
''zfs send -r''.
> 
> I think -r is orthogonal to this.  Because parts of the metadata itself
> might/should be encrypted you could not expect to be able to zfs send -r
> an entire pool unless you''ve unlocked ALL encrypted datasets
therein.
As far as I know ZFS metadata are not going to be encrypted, only file
system metadata, files context, etc. I don''t know where exactly ZFS
metadata ends and file system metadata starts, but I expect you can tell
what dataset you have and which blocks belong to which datasets without
decrypting.
> > > I''ve thought about exactly that as well.  I think this
might still
> > > be useful, particularly in the case of very large pools with
large
> > > numbers of datasets and the reason for doing crypto is either
> > > "untrusted path to  disk (eg SAN, iSCSI)" or
"physical theft of the
> > > storage".
> > 
> > Hmm, it would be nice to separate user-derived keys (passphrase,
> > random bits from a USB-stick, smartcards, etc.), policies (algorithm,
> > key length, master key) and datasets and freely make connections
> > between them. For example:
> > 
> > 	User-derived keys	Policies	Datasets
> > 
> > 	pjd			strong		tank
> > 	joe			fast		tank/pjd
> > 	secofficer		very_strong	tank/joe
> > 						tank/company
> > 						tank/company/topsecret
> > 
> > And we make connections:
> > 
> > 	pjd        ---> fast        ---> tank/pjd
> > 	secofficer ---> fast        ---> tank/pjd
> > 	joe        ---> fast        ---> tank/joe
> > 	secofficer ---> fast        ---> tank/joe
> > 	pjd        ---> strong      ---> tank/company
> > 	secofficer ---> strong      ---> tank/company
> > 	secofficer ---> very_strong ---> tank/company/topsecret
> > 
> > Unfortunately, this will probably introduce too huge administrative
> > overhead:)
> 
> I don''t follow.  I think each dataset should have its own
"locks": the
> master key encrypted in one or more passphrases (one per-user, say).
> Then locks would be related only by whether a they use the same
> passphrase.  Policy here would be attached to each dataset, not to each
> user, but the system could still impose per-user policies at a higher
> level.
I think this is more or less what I tried to explain:)
On my picture, there is pjd''s passphrase which is used to encrypt to
master keys (master key from "fast" policy, and master key from
"strong"
policy). secofficer''s passphrase is used to encrypt three master keys
(from "fast", "strong" and "very_strong"
policies). joe''s passphrase is
used to encrypt only one master key.

The difference between my picture and your description is that I allow
one policy to be used for more than one dataset, which isn''t wise if
the
master key is a policy property (joe will be able to decrypt master key
from the "fast" policy and use it to decrypt tank/pjd). But this is
more
adminitrative issue.

What I''d like to be able to say is:
1. Add policy X, use AES/256/CBC for encryption and HMAC/SHA256 for
   authentication..
2. Policy X may be unlocked by pjd''s, joe''s and
secofficer''s
   passphrases.
3. Use policy X to protect tank/foo, tank/bar and tank/baz.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20070203/2f9824c8/attachment.bin>