zfs crypto discuss - Apr 2007 - crypto & some zfs operations

I''ve been trying to validate my crypto choices against all of the
operations one can do to a dataset today and some of those that may come
soon.

I have some that I think could cause issues with the current design,
where the blkptr_t just has the algorithm and the key is per dataset.

1) clone promotion
=================
The actual promotion of the clone isn''t what causes a problem, but what
it allows you to do afterwards. For example:

# zfs create -o encryption=on p/a
<< create key K-a and store for dataset p/a >>
# zfs snapshot p/a at t1
<< nothing to happen crypto wise here >>
# zfs clone p/a at t1 p/b
<< create new key K-b for data only in p/b >>
<< At this point everything is okay >>
# zfs promote p/b
<< Still okay I think >>
# zfs destroy p/a

This is where we have a possible problem, at this point we would
normally be deleting key K-a but we can''t because data that now
belongs to p/b is still encrypted with K-a because p/b owns the
old p/a at t1 snapshot now.  This means that p/b needs to have both
K-a and K-b and needs to know which blocks are encrypted with each key.
All new data and changes to old blocks are encrypted using K-b, but
we still need K-a to read the older stuff that was originally in p/a
up until p/a at t1.

2) dataset split
===============
# zfs create -o encryption=on p/a
<< create key K-a and store for dataset p/a >>
# zfs split -o atdir=/stuff/home/ p/a p/b
<< At directory /stuff/home in dataset p/a create a new dataset p/b
    containing everything below p/b.
    We have two choices here:
	a) Copy K-a to K-b as we create p/b
	b) Create a new K-b for p/b

Option a) while not great from a pure crypto/security view, because we
now have two datasets using the same encryption key, makes things
easiest from a ZFS view point.  This might not be that bad if the
wrapped key K-b that we store on disk is different in its on disk
representation to the wrapped K-a.  Although zfs history would still
show that this had happened so it would be possible to discover that
the raw K-a and K-b would be the same.

Option b) means we have a mix of blocks encrypted under K-a and K-b.

3) dataset join
==============
3.1) two encrypted datasets

# zfs create -o encryption=on p/t/a
<< create key K-a and store for dataset p/t/a >>
# zfs create -o encryption=on p/t/b
<< create key K-b and store for dataset p/t/b >>

# zfs join p/t/a p/t/b p/s
<< Combine p/t/a and p/t/b as p/s >>

This will mean that p/s has a mix of blocks from p/t/a encrypted
with K-a and blocks from p/t/b encrypted with K-b.  Should p/s
also have K-s for any data created after the join ?

3.2) one encrypted and one in clear
The case above isn''t the biggest issue with join though this one is
potentially worse:

# zfs create -o encryption=on p/t/a
<< create key K-a and store for dataset p/t/a >>
# zfs create -o encryption=off p/t/b
<< p/t/b is a cleartext dataset >>

# zfs join p/t/a p/t/b p/s
<< Combine p/t/a and p/t/b as p/s >>

Now we have a different issue, we only have K-a to deal with but we now
have a mix of blocks that are encrypted and some that aren''t.  This is
equivalent to turning on encryption after a dataset already has content
in it; which is something the current design does not allow.  So maybe
the answer for this case is you can''t do this join.

3.3) two encrypted datasets with a twist

# zfs create -o encryption=aes-128 p/t/a
<< create key K-a and store for dataset p/t/a >>
# zfs create -o encryption=aes-256 p/t/b
<< create key K-b and store for dataset p/t/b >>

# zfs join p/t/a p/t/b p/s
<< Combine p/t/a and p/t/b as p/s >>

This is similar to 3.1 in that we have two keys we need to deal with but
  it also has problems that are similar to 3.2.  If the join is allowed
to succeed we have a mix of blocks encrypted with K-a using aes-128 and
a mix using K-b using aes-256.  So how strong is the key crypto
protecting our dataset p/s ?  The answer for the user is it is 128
because we can''t easily see what blocks were encrypted with the 256
key;
the protection of the data hasn''t changed so from the pool perspective
we aren''t worse off but we shouldn''t lie to the user about a
given
dataset.  So this is a downgrade, but the original intent of the user
might actually have been an upgrade.

Keylength in this case isn''t the only issue, consider that K-a was an
aes-256 key and K-b was foo-256, which one do we choose for the new
blocks ? is aes-256 stronger than foo-256 or vice versa ?  In this case
we can''t pick the weakest to ensure we don''t lie to the user
because we
don''t know which is weaker.


General conclusions
==================
There is a common theme to all these cases.  In the majority of the
cases described above we can end up in a situation where we have more
that one key needed to read and write blocks for a given dataset.  In
the promote and join cases the examples show only two keys but we could
actually end up doing these operations multiple times and that means we
need N keys.

This is exactly the same situation we get into if we provide an explicit
rekey or change key capability; unless a key change operation locked all
writes until everything was rewritten (which I don''t think is at all
acceptable for all but the smallest of datasets).

We haven''t mentioned COW in all of this yet.  In the disjoint algorithm
cases if we picked the "better" of the two rather than the
"weakest" we
would have a problem even after all the old data has gone because of
COW.  The old clear text, or less secure algorithm, data may still be on
disk.


This makes me think that we need to know for every block which key and
algorithm it was encrypted with.  We also shouldn''t allow operations
which would end up with a mix of different algorithms (including on and
off) because it is just too hard to rationalise to end users; one needs
to understand how ZFS actually works to know what situations we could be
in.  Not even a force flag is a good idea here.

Even if we had a capability of scrubbing the blocks on the free list and
rewritting (using COW) the older data it could take a very long time.

Unlike compression what the user observes here really matters a lot.
With compression it is really a space/cpu trade off, with encryption we
are making statements about the confidentiality of the data in a given
dataset.

-- 
Darren J Moffat

On Fri, 2007-04-27 at 18:44 +0100, Darren J Moffat
wrote:
> This makes me think that we need to know for every block which key and
> algorithm it was encrypted with. 

absolutely.  I think there are a couple options: 
	- key version number in the block pointer
	- use the "block birth" value to find the key, via indexing into a
(hopefully small) binary tree of keys.  (I *think* it would be
reasonable to require that every block in a dataset written during the
same transaction group use the same key).

						- Bill

On Fri, Apr 27, 2007 at 06:44:35PM +0100, Darren J Moffat
wrote:
> I''ve been trying to validate my crypto choices against all of the
> operations one can do to a dataset today and some of those that may come
> soon.
> 
> I have some that I think could cause issues with the current design,
> where the blkptr_t just has the algorithm and the key is per dataset.
>
> 1) clone promotion
> =================> 
> The actual promotion of the clone isn''t what causes a problem, but
what
> it allows you to do afterwards. For example:
> 
> # zfs create -o encryption=on p/a
> << create key K-a and store for dataset p/a >>
> # zfs snapshot p/a at t1
> << nothing to happen crypto wise here >>
> # zfs clone p/a at t1 p/b
> << create new key K-b for data only in p/b >>
> << At this point everything is okay >>
> # zfs promote p/b
> << Still okay I think >>
> # zfs destroy p/a
> 
> This is where we have a possible problem, at this point we would
> normally be deleting key K-a but we can''t because data that now
> belongs to p/b is still encrypted with K-a because p/b owns the
> old p/a at t1 snapshot now.  This means that p/b needs to have both
> K-a and K-b and needs to know which blocks are encrypted with each key.
> All new data and changes to old blocks are encrypted using K-b, but
> we still need K-a to read the older stuff that was originally in p/a
> up until p/a at t1.
In my opinion we should not create K-b for the clone - the two datasets
are closly related (share the same data), that''s why I think using the
same key is fine.
> 2) dataset split
> ===============> 
> # zfs create -o encryption=on p/a
> << create key K-a and store for dataset p/a >>
> # zfs split -o atdir=/stuff/home/ p/a p/b
> << At directory /stuff/home in dataset p/a create a new dataset p/b
>    containing everything below p/b.
>    We have two choices here:
> 	a) Copy K-a to K-b as we create p/b
> 	b) Create a new K-b for p/b
> 
> Option a) while not great from a pure crypto/security view, because we
> now have two datasets using the same encryption key, makes things
> easiest from a ZFS view point.  This might not be that bad if the
> wrapped key K-b that we store on disk is different in its on disk
> representation to the wrapped K-a.  Although zfs history would still
> show that this had happened so it would be possible to discover that
> the raw K-a and K-b would be the same.
> 
> Option b) means we have a mix of blocks encrypted under K-a and K-b.
I think a) is acceptable. You give passphrase for the entire pool, not
for individual datasets, right? In other words you can''t assign p/a to
user X and p/b to user Y and allow them to use different passphrases for
their datasets? (users, zone administrators, whatever)

I don''t know how ''zfs split'' is suppose to work
exactly, how does it
deal with hardlinks, etc., but this is off-topic.
> 3) dataset join
> ==============> 
> 3.1) two encrypted datasets
> 
> # zfs create -o encryption=on p/t/a
> << create key K-a and store for dataset p/t/a >>
> # zfs create -o encryption=on p/t/b
> << create key K-b and store for dataset p/t/b >>
> 
> # zfs join p/t/a p/t/b p/s
> << Combine p/t/a and p/t/b as p/s >>
> 
> This will mean that p/s has a mix of blocks from p/t/a encrypted
> with K-a and blocks from p/t/b encrypted with K-b.  Should p/s
> also have K-s for any data created after the join ?
If you can''t change the key after dataset creation, it''s quite
understandable that you also can''t join two datasets with different
keys...
> 3.2) one encrypted and one in clear
> The case above isn''t the biggest issue with join though this one
is
> potentially worse:
> 
> # zfs create -o encryption=on p/t/a
> << create key K-a and store for dataset p/t/a >>
> # zfs create -o encryption=off p/t/b
> << p/t/b is a cleartext dataset >>
> 
> # zfs join p/t/a p/t/b p/s
> << Combine p/t/a and p/t/b as p/s >>
> 
> Now we have a different issue, we only have K-a to deal with but we now
> have a mix of blocks that are encrypted and some that aren''t. 
This is
> equivalent to turning on encryption after a dataset already has content
> in it; which is something the current design does not allow.  So maybe
> the answer for this case is you can''t do this join.
Again, if you can''t configure encryption after dataset creation, you
should not be able to join such dataset as well. It''s actually maybe be
confusing if such an operation will be possible - I''m sure it can rise
questions similar to "Does it mean that all my data is encrypted or does
it mean that it''s all decrypted now?".
> 3.3) two encrypted datasets with a twist
> 
> # zfs create -o encryption=aes-128 p/t/a
> << create key K-a and store for dataset p/t/a >>
> # zfs create -o encryption=aes-256 p/t/b
> << create key K-b and store for dataset p/t/b >>
> 
> # zfs join p/t/a p/t/b p/s
> << Combine p/t/a and p/t/b as p/s >>
> 
> This is similar to 3.1 in that we have two keys we need to deal with but
>  it also has problems that are similar to 3.2.  If the join is allowed
> to succeed we have a mix of blocks encrypted with K-a using aes-128 and
> a mix using K-b using aes-256.  So how strong is the key crypto
> protecting our dataset p/s ?  The answer for the user is it is 128
> because we can''t easily see what blocks were encrypted with the
256 key;
> the protection of the data hasn''t changed so from the pool
perspective
> we aren''t worse off but we shouldn''t lie to the user
about a given
> dataset.  So this is a downgrade, but the original intent of the user
> might actually have been an upgrade.
> 
> Keylength in this case isn''t the only issue, consider that K-a was
an
> aes-256 key and K-b was foo-256, which one do we choose for the new
> blocks ? is aes-256 stronger than foo-256 or vice versa ?  In this case
> we can''t pick the weakest to ensure we don''t lie to the
user because we
> don''t know which is weaker.
And again:) If you can''t change encryption algorithm/key length after
dataset creation, this operation should also be denied.
> General conclusions
> ==================> 
> There is a common theme to all these cases.  In the majority of the
> cases described above we can end up in a situation where we have more
> that one key needed to read and write blocks for a given dataset.  In
> the promote and join cases the examples show only two keys but we could
> actually end up doing these operations multiple times and that means we
> need N keys.
That''s why the simplest and less confusing method is just deny such
configurations, IMHO.
> This is exactly the same situation we get into if we provide an explicit
> rekey or change key capability; unless a key change operation locked all
> writes until everything was rewritten (which I don''t think is at
all
> acceptable for all but the smallest of datasets).
> 
> We haven''t mentioned COW in all of this yet.  In the disjoint
algorithm
> cases if we picked the "better" of the two rather than the
"weakest" we
> would have a problem even after all the old data has gone because of
> COW.  The old clear text, or less secure algorithm, data may still be on
> disk.
In FreeBSD, next to BIO_READ and BIO_WRITE I/O request types we have
BIO_DELETE. BIO_DELETE means "I don''t need this range of data
anymore".
Such request should be send by a file system when you for example delete
a file.  It may be handled in many ways by the driver, it depends on
what kind of driver we send it to. For example if this is a "memory
disk", it should simply free the memory, but if it is a encryption
mechanism, it should securely destroy data at the given range.

I started to look how I could pass BIO_DELETE down from ZFS and ZFS
actually has something similar - ZIO_TYPE_FREE, which point at range of
storage to be freed. If encryption is turned on, we could use
ZIO_TYPE_FREE request to overwrite the given range using some secure
deletion mechanism. This of course may be optional, because the impact
on performance can be significant, but may we very important feature for
some uses (performance or security, pick one).
> This makes me think that we need to know for every block which key and
> algorithm it was encrypted with.  We also shouldn''t allow
operations
> which would end up with a mix of different algorithms (including on and
> off) because it is just too hard to rationalise to end users; one needs
> to understand how ZFS actually works to know what situations we could be
> in.  Not even a force flag is a good idea here.
> 
> Even if we had a capability of scrubbing the blocks on the free list and
> rewritting (using COW) the older data it could take a very long time.
> 
> Unlike compression what the user observes here really matters a lot.
> With compression it is really a space/cpu trade off, with encryption we
> are making statements about the confidentiality of the data in a given
> dataset.
-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20070427/809d0d4e/attachment.bin>

On Fri, 2007-04-27 at 18:44 +0100, Darren J Moffat
wrote:
> 3) dataset join
> ==============
last I heard, work on dataset join was a ways off -- each dataset has
its own inode number space and thus you''d need to renumber inodes and
rewrite all your directories as part of the merge.

unless this has changed this can probably be viewed as out of scope for
now..

					- Bill

Bill Sommerfeld wrote:
> On Fri, 2007-04-27 at 18:44 +0100, Darren J Moffat wrote:
>> 3) dataset join
>> ==============> 
> last I heard, work on dataset join was a ways off -- each dataset has
> its own inode number space and thus you''d need to renumber inodes
and
> rewrite all your directories as part of the merge.
> 
> unless this has changed this can probably be viewed as out of scope for
> now..
Indeed but I thought it was worth going through the excersise to see if 
adding crypto before would constrain how join would work.

I agree it is out of scope for now, but probably worth mentioning in an 
appendix that covers ZFS futures and how dataset crypto might interact 
with them.

-- 
Darren J Moffat

On Fri, Apr 27, 2007 at 03:54:41PM -0400, Bill Sommerfeld
wrote:
> On Fri, 2007-04-27 at 18:44 +0100, Darren J Moffat wrote:
> > This makes me think that we need to know for every block which key and
> > algorithm it was encrypted with. 
> 
> 
> absolutely.  I think there are a couple options: 
> 	- key version number in the block pointer
> 	- use the "block birth" value to find the key, via indexing into
a
> (hopefully small) binary tree of keys.  (I *think* it would be
> reasonable to require that every block in a dataset written during the
> same transaction group use the same key).
Does it mean that keys should be refcounted? How would you know that key
is no longer used and the key slot can be reused for a new key?
I may be wrong, but keys refcounting sounds expensive.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20070430/d996b5bb/attachment.bin>

Bill Sommerfeld wrote:
> On Fri, 2007-04-27 at 18:44 +0100, Darren J Moffat wrote:
>> This makes me think that we need to know for every block which key and
>> algorithm it was encrypted with. 
> 
> 
> absolutely.  I think there are a couple options: 
> 	- key version number in the block pointer
That was my first though.  However I don''t think we have sufficient
space to get a big enough version space.
> 	- use the "block birth" value to find the key, via indexing into
a
> (hopefully small) binary tree of keys.  (I *think* it would be
> reasonable to require that every block in a dataset written during the
> same transaction group use the same key).
At the time we do a zio_write we have both the block birth and a 
zbookmark_t.  When we are doing a zio_read we also have the zbookmark_t.

A bookmark is a four-tuple <objset, object, level, blkid> that uniquely
identifies any block in the pool.

     209 typedef struct zbookmark {
     210 	uint64_t	zb_objset;
     211 	uint64_t	zb_object;
     212 	int64_t		zb_level;
     213 	uint64_t	zb_blkid;
     214 } zbookmark_t;

Because of those properties my POC already has the zbookmark_t
being passed into both the encrypt and decrypt functions.  I''m also
passing the block birth uint64_t into encrypt as well, but I''m not sure
at the moment if it is easily available on read (I can check though)



-- 
Darren J Moffat

On Mon, 2007-04-30 at 12:43 +0200, Pawel Jakub Dawidek
wrote:
> Does it mean that keys should be refcounted? 
I don''t think any refcounting would be required.  
> How would you know that key
> is no longer used and the key slot can be reused for a new key?
If each fileset keeps a set of keys, you could garbage-collect old keys
by walking the blocks referenced from the fileset (well, any block
containing a block pointer) which is probably going to be cheaper than
maintaining refcounts if keys aren''t changed frequently.

(no, I don''t have a completely worked out design for this -- just
tossing out ideas).

						- Bill