zfs crypto discuss - Jan 2008 - Proposal for how per dataset keys are initially setup

Anthony Scarpino wrote (elsewhere):
> While writing up the man page.. I thought of a few things that I was 
> wondering if you considered..
> 
> Can an encrypted dataset (keytype=dataset) reside in a non-encrypted (no 
> kek defined) pool?
I can see a case for and against allowing this when considering it 
purely at the feature level as users/admins see things.

The admin can control wither users can enable encryption or not by 
delegating (or not) the "encryption" and "keytype"
properties. So there
is no need to use the pool wrapping key (ie require the kek to be 
available) to restrict creation of encrytped datasets.

If the admin doesn''t want encryption in any dataset delegated to a user
then don''t delegate those properties and ensure they are inherited as 
encryption=off.


We need to be able to provide key change for keytype=dataset as well. 
That means that the actual encryption key for the dataset needs to be 
done the same way as keytype=pool, ie it is a wrapped key that is stored 
in the "wrappedkey" property not the actual encryption key.  
Regardless
of the need for key change the encryption key needs to be generated 
while the dataset is being created; and for the filesystem case it needs 
to happen before the root directory of the filesystem is created - 
leaving it unmounted isn''t actually enough.

So this means that we have an ordering problem for the initial key 
setup. We need the per dataset wrapping key to wrap the new randomly 
generated dataset encryption key while we are creating the dataset. That 
assumes that ''zfs key -l'' has been run, but we can''t
do that because the
dataset doesn''t exist yet.

We also worked out that this could be tricky for the cli, because it 
seems to imply a three step process that isn''t atomic - and this I
think
is a bad model.

Example 1:

	1$ zfs create -o encryption=on tank/foo

At this point we have a randomly generated encryption key wrapped by the
pool key.

	2$ zfs set keytype=dataset tank/foo

The property gets changed but we can''t re-wrap the encryption key 
because we don''t have the new per dataset wrapping key.

	3$ zfs key -l tank/foo
	Enter new passphrase:
	Re-Enter passphrase:
	4$ zfs mount tank/foo

Here ''zfs key -l'' "defines" the dataset key and does
an unwrap using the
pool key (which had to have been present or step 1 would have failed) 
and a (re)wrap using the newly defined dataset key.

[ Note that there is a per dataset property for choosing what type of 
key we are using - passphrase or file in phase 1 with pkcs11 coming in 
1.1, I''m just using passphrase for the examples in this discussion ].


That could be a problem since step 2 without step 3 leaves the dataset 
in a possibly unusable configuration.  However we do know that the per 
dataset key hasn''t actually be loaded yet (because the hidden per 
dataset property key_check would be undefined).  The big concern I have 
with this is how it looks to the user/admin when looking at the 
properties and the steps they have to take. After step 2 but before step 
3 they would likely see:

	encryption=on
	keytype=dataset
	kek=passphrase
	keystatus=undefined

Example 2:

A similar case with the same issue is this one:

	1$ zfs create -o encryption=on -o keytype=dataset tank/foo
	2$ zfs key -l tank/foo
	Enter new passphrase:
	Re-Enter passphrase:
	3$ zfs mount tank/foo


Some discussion that Tony and I had recently I think we came to the 
conclusion that what it might be better done like this:

Example 3:

	1$ zfs create -o encryption=on tank/foo

At this point we have a randomly generated encryption key wrapped by the
pool key.

	2$ zfs key -c -o keytype=dataset tank/foo
	Enter new passphrase:
	Re-Enter passphrase:
	3$ zfs mount tank/foo

We change the property, define the new per dataset wrapping key and 
re-wrap the dataset key and store it on disk using the new per dataset 
wrapping key, all as part of that single action.

Note here that the passphrase is the new one for the dataset not the 
pool passphrase.

This makes keytype a "strange" property because it means it can only
be
changed by ''zfs key'' and can''t be set at create time
to anything except
''pool'' (and then only if the pool wrapping key is present) -
which could
be an issue for property inheritance.


Does anyone have an alternate proposal ? [ please don''t nitpick on the 
property names at this time what I want discussed is the semantics not 
the syntax ].

-- 
Darren J Moffat

I''m not quite following all the cases here, but does it help to
collapse
the ''zfs create'' and ''zfs key -l'' steps into
one?

Cindi


Darren J Moffat wrote:
> Anthony Scarpino wrote (elsewhere):
>> While writing up the man page.. I thought of a few things that I was 
>> wondering if you considered..
>>
>> Can an encrypted dataset (keytype=dataset) reside in a non-encrypted
(no
>> kek defined) pool?
> 
> I can see a case for and against allowing this when considering it 
> purely at the feature level as users/admins see things.
> 
> The admin can control wither users can enable encryption or not by 
> delegating (or not) the "encryption" and "keytype"
properties. So there
> is no need to use the pool wrapping key (ie require the kek to be 
> available) to restrict creation of encrytped datasets.
> 
> If the admin doesn''t want encryption in any dataset delegated to a
user
> then don''t delegate those properties and ensure they are inherited
as
> encryption=off.
> 
> 
> We need to be able to provide key change for keytype=dataset as well. 
> That means that the actual encryption key for the dataset needs to be 
> done the same way as keytype=pool, ie it is a wrapped key that is stored 
> in the "wrappedkey" property not the actual encryption key.  
Regardless
> of the need for key change the encryption key needs to be generated 
> while the dataset is being created; and for the filesystem case it needs 
> to happen before the root directory of the filesystem is created - 
> leaving it unmounted isn''t actually enough.
> 
> So this means that we have an ordering problem for the initial key 
> setup. We need the per dataset wrapping key to wrap the new randomly 
> generated dataset encryption key while we are creating the dataset. That 
> assumes that ''zfs key -l'' has been run, but we
can''t do that because the
> dataset doesn''t exist yet.
> 
> We also worked out that this could be tricky for the cli, because it 
> seems to imply a three step process that isn''t atomic - and this I
think
> is a bad model.
> 
> Example 1:
> 
> 	1$ zfs create -o encryption=on tank/foo
> 
> At this point we have a randomly generated encryption key wrapped by the
> pool key.
> 
> 	2$ zfs set keytype=dataset tank/foo
> 
> The property gets changed but we can''t re-wrap the encryption key 
> because we don''t have the new per dataset wrapping key.
> 
> 	3$ zfs key -l tank/foo
> 	Enter new passphrase:
> 	Re-Enter passphrase:
> 	4$ zfs mount tank/foo
> 
> Here ''zfs key -l'' "defines" the dataset key and
does an unwrap using the
> pool key (which had to have been present or step 1 would have failed) 
> and a (re)wrap using the newly defined dataset key.
> 
> [ Note that there is a per dataset property for choosing what type of 
> key we are using - passphrase or file in phase 1 with pkcs11 coming in 
> 1.1, I''m just using passphrase for the examples in this discussion
].
> 
> 
> That could be a problem since step 2 without step 3 leaves the dataset 
> in a possibly unusable configuration.  However we do know that the per 
> dataset key hasn''t actually be loaded yet (because the hidden per 
> dataset property key_check would be undefined).  The big concern I have 
> with this is how it looks to the user/admin when looking at the 
> properties and the steps they have to take. After step 2 but before step 
> 3 they would likely see:
> 
> 	encryption=on
> 	keytype=dataset
> 	kek=passphrase
> 	keystatus=undefined
> 
> Example 2:
> 
> A similar case with the same issue is this one:
> 
> 	1$ zfs create -o encryption=on -o keytype=dataset tank/foo
> 	2$ zfs key -l tank/foo
> 	Enter new passphrase:
> 	Re-Enter passphrase:
> 	3$ zfs mount tank/foo
> 
> 
> Some discussion that Tony and I had recently I think we came to the 
> conclusion that what it might be better done like this:
> 
> Example 3:
> 
> 	1$ zfs create -o encryption=on tank/foo
> 
> At this point we have a randomly generated encryption key wrapped by the
> pool key.
> 
> 	2$ zfs key -c -o keytype=dataset tank/foo
> 	Enter new passphrase:
> 	Re-Enter passphrase:
> 	3$ zfs mount tank/foo
> 
> We change the property, define the new per dataset wrapping key and 
> re-wrap the dataset key and store it on disk using the new per dataset 
> wrapping key, all as part of that single action.
> 
> Note here that the passphrase is the new one for the dataset not the 
> pool passphrase.
> 
> This makes keytype a "strange" property because it means it can
only be
> changed by ''zfs key'' and can''t be set at create
time to anything except
> ''pool'' (and then only if the pool wrapping key is
present) - which could
> be an issue for property inheritance.
> 
> 
> Does anyone have an alternate proposal ? [ please don''t nitpick on
the
> property names at this time what I want discussed is the semantics not 
> the syntax ].
>

Cynthia McGuire wrote:
> I''m not quite following all the cases here, but does it help to
collapse
> the ''zfs create'' and ''zfs key -l'' steps
into one?
That would mean that ''zfs create'' could potential prompt, this
was
something I explicitly wanted to avoid.

In the case where the key isn''t, derived from, a passphrase then there 
wouldn''t be any prompting since it would just require that the key file
be present.

Also for the case where keytype=pool, ie we are using the pool wrapping 
key there is no need to ever run ''zfs key -l'' and all this is
a non
issue since ''zfs create'' works exactly as it does today
without crypto.

It is only the case where we are explicitly providing the user with the 
ability to manage a per dataset (wrapping) key that has the issue.

-- 
Darren J Moffat

Darren J Moffat wrote:
> Cynthia McGuire wrote:
>> I''m not quite following all the cases here, but does it help
to
>> collapse the ''zfs create'' and ''zfs key
-l'' steps into one?
> 
> That would mean that ''zfs create'' could potential prompt,
this was
> something I explicitly wanted to avoid.
I''m suggesting a new subcommand that does the encryption setup, create 
and mount in-lieu of ''zfs create''.  Something like:

zfs create-crypto -o keytype=dataset tank/foo

This way, the administrative experience and function remains the same 
for zfs create and we obviate the need for the extra ''zfs key
-l'' for
such cases that require it.

I''m just wondering if that option would help you with some of the 
ordering problems and whether it was considered as part of the original 
design conversations with the ZFS team.
> 
> In the case where the key isn''t, derived from, a passphrase then
there
> wouldn''t be any prompting since it would just require that the key
file
> be present.
> 
> Also for the case where keytype=pool, ie we are using the pool wrapping 
> key there is no need to ever run ''zfs key -l'' and all
this is a non
> issue since ''zfs create'' works exactly as it does today
without crypto.
> 
> It is only the case where we are explicitly providing the user with the 
> ability to manage a per dataset (wrapping) key that has the issue.
> 
Understood.

Cindi

On Fri, Jan 11, 2008 at 12:08:39PM +0000, Darren J Moffat
wrote:
> Anthony Scarpino wrote (elsewhere):
> > While writing up the man page.. I thought of a few things that I was 
> > wondering if you considered..
> > 
> > Can an encrypted dataset (keytype=dataset) reside in a non-encrypted
(no
> > kek defined) pool?
In terms of usability, I think that''d be real nice -- no need to
rebuild
existing pools!
> I can see a case for and against allowing this when considering it 
> purely at the feature level as users/admins see things.
> 
> The admin can control wither users can enable encryption or not by 
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...
> delegating (or not) the "encryption" and "keytype"
properties. So there
> is no need to use the pool wrapping key (ie require the kek to be 
> available) to restrict creation of encrytped datasets.
ZFS crypto does work for threat models where the adversary has access to
the keys, or to the cleartext, as the admin surely can have (once the
users provide the keys, unless tokens are used, in which case the admins
can access any cleartext that the users access).

So why wouldn''t the admin want to allow users to encrypt their data?
Because of resource consumption?  But user processes be charged for
whatever cycles they use for ZFS crypto?  (I guess additional memory use
is another story.)
> We also worked out that this could be tricky for the cli, because it 
> seems to imply a three step process that isn''t atomic - and this I
think
> is a bad model.
> 
> Example 1:
> 
> 	1$ zfs create -o encryption=on tank/foo
> 
> At this point we have a randomly generated encryption key wrapped by the
> pool key.
But if you''d allow use without pool keys then what?

I think you''d have to: a) insist on keytype=dataset at dataset creation
time, b) acquire the wrap key at dataset creation time.

(b) means that zfs create -o encryption=on could be interactive.  I
think that''s OK, but see below.
> Does anyone have an alternate proposal ? [ please don''t nitpick on
the
> property names at this time what I want discussed is the semantics not 
> the syntax ].
Let zfs create be interactive.

Alternatively, let zfs create -o encryption=on create "larval"
datasets
that aren''t actually created until keyed, but which do persist on disk
until destroyed, and can be listed, even renamed, but not snapshotted
nor mounted.

Nico
--

Darren J Moffat wrote:
> Anthony Scarpino wrote (elsewhere):
>> While writing up the man page.. I thought of a few things that I was 
>> wondering if you considered..
>>
>> Can an encrypted dataset (keytype=dataset) reside in a non-encrypted
(no
>> kek defined) pool?
> 
> I can see a case for and against allowing this when considering it 
> purely at the feature level as users/admins see things.
> 
> The admin can control wither users can enable encryption or not by 
> delegating (or not) the "encryption" and "keytype"
properties. So there
> is no need to use the pool wrapping key (ie require the kek to be 
> available) to restrict creation of encrytped datasets.
> 
> If the admin doesn''t want encryption in any dataset delegated to a
user
> then don''t delegate those properties and ensure they are inherited
as
> encryption=off.
>
Depending on 6647661 (setting pool level dataset properties at 
creation), delegation may be the only control as encryption can''t be
set
post-creation.
> 
> We need to be able to provide key change for keytype=dataset as well. 
> That means that the actual encryption key for the dataset needs to be
> done the same way as keytype=pool, ie it is a wrapped key that is stored 
> in the "wrappedkey" property not the actual encryption key.  
Regardless
> of the need for key change the encryption key needs to be generated 
> while the dataset is being created; and for the filesystem case it needs 
> to happen before the root directory of the filesystem is created - 
> leaving it unmounted isn''t actually enough.
> 
> So this means that we have an ordering problem for the initial key 
> setup. We need the per dataset wrapping key to wrap the new randomly 
> generated dataset encryption key while we are creating the dataset. That 
> assumes that ''zfs key -l'' has been run, but we
can''t do that because the
> dataset doesn''t exist yet.
> 
> We also worked out that this could be tricky for the cli, because it 
> seems to imply a three step process that isn''t atomic - and this I
think
> is a bad model.
> 
Why can''t we just hold off dataset key generation for keytype=dataset 
until the ''zfs key -l" command?  We will know from the kek_check
that
it''s unset and the zfs command will know from the keytype to not do the
keygen/wrapping operations yet like it would do for keytype=pool?

> Example 1:
> 
> 	1$ zfs create -o encryption=on tank/foo
> 
> At this point we have a randomly generated encryption key wrapped by the
> pool key.
> 
> 	2$ zfs set keytype=dataset tank/foo
> 
> The property gets changed but we can''t re-wrap the encryption key 
> because we don''t have the new per dataset wrapping key.
> 
> 	3$ zfs key -l tank/foo
> 	Enter new passphrase:
> 	Re-Enter passphrase:
> 	4$ zfs mount tank/foo
> 
> Here ''zfs key -l'' "defines" the dataset key and
does an unwrap using the
> pool key (which had to have been present or step 1 would have failed) 
> and a (re)wrap using the newly defined dataset key.
> 
> [ Note that there is a per dataset property for choosing what type of 
> key we are using - passphrase or file in phase 1 with pkcs11 coming in 
> 1.1, I''m just using passphrase for the examples in this discussion
].
> 
> 
> That could be a problem since step 2 without step 3 leaves the dataset 
> in a possibly unusable configuration.  However we do know that the per 
> dataset key hasn''t actually be loaded yet (because the hidden per 
> dataset property key_check would be undefined).  The big concern I have 
> with this is how it looks to the user/admin when looking at the 
> properties and the steps they have to take. After step 2 but before step 
> 3 they would likely see:
> 
> 	encryption=on
> 	keytype=dataset
> 	kek=passphrase
> 	keystatus=undefined
> 
> Example 2:
> 
> A similar case with the same issue is this one:
> 
> 	1$ zfs create -o encryption=on -o keytype=dataset tank/foo
> 	2$ zfs key -l tank/foo
> 	Enter new passphrase:
> 	Re-Enter passphrase:
> 	3$ zfs mount tank/foo
> 
> 
> Some discussion that Tony and I had recently I think we came to the 
> conclusion that what it might be better done like this:
> 
> Example 3:
> 
> 	1$ zfs create -o encryption=on tank/foo
> 
> At this point we have a randomly generated encryption key wrapped by the
> pool key.
> 
> 	2$ zfs key -c -o keytype=dataset tank/foo
> 	Enter new passphrase:
> 	Re-Enter passphrase:
> 	3$ zfs mount tank/foo
> 
> We change the property, define the new per dataset wrapping key and 
> re-wrap the dataset key and store it on disk using the new per dataset 
> wrapping key, all as part of that single action.
> 
> Note here that the passphrase is the new one for the dataset not the 
> pool passphrase.
> 
> This makes keytype a "strange" property because it means it can
only be
> changed by ''zfs key'' and can''t be set at create
time to anything except
> ''pool'' (and then only if the pool wrapping key is
present) - which could
> be an issue for property inheritance.
> 
> 
> Does anyone have an alternate proposal ? [ please don''t nitpick on
the
> property names at this time what I want discussed is the semantics not 
> the syntax ].
>

On Fri, Jan 11, 2008 at 09:55:26AM -0800, Anthony Scarpino
wrote:
> Why can''t we just hold off dataset key generation for
keytype=dataset
> until the ''zfs key -l" command?  We will know from the
kek_check that
> it''s unset and the zfs command will know from the keytype to not
do the
> keygen/wrapping operations yet like it would do for keytype=pool?
That means holding off on creation of the dataset.  Larval datasets.  I
think that''s OK (and I just suggested that a couple of minutes ago).

Nico
--

Nicolas Williams wrote:
> On Fri, Jan 11, 2008 at 12:08:39PM +0000, Darren J Moffat wrote:
>> Anthony Scarpino wrote (elsewhere):
>>> While writing up the man page.. I thought of a few things that I
was
>>> wondering if you considered..
>>>
>>> Can an encrypted dataset (keytype=dataset) reside in a
non-encrypted (no
>>> kek defined) pool?
> 
> In terms of usability, I think that''d be real nice -- no need to
rebuild
> existing pools!
You wouldn''t need to rebuild just have the pool admin run:

	zpool set kek=....  tank
	zpool key -l tank

All existing datasets remain untouched.
>> I can see a case for and against allowing this when considering it 
>> purely at the feature level as users/admins see things.
>>
>> The admin can control wither users can enable encryption or not by 
>   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^...
>> delegating (or not) the "encryption" and "keytype"
properties. So there
>> is no need to use the pool wrapping key (ie require the kek to be 
>> available) to restrict creation of encrytped datasets.
> 
> ZFS crypto does work for threat models where the adversary has access to
> the keys, or to the cleartext, as the admin surely can have (once the
> users provide the keys, unless tokens are used, in which case the admins
> can access any cleartext that the users access).
> 
> So why wouldn''t the admin want to allow users to encrypt their
data?
> Because of resource consumption?  But user processes be charged for
> whatever cycles they use for ZFS crypto?  (I guess additional memory use
> is another story.)
That is one reason, another might be site policy (for what ever reason) 
or because they are using crypto below ZFS using some other
"hardware".
>> We also worked out that this could be tricky for the cli, because it 
>> seems to imply a three step process that isn''t atomic - and
this I think
>> is a bad model.
>>
>> Example 1:
>>
>> 	1$ zfs create -o encryption=on tank/foo
>>
>> At this point we have a randomly generated encryption key wrapped by
the
>> pool key.
> 
> But if you''d allow use without pool keys then what?
> 
> I think you''d have to: a) insist on keytype=dataset at dataset
creation
> time, b) acquire the wrap key at dataset creation time.
> 
> (b) means that zfs create -o encryption=on could be interactive.  I
> think that''s OK, but see below.
I was explicitly trying to avoid making any existing command go interactive.
>> Does anyone have an alternate proposal ? [ please don''t
nitpick on the
>> property names at this time what I want discussed is the semantics not 
>> the syntax ].
> 
> Let zfs create be interactive.
> 
> Alternatively, let zfs create -o encryption=on create "larval"
datasets
> that aren''t actually created until keyed, but which do persist on
disk
> until destroyed, and can be listed, even renamed, but not snapshotted
> nor mounted.
I''m looking at that just now but it isn''t as trivial as it
might appear.

-- 
Darren J Moffat

Anthony Scarpino wrote:
> Darren J Moffat wrote:
>> Anthony Scarpino wrote (elsewhere):
>>> While writing up the man page.. I thought of a few things that I
was
>>> wondering if you considered..
>>>
>>> Can an encrypted dataset (keytype=dataset) reside in a
non-encrypted
>>> (no kek defined) pool?
>>
>> I can see a case for and against allowing this when considering it 
>> purely at the feature level as users/admins see things.
>>
>> The admin can control wither users can enable encryption or not by 
>> delegating (or not) the "encryption" and "keytype"
properties. So
>> there is no need to use the pool wrapping key (ie require the kek to 
>> be available) to restrict creation of encrytped datasets.
>>
>> If the admin doesn''t want encryption in any dataset delegated
to a
>> user then don''t delegate those properties and ensure they are 
>> inherited as encryption=off.
>>
> 
> Depending on 6647661 (setting pool level dataset properties at 
> creation), delegation may be the only control as encryption can''t
be set
> post-creation.
I don''t see the connection here.  encryption defaults to off.  6647661 
means that the top level dataset always has encryption=off but has no 
impact on what happens to any other dataset, nor does it have any impact 
on setting the delegation of encryption at the top level, ie even with 
6647661 this works:

	# zpool create tank c0d0
	# zfs allow everyone encryption tank

	darren$ zfs create encryption=on tank/darren
>> We need to be able to provide key change for keytype=dataset as well. 
>> That means that the actual encryption key for the dataset needs to be
>> done the same way as keytype=pool, ie it is a wrapped key that is 
>> stored in the "wrappedkey" property not the actual encryption
key.
>> Regardless of the need for key change the encryption key needs to be 
>> generated while the dataset is being created; and for the filesystem 
>> case it needs to happen before the root directory of the filesystem is 
>> created - leaving it unmounted isn''t actually enough.
>>
>> So this means that we have an ordering problem for the initial key 
>> setup. We need the per dataset wrapping key to wrap the new randomly 
>> generated dataset encryption key while we are creating the dataset. 
>> That assumes that ''zfs key -l'' has been run, but we
can''t do that
>> because the dataset doesn''t exist yet.
>>
>> We also worked out that this could be tricky for the cli, because it 
>> seems to imply a three step process that isn''t atomic - and
this I
>> think is a bad model.
>>
> 
> Why can''t we just hold off dataset key generation for
keytype=dataset
> until the ''zfs key -l" command? 
Because we need the key to encrypt things that are written to disk 
during the dataset creation - specifically the root directory of the 
dataset.

-- 
Darren J Moffat

On Fri, Jan 11, 2008 at 06:05:48PM +0000, Darren J Moffat
wrote:
> >>>Can an encrypted dataset (keytype=dataset) reside in a
non-encrypted (no
> >>>kek defined) pool?
> >
> >In terms of usability, I think that''d be real nice -- no need
to rebuild
> >existing pools!
> 
> You wouldn''t need to rebuild just have the pool admin run:
Oh good.
> >So why wouldn''t the admin want to allow users to encrypt their
data?
> >Because of resource consumption?  But user processes be charged for
> >whatever cycles they use for ZFS crypto?  (I guess additional memory
use
> >is another story.)
> 
> That is one reason, another might be site policy (for what ever reason) 
> or because they are using crypto below ZFS using some other
"hardware".
*shrug*

Resource usage is the only really good reason for restricting ZFS crypto
usage that I can come up with.
> >(b) means that zfs create -o encryption=on could be interactive.  I
> >think that''s OK, but see below.
> 
> I was explicitly trying to avoid making any existing command go
interactive.
As long as it only happens with new options I don''t see why not.  What
sort of principle applies that couldn''t be excepted?
> >Alternatively, let zfs create -o encryption=on create
"larval" datasets
> >that aren''t actually created until keyed, but which do persist
on disk
> >until destroyed, and can be listed, even renamed, but not snapshotted
> >nor mounted.
> 
> I''m looking at that just now but it isn''t as trivial as
it might appear.
Oh, I imagine it isn''t.  It might be, but probably isn''t.

Darren J Moffat wrote:
> Anthony Scarpino wrote:
>> Darren J Moffat wrote:
>>> Anthony Scarpino wrote (elsewhere):
>>>> While writing up the man page.. I thought of a few things that
I was
>>>> wondering if you considered..
>>>>
>>>> Can an encrypted dataset (keytype=dataset) reside in a
non-encrypted
>>>> (no kek defined) pool?
>>> I can see a case for and against allowing this when considering it 
>>> purely at the feature level as users/admins see things.
>>>
>>> The admin can control wither users can enable encryption or not by 
>>> delegating (or not) the "encryption" and
"keytype" properties. So
>>> there is no need to use the pool wrapping key (ie require the kek
to
>>> be available) to restrict creation of encrytped datasets.
>>>
>>> If the admin doesn''t want encryption in any dataset
delegated to a
>>> user then don''t delegate those properties and ensure they
are
>>> inherited as encryption=off.
>>>
>> Depending on 6647661 (setting pool level dataset properties at 
>> creation), delegation may be the only control as encryption
can''t be set
>> post-creation.
> 
> I don''t see the connection here.  encryption defaults to off. 
6647661
> means that the top level dataset always has encryption=off but has no 
> impact on what happens to any other dataset, nor does it have any impact 
> on setting the delegation of encryption at the top level, ie even with 
> 6647661 this works:
> 
> 	# zpool create tank c0d0
> 	# zfs allow everyone encryption tank
> 
> 	darren$ zfs create encryption=on tank/darren
Yes, yes.. you''re right.. I misread the delegation the first time..
> 
>>> We need to be able to provide key change for keytype=dataset as
well.
>>> That means that the actual encryption key for the dataset needs to
be
>>> done the same way as keytype=pool, ie it is a wrapped key that is 
>>> stored in the "wrappedkey" property not the actual
encryption key.
>>> Regardless of the need for key change the encryption key needs to
be
>>> generated while the dataset is being created; and for the
filesystem
>>> case it needs to happen before the root directory of the filesystem
is
>>> created - leaving it unmounted isn''t actually enough.
>>>
>>> So this means that we have an ordering problem for the initial key 
>>> setup. We need the per dataset wrapping key to wrap the new
randomly
>>> generated dataset encryption key while we are creating the dataset.
>>> That assumes that ''zfs key -l'' has been run, but
we can''t do that
>>> because the dataset doesn''t exist yet.
>>>
>>> We also worked out that this could be tricky for the cli, because
it
>>> seems to imply a three step process that isn''t atomic -
and this I
>>> think is a bad model.
>>>
>> Why can''t we just hold off dataset key generation for
keytype=dataset
>> until the ''zfs key -l" command? 
> 
> Because we need the key to encrypt things that are written to disk 
> during the dataset creation - specifically the root directory of the 
> dataset.
> 
So given option 3, you would either add a ''zfs key -l" before
"zfs key
-c" to set the key first, or "zfs key -c" has to learn it can
also do an
initial set.. hmm

Nicolas Williams wrote:
>>> (b) means that zfs create -o encryption=on could be interactive.  I
>>> think that''s OK, but see below.
>> I was explicitly trying to avoid making any existing command go
interactive.
> 
> As long as it only happens with new options I don''t see why not. 
What
> sort of principle applies that couldn''t be excepted?
There is no new option, just a new key=value to an existing command, 
verb and option.

The big issue I have here is I really really don''t want zfs create to 
change behaviour from what it does now and for it to differ between the 
different key management styles (either the two (pool & dataset) that we 
are delivering for phase 1 or any future ones).

However don''t just think about the CLI but also how this would work in
a
GUI (web or local) or from some other application calling directly into 
libzfs that isn''t the current CLI.

-- 
Darren J Moffat

On Mon, Jan 14, 2008 at 10:10:19AM +0000, Darren J Moffat
wrote:
> Nicolas Williams wrote:
> >>>(b) means that zfs create -o encryption=on could be
interactive.  I
> >>>think that''s OK, but see below.
> >>I was explicitly trying to avoid making any existing command go 
> >>interactive.
> >
> >As long as it only happens with new options I don''t see why
not.  What
> >sort of principle applies that couldn''t be excepted?
> 
> There is no new option, just a new key=value to an existing command, 
> verb and option.
If need be you could add a new option (-i for interactive).
> The big issue I have here is I really really don''t want zfs create
to
> change behaviour from what it does now and for it to differ between the 
> different key management styles (either the two (pool & dataset) that
we
> are delivering for phase 1 or any future ones).
Which means you either have to:

a) create the keys beforehand
b) create larval datasets that come to life when the keys are created
c) do both at create time (zfs create becomes interactive)

The keys in (a) have to relate to yet-to-be-created datasets somehow; if
by dataset name then (a) and (b) are very similar.

Are there other possibilities?  How about:

d) re-write the entire dataset when keys are furnished
> However don''t just think about the CLI but also how this would
work in a
> GUI (web or local) or from some other application calling directly into 
> libzfs that isn''t the current CLI.
What about them?  GUIs will have more freedom here and fewer constraints
(though the constraint that zfs create never being interactive strikes
me as artificial).

As a user my preference runs from (c) to (b) to (a).  If (c) I''d say do
an option so scripts can''t trip over it by accident.

Nico
--

Thanks to some white borad time with Alec Muffett I think I might have a 
solution that a) doesn''t require the pool key for initial dataset 
creation and b) doesn''t require but allows for zfs(1) to be
interactive.

See the attached which includes examples of usage for initial dataset 
creation and key change.

I believe that, while not listed in the document, this helps interfaces 
that would be calling libzfs directly (rather than scripting around 
zfs(1)) as well.

--
Darren J Moffat
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: keymgmt-cli.txt
URL:
<http://mail.opensolaris.org/pipermail/zfs-crypto-discuss/attachments/20080116/569850dc/attachment.txt>

On Wed, 2008-01-16 at 11:56 +0000, Darren J Moffat
wrote:
> wrappedkey=<private binary format>
> 
> 	Actual key (length determined by encryption property)
> 	wrapped using AES_CBC_PAD using pool guid as the IV.
this looks like poor crypto hygiene.

this seems to imply multiple wrappedkeys (different dataset, same pool)
will be encrypted using the same IV.  the point of an IV is that is
different for every encryption -- if I repeatedly encrypt the same
message over and over, an attacker can''t tell I''m sending the
same
message because the IV is always changing and thus the ciphertext will
on average have at least half of its bits different from any other
message of the same size.

Hi - I''m new to ZFS but not to Solaris.

Is there a search able interface to the zfs-discuss mail archives?

We have a Windows 2003 Cluster with 200 TB SAN running under
Active Directory with file system compression.

Half the population is running Linux and the other half is running
Windows XP.

I''m interested in replacing the Window 2003 Cluster and filesystem
compression with Solaris 10 and ZFS compression.

Pardon my ignorance, but is ZFS with compression safe to use in a
production environment?

Any help would be greatly appreciated.

-- Ken

--

        "We are drowning in information and starving for knowledge."

                                                                              
- Rutherford D. Roger

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=GB2312"
http-equiv="Content-Type">
  <title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Darren J Moffat ÒÑдÈë:
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">Thanks to
some white borad time with Alec Muffett I think I might have a solution
that a) doesn''t require the pool key for initial dataset creation and
b) doesn''t require but allows for zfs(1) to be interactive.
  <br>
</blockquote>
Does this solution mean without the pool/dataset key, datasets can be
created? But only the pool or dataset key is loaded, datasets can be
mounted.<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite"><br>
See the attached which includes examples of usage for initial dataset
creation and key change.
  <br>
  <br>
I believe that, while not listed in the document, this helps interfaces
that would be calling libzfs directly (rather than scripting around
zfs(1)) as well.
  <br>
  <br>
--
  <br>
Darren J Moffat
  <br>
  <pre wrap="">
<hr size="4" width="90%">
Visible dataset properties:
==========================
encryption="on | off | aes-128-ccm | aes-256-ccm"

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspWhich
algorithm and mode we use.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspdefault:
off
  </pre>
</blockquote>
Which algorithm does ''on'' represent? And why?<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">
keyscope="dataset | pool"
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspWhich
"key" property to look at.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspdefault:
no default value unless encryption != off, then
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp
default is "pool".

key=<format>,<locator> | -

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspOnly
valid when keyscope=dataset, otherwise look at
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsppool
level "key" property.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspSee
below for details.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspThis
is not the actual key used for encryption but
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspinformation
on how to get they key used to wrap
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspthe
actual encryption key.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspThe
actual data encryption keys are randomly generated
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspand
always stored in wrapped form.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspThis
is always an AES 256 bit key at this time, but
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspcould
be something else in the future, such as an
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspRSA
private key + cert.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspdefault:
no default value, must be explicitly set, if
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp
keyscope=dataset.
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp
keystatus="available | unavailable"

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspreadonly
status indicator.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspdefault:
no default value unless encryption != off
  </pre>
</blockquote>
''keystatus'' is the wrapping key status. If keyscope=pool, the
''keystatus'' value of a dataset equals to the
''keystatus'' value of a
pool, right?<br>
Why the "undefined" value is omitted? If a dataset has encryption=off,
what''s the ''keystatus'' value? It will be
''-''?<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">

Hidden dataset properties:
=========================
wrappedkey=<private binary format>

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspActual
key (length determined by encryption property)
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspwrapped
using AES_CBC_PAD using pool guid as the IV.


Visible pool properties:
======================
key=<format>,<locator>
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspAs
above

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspUsed
only for datasets with keyscope=pool

keystatus&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspAs
above
  </pre>
</blockquote>
If the ''key'' isn''t defined for a pool,
what''s the ''keystatus'' value?<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">
Hidden pool properties:
======================
key_check
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspPool
guid encrypted with AES_CBC_PAD, this is used
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspas
a verify when loading the key for keyscope=pool
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspwith
''zpool key -l <pool>''.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspThis
is needed since the key could be defined even
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspwhen
no encrypted datasets exist.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspNeed
to ensure only the correct key is loaded and
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsprequired
for key change to work.


"key" property detail
====================
<format>: raw | hex | passphrase | token

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspraw: A
256 bit AES key in either binary.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsphex: As
raw but encoded in hex - so it is printable and easily stored.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsppassphrase:

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspUpto
256 char long passphrase.
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspPassed
through PKCS#5 PBE, salt is pool/dataset guid.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspNeed
to get some sort of quality check, pam_authtok_check
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspdoes
what we need, so consider in pam.conf

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspzpool
password required pam_authtok_check.so.1
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspzfs
password required pam_authtok_check.so.1

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsptoken:
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspKey
is stored in a token, currently the only token
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspURI
support is pkcs11.


<locator>:
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspprompt |
URI

prompt:
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspFor the
ZFS CLI this means that stdin will be opened
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspwith
echo off to enter the information.

URI:
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp<a
class="moz-txt-link-freetext"
href="file://">file://</a>
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspNote
that file could be a pipe or /dev/fd/0

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp<a
class="moz-txt-link-freetext"
href="https://">https://</a>
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspNot
for phase 1.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspWe
could use libcurl for this but ideally KMF would
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspdeal
with this for us, particular for the <a
class="moz-txt-link-freetext"
href="https://">https://</a> case
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspsince
we will need trust anchors for the TLS connection.

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsppkcs11:

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspThis is
a PKCS#11 URI (TBD by Cryptoki WG)

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsppkcs11://token="My
Token";label="My Key";getpin=prompt
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsppkcs11://token="My
Token";label="My Key";getpin=<a
class="moz-txt-link-freetext"
href="file://path/to/pin">file://path/to/pin</a>

&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspFull
details subject to external document.  Note that if getpin
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspis not
specified prompt is assumed.
  </pre>
</blockquote>
Will phase 1 support pkcs11 URI?<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">
Full examples:
=============
Example 1:
----------
Encrypted dataset with key scope of pool. Pool key in file:

# zpool set key=raw,<a class="moz-txt-link-freetext"
href="file:///rmdisk/keyring1/tank.key">file:///rmdisk/keyring1/tank.key</a>
tank

# zpool key -l tank

$ zfs create -o encryption=on tank/home/bob/Documents
  </pre>
</blockquote>
Can the encrypted dataset be created without ''zpool set key''?
I think
no.<br>
Can the encrypted dataset be created without ''zpool key -l''? I
think
yes, but it remains unmounted.<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">$

Example 2:
----------
Encrypted dataset with key scope of pool, prompt for passphrase:

# zpool set key=passphrase,prompt tank

# zpool key -l tank
Enter passphrase: ********
Re-enter passphrase: ********
#

[ The Re-enter only happens the first time a passphrase is entered ]

...
$ zfs create -o encryption=on tank/home/bob/Documents

Example 3:
----------
Changing the pool level key file:

The old key has to have been loaded before a key change can happen.

# zpool set key=raw,<a class="moz-txt-link-freetext"
href="file:///rmdisk/keyring1/200712">file:///rmdisk/keyring1/200712</a>
tank
...
# zpool key -l tank
...
# zpool key -c -o key=raw,<a class="moz-txt-link-freetext"
href="file:///rmdisk/keyring2/200801">file:///rmdisk/keyring2/200801</a>
tank
#

Example 4:
----------
Change the pool level passphrase:

# zpool key -c tank
Enter existing passphrase: ********
Enter new passphrase: ********
Re-enter new passphrase: ********
#

Example 5:
----------
Key scope of dataset with raw key, generated by pktool, in a file:

$ pktool genkey keystore=file outfile=mykey outdir=/rmdisk/stick-1 \
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspkeytype=aes
keylen=256

$ zfs create -o keyscope=dataset \
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp-o
key=raw,<a class="moz-txt-link-freetext"
href="file:///rmdisk/stick-1/mykey">file:///rmdisk/stick-1/mykey</a>
tank/home/bob
$ zfs key -l tank/home/bob
$&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp

Example 6:
----------
Hex version of key, generated by pktool with print=y, in file:

$ pktool genkey keystore=file outfile=mykey outdir=/rmdisk/stick-1 \
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspkeytype=aes
keylen=256
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspKey
Value ="5321de1dadc926b05eaf5f793686346b"

$ echo "5321de1dadc926b05eaf5f793686346b\c" > /rmdisk/stick-1/mykey

$ zfs create -o keyscope=dataset \
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp-o
key=hex,<a class="moz-txt-link-freetext"
href="file:///rmdisk/stick-1/mykey">file:///rmdisk/stick-1/mykey</a>
tank/home/bob
$ zfs key -l tank/home/bob
$&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp

Example 7:
----------
Hex version of key, generated by pktool with print=y, prompted:

$ pktool genkey outfile=/dev/null keytype=aes keylen=256 print=y
$ zfs create -o keyscope=dataset -o key=hex,prompt tank/home/bob
Enter key: ********************************
$

Example 8:
----------
Key is a per dataset passphrase that we prompt for:

$ zfs create -o keyscope=dataset -o key=passphrase,prompt tank/home/bob
Enter passphrase: 
Re-enter passphrase:
$
  </pre>
</blockquote>
For the Example 7 and 8, is ''zfs key -l'' required to mount the
dataset?
If no, why?<br>
Does this mean ''zfs key -l'' is only required when
''locator!=prompt'' and
if ''locator=prompt'', ''zfs create'' will mount
the dataset automatically
after creation?<br>
<br>
Thanks,<br>
- Grace<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">
After reboot (or pool export/import) reload the key

$ zfs key -l tank/home/bob
Enter passphrase:
$ cd /tank/home/bob
$

Example 9:
----------
Changing the passphrase for keyscope dataset:
[If the dataset was mounted it will remain mounted during the change]

$ zfs key -c tank/home/bob
Changing key for tank/home/bob
Enter existing passphrase:
Enter new passphrase:
Re-Enter new passphrase:
$

Example 10:
-----------
Key is a per dataset passphrase but it is stored in a file:

$ zfs create -o key=passphrase,<a class="moz-txt-link-freetext"
href="file:///rmdisk/stick-1/mypass">file:///rmdisk/stick-1/mypass</a>
\
    tank/home/bob
$ zfs key -l tank/home/bob
$&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp

Example 11:
-----------
Key is stored in a pkcs11 token as a private object (ie the PIN
is requried to view/use the key object).

# First create the key with pktool

$ pktool genkey keystore=pkcs11 token="Bob''s Card" label=ek1
\
&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbsp&nbspkeytype=aes
keylen=256
Enter PIN for Bob''s Card: *****
$
$ zfs create -o keyscope=dataset \
    -o key=token,pkcs11://token="Bob''s
Card";label=ek1;getpin=prompt\
    tank/home/bob
  </pre>
</blockquote>
Why PIN isn''t needed in this step?<br>
<blockquote cite="mid478DF0E4.4000708@Sun.COM"
type="cite">
  <pre wrap="">$ zfs key -l tank/home/bob
Enter PIN for Bob''s Card: ********
$

Example 12:
-----------
Key is stored in a pkcs11 token as a public object (ie the PIN
is NOT requried to view/use the key object).

In this example keyscope is inherited from a parent dataset.

# zfs set keyscope=dataset tank/home

$ zfs create -o key=token,pkcs11://token="Bob''s
Card";label=ek1 tank/home/bob
$ zfs key -l tank/home/bob
$

END OF EXAMPLES
  </pre>
  <pre wrap="">
<hr size="4" width="90%">
_______________________________________________
zfs-crypto-discuss mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:zfs-crypto-discuss@opensolaris.org">zfs-crypto-discuss@opensolaris.org</a>
<a class="moz-txt-link-freetext"
href="http://mail.opensolaris.org/mailman/listinfo/zfs-crypto-discuss">http://mail.opensolaris.org/mailman/listinfo/zfs-crypto-discuss</a>
  </pre>
</blockquote>
<br>
</body>
</html>

Hello Agile,

Comments in-between


Thursday, January 17, 2008, 2:20:42 AM, you wrote:

AA> Hi - I''m new to ZFS but not to Solaris.

AA> Is there a search able interface to the zfs-discuss mail archives?


http://opensolaris.org/os/discussions/
and look for zfs-discuss list.

AA> We have a Windows 2003 Cluster with 200 TB SAN running under
AA> Active Directory with file system compression.

AA> Half the population is running Linux and the other half is running
AA> Windows XP.

AA> I''m interested in replacing the Window 2003 Cluster and
filesystem
AA> compression with Solaris 10 and ZFS compression.

AA> Pardon my ignorance, but is ZFS with compression safe to use in a
AA> production environment?

Yes, it is.

Keep in mind that if you go for Solaris 10 the only compression
supported right now is lzjb. Open Solaris additionally suppports gzip.


-- 
Best regards,
 Robert Milkowski                           mailto:milek at task.gda.pl
                                       http://milek.blogspot.com

Bill Sommerfeld wrote:
> On Wed, 2008-01-16 at 11:56 +0000, Darren J Moffat wrote:
>> wrappedkey=<private binary format>
>>
>> 	Actual key (length determined by encryption property)
>> 	wrapped using AES_CBC_PAD using pool guid as the IV.
> 
> this looks like poor crypto hygiene.
> 
> this seems to imply multiple wrappedkeys (different dataset, same pool)
> will be encrypted using the same IV.  the point of an IV is that is
> different for every encryption -- if I repeatedly encrypt the same
> message over and over, an attacker can''t tell I''m sending
the same
> message because the IV is always changing and thus the ciphertext will
> on average have at least half of its bits different from any other
> message of the same size.
Sorry that is a typo I meant the dataset guid as the IV not the pool 
guid - for exactly that reason.

-- 
Darren J Moffat

Grace Tang wrote:
> Darren J Moffat ???:
>> Thanks to some white borad time with Alec Muffett I think I might have 
>> a solution that a) doesn''t require the pool key for initial
dataset
>> creation and b) doesn''t require but allows for zfs(1) to be
interactive.
> Does this solution mean without the pool/dataset key, datasets can be 
> created? But only the pool or dataset key is loaded, datasets can be 
> mounted.
It means that you can create a dataset with encryption using a per
dataset key even if there has never been a pool level key defined.  It
provides a way to specify the new per dataset key at create time - so
that we don''t need the pool key to do the initial key wrapping during
dataset creation.

Datasets will need the appropriate key (pool or dataset) to be loaded
for them to mount.

-- 
Darren J Moffat

Agile Aspect wrote:
> Hi - I''m new to ZFS but not to Solaris.
> 
> Is there a search able interface to the zfs-discuss mail archives?
Use google against the mailman archives:

http://mail.opensolaris.org/pipermail/zfs-discuss/
> Pardon my ignorance, but is ZFS with compression safe to use in a
> production environment?
Yes, why wouldn''t it be ?  If it wasn''t safe it
wouldn''t have been
delivered.

What kind of "unsafe" behaviour are you worried about ?  It is
possible
that other ZFS features mitigate issues you may be worried about or have 
experienced elsewhere.

-- 
Darren J Moffat

On Thu, 2008-01-17 at 10:06 +0000, Darren J Moffat
wrote:
> Sorry that is a typo I meant the dataset guid as the IV not the pool 
> guid - for exactly that reason.
That still sounds like you''ll be doing IV reuse when you rekey (I
realize rekey isn''t all there yet).

Would it be too difficult to add a key version number or equivalent?

					- Bill

Bill Sommerfeld wrote:
> On Thu, 2008-01-17 at 10:06 +0000, Darren J Moffat wrote:
>> Sorry that is a typo I meant the dataset guid as the IV not the pool 
>> guid - for exactly that reason.
> 
> That still sounds like you''ll be doing IV reuse when you rekey (I
> realize rekey isn''t all there yet).
Good point.  Rekey for phase 1 is basically unwrap and rewrap with the 
new key, so you are correct that the same IV would be getting used with 
the same cleartext and a different key.
> Would it be too difficult to add a key version number or equivalent?
I think that would be easy enough to add in as part of the IV.

-- 
Darren J Moffat

> Pardon my ignorance, but is ZFS with compression safe to use in a
> production environment?
I''d say, as safe as ZFS in general.  ZFS has been well-tested by Sun,
but it''s not as mature as UFS, say.  There is not yet a fsck equivalent
for ZFS, so if a bug results in damage to your ZFS data pool, you''ll
need to restore the whole pool from backups.  This may or may not be an issue
depending on the amount of downtime you can tolerate (and the size of your
pool).

Adding compression to the mix doesn''t really increase risk, IMO.

As always, test yourself, in your environment before deploying.  :-)

Anton
 
 
This message posted from opensolaris.org

On 1/17/08, Darren J Moffat <darrenm at opensolaris.org>
wrote:
> > Pardon my ignorance, but is ZFS with compression safe to use in a
> > production environment?
>
> Yes, why wouldn''t it be ?  If it wasn''t safe it
wouldn''t have been
> delivered.
Few reasons -
http://prefetch.net/blog/index.php/2007/11/28/is-zfs-ready-for-primetime/

-- 
_________________________________/ sengork.blogspot.com /????

Sengor wrote:
> On 1/17/08, Darren J Moffat <darrenm at opensolaris.org> wrote:
>>> Pardon my ignorance, but is ZFS with compression safe to use in a
>>> production environment?
>> Yes, why wouldn''t it be ?  If it wasn''t safe it
wouldn''t have been
>> delivered.
> 
> Few reasons -
http://prefetch.net/blog/index.php/2007/11/28/is-zfs-ready-for-primetime/
The article (not the comments) is complete free of content and is scare 
mongering.  It doesn''t even say wither this is ZFS on Solaris (vs BSD
or
MacOS X) never mind what release or the configuration of the pool or 
even what the actual "bug" apparently.  Was the pool redundant if
there
were bugs what are the bug numbers and are they fixed.

Simply FUD.

-- 
Darren J Moffat

On 1/18/08, Darren J Moffat <darrenm at opensolaris.org>
wrote:
> Simply FUD.
I don''t see many enterprises adopting ZFS even though it''s
been
officially out for a while now. Looking over the mailing list and
numbers of ZFS patches, it''s enough to scare lots of people away.

Don''t get me wrong, I believe ZFS is a great product to have come out
of Sun''s software group, however I don''t think it''s
matured enough to
be relied upon with mission crititcal systems. ZFS is changing too
fast to be considered stable in my opinion...

I still see VxSF (for those who can afford it) being the defacto choice.

-- 
_________________________________/ sengork.blogspot.com /????

On 1/18/08, Sengor <sengork at gmail.com> wrote:
> I don''t see many enterprises adopting ZFS even though
it''s been
> officially out for a while now. Looking over the mailing list and
> numbers of ZFS patches, it''s enough to scare lots of people away.
        I suspect that the amount of changes / discussion is no less
for ZFS than for any new filesystem, just that due to the open source
nature of it the discussions are in public view. The fact that the
issues *are* being discussed is a huge advantage in my mind. At least
we *know* that the issue are being recognized. I don''t know how many
times I have filed bug reports on various aspects of an OS and never
get a good response that the issue has been recognized as such.
> Don''t get me wrong, I believe ZFS is a great product to have come
out
> of Sun''s software group, however I don''t think
it''s matured enough to
> be relied upon with mission crititcal systems. ZFS is changing too
> fast to be considered stable in my opinion...
        I expect that UFS was not changing much because it had spent
so many years changing already ;-) Seriously, I was seeing serious
changes in SLVM/UFS up until about a year or two ago. I even ran into
one of the issues created by fixing another issue with SLVM about a
year ago. We are using ZFS in a couple ''production'' roles,
only one of
which is critical, and we picked ZFS because no other FS we tested
scaled the way we needed it to.
> I still see VxSF (for those who can afford it) being the defacto choice.
        VxFS did not have the performance we needed without lots of
tuning, while ZFS did fine right out of the box. We had moved away
from VxVM/VxFS years ago due to SLVM maturing and giving us the
features we really needed, SLVM was easier to manage, and OS upgrades
are *much* simpler with SLVM than with VxVM/VxFS. There was no real
justification for the cost and more difficult management of VxVM/VxFS.

        For some background, I have been using both VxVM/VxFS and
DiskSuite / SLVM since about 1996.

-- 
{--------1---------2---------3---------4---------5---------6---------7---------}
Paul Kraus
-> Sound Designer, Noel Coward''s Hay Fever
    @ Albany Civic Theatre, Feb./Mar. 2008
-> Facilities Coordinator, Albacon 2008

Sengor wrote:
> On 1/18/08, Darren J Moffat <darrenm at opensolaris.org> wrote:
>> Simply FUD.
> 
> I don''t see many enterprises adopting ZFS even though
it''s been
> officially out for a while now.
On what do you base that statement ?

How do you "see" what enterprises are adopting ?

State your sources please.

-- 
Darren J Moffat

Many enterprise customers have told us that they are waiting for
two features, not yet available in ZFS, before they will adopt
it widely in their datacenter environments:

- The ability to boot off of a ZFS partition.  I don''t actually
   understand this one, since you can''t boot Solaris from a VxFS
   file system either, but I''ve heard it many times.  The work
   to enable this is in progress.

- The ability to permanently shrink the size of a zpool by
   removing a lun after migrating all the data off of it.  This
   makes sense to me.  The work to enable this is in progress.

But, of course, many other enterprise customers _have_ adopted
ZFS, and are quite happy with it.  For a list of ZFS reference
customers please contact Solaris Marketing.  ZFS is used in many
mission critical roles today, and by and large our customers
are thrilled with it.

-- Fred

Sengor wrote:
> On 1/18/08, Darren J Moffat <darrenm at opensolaris.org> wrote:
>> Simply FUD.
> 
> I don''t see many enterprises adopting ZFS even though
it''s been
> officially out for a while now. Looking over the mailing list and
> numbers of ZFS patches, it''s enough to scare lots of people away.
> 
> Don''t get me wrong, I believe ZFS is a great product to have come
out
> of Sun''s software group, however I don''t think
it''s matured enough to
> be relied upon with mission crititcal systems. ZFS is changing too
> fast to be considered stable in my opinion...
> 
> I still see VxSF (for those who can afford it) being the defacto choice.
> 
-- 
Fred Zlotnick
Senior Director, Solaris NAS
Sun Microsystems, Inc.
fred.zlotnick at sun.com
x85006/+1 650 786 5006

Paul Kraus wrote:
> On 1/18/08, Sengor <sengork at gmail.com> wrote:
>
>   
>> Don''t get me wrong, I believe ZFS is a great product to have
come out
>> of Sun''s software group, however I don''t think
it''s matured enough to
>> be relied upon with mission crititcal systems. ZFS is changing too
>> fast to be considered stable in my opinion...
>>     
>
>         I expect that UFS was not changing much because it had spent
> so many years changing already ;-) 
In the software life cycle, there is a time where you stop modifying
code because it is so fragile and its interdependencies are so complex
that you can''t risk change.  This is where UFS is.  There are many
bugs and RFEs which will never be implemented in UFS.  When
this happens, something else inevitably comes along to replace it,
hopefully with a modern design using modern development methods.
That is where ZFS is.  For example, the test suite for ZFS is both
open and comprehensive.  We all wish that UFS had something
like it all those years ago...
    http://blogs.sun.com/bill/entry/zfs_and_the_all_singing

So there are two ways of looking at unchanging (stable?) code,
either it is truly perfect or it is at its end of life.  I have a nice
bottle of champagne for the day I find perfect code :-)
 -- richard

On Jan 18, 2008, at 4:23 AM, Sengor wrote:
> On 1/17/08, Darren J Moffat <darrenm at opensolaris.org> wrote:
>>> Pardon my ignorance, but is ZFS with compression safe to use in a
>>> production environment?
>>
>> Yes, why wouldn''t it be ?  If it wasn''t safe it
wouldn''t have been
>> delivered.
>
> Few reasons - http://prefetch.net/blog/index.php/2007/11/28/is-zfs- 
> ready-for-primetime/
The only details we have from that blog entry are bugs: 6454482 and  
6458218.

6454482 was a bug found in prototype code and is closed as non  
reproducible.

6458218 has been fixed in snv_60 (over 10 months ago).

10 months ago seems like a long time to me, but i don''t control  
admin''s upgrade schedules.

Matty, if you really did see an instance of 6454482, please let us know.

eric


>
> -- 
> _________________________________/ sengork.blogspot.com /????
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On 1/19/08, Darren J Moffat <darrenm at opensolaris.org>
wrote:
> On what do you base that statement ?
>
> How do you "see" what enterprises are adopting ?
>
> State your sources please.
Out of the many I work on only one''s been keen on adopting it any time
soon, another one''s planning to look into it but doesn''t see
many
benefits in it over VxSF (storage foundation). Main struggle for
companies is migrating to ZFS from other solutions out there, and
still keeping the functionality/processes/standards they need on
regular basis (eg. data migrations, BCVs and backup strategies, remote
replication, DR, and general uptime of their applications).

People should not only see the enterprises which have adopted ZFS,
there''s plenty of them which haven''t done so for various
good/bad
reasons. Partially due to their scale and complexity/worthiness of
deploying something new cross company wide, which might not be
necessarely a ZFS specific obstacle.

-- 
_________________________________/ sengork.blogspot.com /????

On 1/19/08, Paul Kraus <pk1048 at gmail.com>
wrote:
>         I suspect that the amount of changes / discussion is no less
> for ZFS than for any new filesystem, just that due to the open source
> nature of it the discussions are in public view. The fact that the
> issues *are* being discussed is a huge advantage in my mind. At least
> we *know* that the issue are being recognized. I don''t know how
many
> times I have filed bug reports on various aspects of an OS and never
> get a good response that the issue has been recognized as such.
I''ve always argued about Sun and their sheer amount of patches for any
software they seem to release. It''s a good/bad situation. On the
bright side they are pro active in creating and releasing patches,
even their IDR process is a good idea and very pro active of them. On
the contrary you could look at it as if their software always ends up
needing more and more patches no matter how many patches you end up
applying, you''re almost always out of date and vulnerable to
potentially critical known/unknown issues.

AIX doesn''t seem to have lots of patches out there (nowhere as many as
Solaris). One could argue that the software it self does not need to
be patched after it''s released. Then again you could argue
vendor''s
not too proactive in chasing up patches and the userbase doesn''t find
many bugs to begin with...

I guess both have to do with company''s support strategy (in one way or
another).
>         VxFS did not have the performance we needed without lots of
> tuning, while ZFS did fine right out of the box. We had moved away
> from VxVM/VxFS years ago due to SLVM maturing and giving us the
> features we really needed, SLVM was easier to manage, and OS upgrades
> are *much* simpler with SLVM than with VxVM/VxFS. There was no real
> justification for the cost and more difficult management of VxVM/VxFS.
>
>         For some background, I have been using both VxVM/VxFS and
> DiskSuite / SLVM since about 1996.
VxSF is not a typo it seems to be. I was referring to Veritas Storage
Foundation, not only VxFS. ZFS isn''t only a filesystem hence should
not be compared to only VxFS.

We avoid using VxSF for OS volumes, use SVM for those instead because
as you say it''s easier and cleaner to maintain. VxSF overcomplicates
simple environments, however it makes complex environments easier to
manage. I''d not want to manage 100s of multipathed LUNs via SVM on a
clustered system when VxSF is an option.
>         I expect that UFS was not changing much because it had spent
> so many years changing already ;-) Seriously, I was seeing serious
> changes in SLVM/UFS up until about a year or two ago. I even ran into
> one of the issues created by fixing another issue with SLVM about a
> year ago. We are using ZFS in a couple ''production''
roles, only one of
> which is critical, and we picked ZFS because no other FS we tested
> scaled the way we needed it to.
Again ZFS and VxSF all fit differing purposes. I see ZFS trying to
compete with VxSF, but not the other way around (at least not in
techincal aspects).

-- 
_________________________________/ sengork.blogspot.com /????

On 1/19/08, Fred Zlotnick <fred.zlotnick at sun.com>
wrote:
> But, of course, many other enterprise customers _have_ adopted
> ZFS, and are quite happy with it.  For a list of ZFS reference
> customers please contact Solaris Marketing.  ZFS is used in many
> mission critical roles today, and by and large our customers
> are thrilled with it.
I''d wonder what the real word proportions are. I suspect
marketing''d
mainly know of the ones who did adopt it.

-- 
_________________________________/ sengork.blogspot.com /????

Sengor wrote:
> On 1/19/08, Fred Zlotnick <fred.zlotnick at sun.com> wrote:
>   
>> But, of course, many other enterprise customers _have_ adopted
>> ZFS, and are quite happy with it.  For a list of ZFS reference
>> customers please contact Solaris Marketing.  ZFS is used in many
>> mission critical roles today, and by and large our customers
>> are thrilled with it.
>>     
>
> I''d wonder what the real word proportions are. I suspect
marketing''d
> mainly know of the ones who did adopt it.
>
>   
Enterprise customers tend to be late adopters.  Indeed, there is
still a healthy number of new deployments of Solaris 8 today,
4 years after the last Solaris 8 update.  Obviously these people
do not deploy ZFS. Yes, we have a sense of this, at least for
people who purchase support from Sun, but in this open world,
it becomes harder to determine the installed base.

OTOH, 100% of the people who are experimenting with project
Indiana use ZFS.
 -- richard

On Jan 18, 2008 7:35 AM, Darren J Moffat <darrenm at opensolaris.org>
wrote:
> Sengor wrote:
> > On 1/17/08, Darren J Moffat <darrenm at opensolaris.org> wrote:
> >>> Pardon my ignorance, but is ZFS with compression safe to use
in a
> >>> production environment?
> >> Yes, why wouldn''t it be ?  If it wasn''t safe it
wouldn''t have been
> >> delivered.
> >
> > Few reasons -
http://prefetch.net/blog/index.php/2007/11/28/is-zfs-ready-for-primetime/
>
> The article (not the comments) is complete free of content and is scare
> mongering.  It doesn''t even say wither this is ZFS on Solaris (vs
BSD or
> MacOS X) never mind what release or the configuration of the pool or
> even what the actual "bug" apparently.  Was the pool redundant if
there
> were bugs what are the bug numbers and are they fixed.
I don''t think this is scare mongering at all. I wrote the blog entry
after a ZFS bug (6454482) corrupted a pool on one of our production
servers, and a yet unidentified bug (which appears to be different
than 6454482) corrupted a pool on another system. ZFS is an incredible
file system, but based on the fact that we lost data twice, I am
somewhat hesitant to continue to using it.

Just my .02,
- Ryan
-- 
UNIX Administrator
http://prefetch.net

Matty wrote:
> On Jan 18, 2008 7:35 AM, Darren J Moffat <darrenm at opensolaris.org>
wrote:
>> Sengor wrote:
>>> On 1/17/08, Darren J Moffat <darrenm at opensolaris.org>
wrote:
>>>>> Pardon my ignorance, but is ZFS with compression safe to
use in a
>>>>> production environment?
>>>> Yes, why wouldn''t it be ?  If it wasn''t safe
it wouldn''t have been
>>>> delivered.
>>> Few reasons -
http://prefetch.net/blog/index.php/2007/11/28/is-zfs-ready-for-primetime/
>> The article (not the comments) is complete free of content and is scare
>> mongering.  It doesn''t even say wither this is ZFS on Solaris
(vs BSD or
>> MacOS X) never mind what release or the configuration of the pool or
>> even what the actual "bug" apparently.  Was the pool
redundant if there
>> were bugs what are the bug numbers and are they fixed.
> 
> I don''t think this is scare mongering at all. I wrote the blog
entry
> after a ZFS bug (6454482) corrupted a pool on one of our production
> servers, and a yet unidentified bug (which appears to be different
> than 6454482) corrupted a pool on another system. ZFS is an incredible
> file system, but based on the fact that we lost data twice, I am
> somewhat hesitant to continue to using it.
I have no issue at all with people blogging bugs and comments.  What I 
had (and have) an issue with is when no details are given that is when 
it is scare mongering and FUD.

Provide at least a little detail, eg what release you are running on and 
at least a symptom of the problem (eg zpool status said everything was 
faulted), ideally as much of the storage config as you can - 
particularly the pool config (mirror vs raidz vs single disk).


-- 
Darren J Moffat