zfs crypto discuss - Sep 2007 - Use case for encrypted file data cleartext metadata

It would be relatively easy to provide a mode (even per dataset) where 
file content is encrypted but all file and ZFS metadata (ACLs, filename, 
permissions etc) are in the clear.  In this case a mount of the 
filesystem would succeed even if the key isn''t present but the key
would
be needed to read the data.

However I can''t find a useful real world case where this would be 
deployed.  In fact quite the opposite filenames and directory structure 
are often just as sensitive as the file content so really should be 
encrypted.

Does anyone have a use case for clear text filenames and metadata but 
filesystem encrypted content ?

-- 
Darren J Moffat

On Thu, Sep 06, 2007 at 05:27:43PM +0100, Darren J Moffat
wrote:
> It would be relatively easy to provide a mode (even per dataset) where 
> file content is encrypted but all file and ZFS metadata (ACLs, filename, 
> permissions etc) are in the clear.  In this case a mount of the 
> filesystem would succeed even if the key isn''t present but the key
would
> be needed to read the data.
> 
> However I can''t find a useful real world case where this would be 
> deployed.  In fact quite the opposite filenames and directory structure 
> are often just as sensitive as the file content so really should be 
> encrypted.
> 
> Does anyone have a use case for clear text filenames and metadata but 
> filesystem encrypted content ?
Traditional backups.

Personally I think that whole and incremental zfs sends (with ciphertext
as it appears on disk, or perhaps re-encrypted) are the way to deal with
backups of encrypted filesystems, and that metadata should be encrypted
as well.  But I''m sure many will disagree.

Using zfs sends exclusively for backups means that restores have to be
of entire datasets, not just of individual files.  But I don''t think
production environments are accustomed to the consequences of using zfs
sends instead of traditional archives.

Nicolas Williams wrote:
>> Does anyone have a use case for clear text filenames and metadata but 
>> filesystem encrypted content ?
> 
> Traditional backups.
If by traditional you mean things like ''tar''.   How would you
restore ?
   Note that the on disk ciphertext has an IV derived from the 
transaction group that the data was written in.  That info wouldn''t be 
in the backup so it couldn''t be restored.
> Personally I think that whole and incremental zfs sends (with ciphertext
> as it appears on disk, or perhaps re-encrypted) are the way to deal with
> backups of encrypted filesystems, and that metadata should be encrypted
> as well.  But I''m sure many will disagree.
I''m thinking about that separately, I first need to work out exactly
how
  zfs send with property passing is going to work.
> Using zfs sends exclusively for backups means that restores have to be
> of entire datasets, not just of individual files.  But I don''t
think
> production environments are accustomed to the consequences of using zfs
> sends instead of traditional archives.
zfs send isn''t a viable long term backup due to this statement in
zfs(1):

      The format of the stream is evolving. No backwards  compati-
      bility  is  guaranteed.  You may not be able to receive your
      streams on future versions of ZFS.

zfs send is very useful for the "intermediate" staging process of 
getting the data off the machine with the pool to the machine with the 
tape drive.

-- 
Darren J Moffat

On Thu, Sep 06, 2007 at 06:44:06PM +0100, Darren J Moffat
wrote:
> Nicolas Williams wrote:
> >>Does anyone have a use case for clear text filenames and metadata
but
> >>filesystem encrypted content ?
> >
> >Traditional backups.
> 
> If by traditional you mean things like ''tar''.   How would
you restore ?
>   Note that the on disk ciphertext has an IV derived from the 
> transaction group that the data was written in.  That info
wouldn''t be
> in the backup so it couldn''t be restored.
You''d have to find a way to include the IV in the backup (e.g., in an
extended or system attr).
> >Personally I think that whole and incremental zfs sends (with
ciphertext
> >as it appears on disk, or perhaps re-encrypted) are the way to deal
with
> >backups of encrypted filesystems, and that metadata should be encrypted
> >as well.  But I''m sure many will disagree.
> 
> I''m thinking about that separately, I first need to work out
exactly how
>  zfs send with property passing is going to work.
Sure.
> >Using zfs sends exclusively for backups means that restores have to be
> >of entire datasets, not just of individual files.  But I don''t
think
> >production environments are accustomed to the consequences of using zfs
> >sends instead of traditional archives.
> 
> zfs send isn''t a viable long term backup due to this statement in
zfs(1):
> 
>      The format of the stream is evolving. No backwards  compati-
>      bility  is  guaranteed.  You may not be able to receive your
>      streams on future versions of ZFS.
The ZFS or ZFS crypto teams could revise the format so that it is
sufficiently extensible as to support backwards compatibility.  There''s
nothing fundamental about zfs send that prevents it, just a
''small''
matter of protocol design and implementation.

Nico
--

On Sep 6, 2007, at 10:57 AM, Nicolas Williams wrote:
> On Thu, Sep 06, 2007 at 06:44:06PM +0100, Darren J Moffat wrote:
>> Nicolas Williams wrote:
>>>> Does anyone have a use case for clear text filenames and
metadata
>>>> but
>>>> filesystem encrypted content ?
Seemingly unrelated question, but maybe not.

Can we boot a machine, open a pool containing many datasets, at least  
one of which is encrypted with a key that is *not* know to the kernel  
at boot time, and then later a user can mount the encrypted dataset?  
(A simple yes would  be cool.)
>>> Personally I think that whole and incremental zfs sends (with  
>>> ciphertext
>>> as it appears on disk, or perhaps re-encrypted) are the way to  
>>> deal with
>>> backups of encrypted filesystems, and that metadata should be  
>>> encrypted
>>> as well.  But I''m sure many will disagree.
>>
>> I''m thinking about that separately, I first need to work out  
>> exactly how
>> zfs send with property passing is going to work.
>
> Sure.
Fully encrypted ZFS-send is more valuable than tar balls of filenames  
in the clear and file encrypted. File metadata leaks a lot of  
information.

james hughes wrote:
> On Sep 6, 2007, at 10:57 AM, Nicolas Williams wrote:
> 
>> On Thu, Sep 06, 2007 at 06:44:06PM +0100, Darren J Moffat wrote:
>>> Nicolas Williams wrote:
>>>>> Does anyone have a use case for clear text filenames and
metadata
>>>>> but
>>>>> filesystem encrypted content ?
> 
> Seemingly unrelated question, but maybe not.
> 
> Can we boot a machine, open a pool containing many datasets, at least  
> one of which is encrypted with a key that is *not* know to the kernel  
> at boot time, and then later a user can mount the encrypted dataset?  
> (A simple yes would  be cool.)
> 
yes.. that''s the plan.

Anthony Scarpino wrote:
> james hughes wrote:
>> On Sep 6, 2007, at 10:57 AM, Nicolas Williams wrote:
>>
>>> On Thu, Sep 06, 2007 at 06:44:06PM +0100, Darren J Moffat wrote:
>>>> Nicolas Williams wrote:
>>>>>> Does anyone have a use case for clear text filenames
and metadata
>>>>>> but
>>>>>> filesystem encrypted content ?
>>
>> Seemingly unrelated question, but maybe not.
>>
>> Can we boot a machine, open a pool containing many datasets, at least  
>> one of which is encrypted with a key that is *not* know to the kernel  
>> at boot time, and then later a user can mount the encrypted dataset?  
>> (A simple yes would  be cool.)
>>
> 
> yes.. that''s the plan.
not only the plan but it works in the current code base - I couldn''t 
easily do testing without it now.

The way this works just now is that during boot the ''zfs mount
-a'' will
skip over all datasets that are encrypted if the key isn''t present. 
They can then be later mounted manually.  If ZFS ever gains an 
automounting capability these would late be mounted automatically when 
the key is presented, for now an explicit zfs mount or zfs mount -a is 
required.

-- 
Darren J Moffat