zfs discuss - Dec 2006 - Thoughts on ZFS Secure Delete - without using Crypto

[ This is for discussion, it doesn''t mean I''m actively working
on this
   functionality at this time or that I might do so in the future. ]

When we get crypto support one way to do "secure delete" is to destroy
the key.  This is usually a much simpler and faster task than erasing 
and overwriting all the data and metadata blocks that have been used. 
It also provides the ability of doing time/policy based "deletion".

However for some use cases this will not be deemed sufficient [3]. It 
also doesn''t cover the case where crypto wasn''t always used on
that
physical storage.

There are userland applications like GNU shred that attempt to cover 
this for a single file by opening it up and doing overwriting before 
finally calling unlink(2) on it.  However shred and anything like it 
that works outside of ZFS will not work with ZFS because of COW.

I''m going to use the term "bleach" here because ZFS already
uses "scrub"
  which is what some other people use here, and because bleach doesn''t 
overload any of the specific terms used in this area.

I believe that ZFS should provide a method of bleaching a disk or part 
of it that works without crypto having ever been involved.

Currently OpenSolaris only provides what format(1M) gives with the 
analyze/purge command[1].

This doesn''t help in some cases because it requires that it be run on 
the whole disk.  It is also not implementing the current recommendation 
from NIST as documented in NIST-800-88 [2].

I think with ZFS we need to provide a method of bleaching that is 
compatible with COW, and doesn''t require we do it on the whole disk, 
because this is very time consuming. Ideally it should also be 
compatible with hot sparing.

I think we need 5 distinct places to set the policy:

1) On file delete
	This would be a per dataset policy.
	The bleaching would happen in a new transaction group
	created by the one that did the normal deletion, and would
	run only if the	original one completed.  It needs to be done in
	such away that the file blocks aren''t on the free list until
	after the bleaching txg is completed.

2) On ZFS data set destroy
	A per pool policy and possibly per dataset with inheritance.
	As above for the txg and the free blocks.

3) On demand for a pool without destroying active data.
	This is similar to today''s scrub, it is a background task
	that we start off periodically and view the status of it
	via zpool status.

4) On pool destroy (purposely breaks import -d)

5) On hotsparing, bleach the outgoing disk.

When doing all of these (but particularly 2 through 5) we need a way to 
tell the admin that this has been completed:  I think zpool status is 
probably sufficient initially but we might want to do more.

For case 4 zpool destroy would not return until all the data has been 
bleached - this might take some time so we should provide progress.

Option 3 is needed for the case where we have no policy, ie 1 and 2 
aren''t in effect and we know that soon we need to do a disk
replacement.

With option 5 we would spare in the new disk and start doing a full 
media bleaching on the outgoing disk.  In this case zpool status would 
show that a bleaching is in progress on that disk and that the admin 
should wait until it completes before physical removal.

Instead of just implementing the current NIST algorithm it would be much 
more helpful if we made this exensible, not necessarily fully plugable 
without modifying the ZFS source. The CMRR project at UCSD has a good 
paper on the security/speed tradeoffs [4]. Initially we would provide at 
least the following bleaching methods:

1) N passes of zeros, default being 1.
2) Same algorithm that format(1M) uses today.
3) NIST-800-88 compliant method.
4) others to be discussed.

One of the tools from the CMRR project specifically takes advantage of 
features of ATA drives, I don''t think we should do that in ZFS because 
we could be dealing with a pool created from a mix of different drive 
types or we could be dealing with things like iSCSI targets which are 
really ZVOLs "on the other side".


Thoughts ?


References:

[1] 
http://www.sun.com/software/solaris/trustedsolaris/ts_tech_faq/faqs/purge.xml

[2] http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf

[3] 09-11-06 update to [2] on page 7.

[4] http://cmrr.ucsd.edu/hughes/subpgset.htm


-- 
Darren J Moffat

Darren J Moffat wrote:
> I think we need 5 distinct places to set the policy:
> 
> 1) On file delete
>     This would be a per dataset policy.
>     The bleaching would happen in a new transaction group
>     created by the one that did the normal deletion, and would
>     run only if the    original one completed.  It needs to be done in
>     such away that the file blocks aren''t on the free list until
>     after the bleaching txg is completed.
> 
> 2) On ZFS data set destroy
>     A per pool policy and possibly per dataset with inheritance.
>     As above for the txg and the free blocks.
> 
> 3) On demand for a pool without destroying active data.
>     This is similar to today''s scrub, it is a background task
>     that we start off periodically and view the status of it
>     via zpool status.
> 
> 4) On pool destroy (purposely breaks import -d)
> 
> 5) On hotsparing, bleach the outgoing disk.
There is one other one that I forgot to mention....

6) When modifying any file you want to bleach the old blocks in a way 
very simlar to case 1 above.

This and 1) will have some interesting issues around snapshots and clones.


-- 
Darren J Moffat

On 12/18/06, Darren J Moffat <Darren.Moffat at sun.com>
wrote:
>
> [ This is for discussion, it doesn''t mean I''m actively
working on this
>    functionality at this time or that I might do so in the future. ]
>
> When we get crypto support one way to do "secure delete" is to
destroy
> the key.  This is usually a much simpler and faster task than erasing
> and overwriting all the data and metadata blocks that have been used.
> It also provides the ability of doing time/policy based
"deletion".
>
> However for some use cases this will not be deemed sufficient [3]. It
> also doesn''t cover the case where crypto wasn''t always
used on that
> physical storage.
>
> There are userland applications like GNU shred that attempt to cover
> this for a single file by opening it up and doing overwriting before
> finally calling unlink(2) on it.  However shred and anything like it
> that works outside of ZFS will not work with ZFS because of COW.
>
> I''m going to use the term "bleach" here because ZFS
already uses "scrub"
>   which is what some other people use here, and because bleach
doesn''t
> overload any of the specific terms used in this area.
>
> I believe that ZFS should provide a method of bleaching a disk or part
> of it that works without crypto having ever been involved.
>
> Currently OpenSolaris only provides what format(1M) gives with the
> analyze/purge command[1].
>
> This doesn''t help in some cases because it requires that it be run
on
> the whole disk.  It is also not implementing the current recommendation
> from NIST as documented in NIST-800-88 [2].
>
> I think with ZFS we need to provide a method of bleaching that is
> compatible with COW, and doesn''t require we do it on the whole
disk,
> because this is very time consuming. Ideally it should also be
> compatible with hot sparing.
>
> I think we need 5 distinct places to set the policy:
>
> 1) On file delete
>         This would be a per dataset policy.
>         The bleaching would happen in a new transaction group
>         created by the one that did the normal deletion, and would
>         run only if the original one completed.  It needs to be done in
>         such away that the file blocks aren''t on the free list
until
>         after the bleaching txg is completed.
>
> 2) On ZFS data set destroy
>         A per pool policy and possibly per dataset with inheritance.
>         As above for the txg and the free blocks.
>
> 3) On demand for a pool without destroying active data.
>         This is similar to today''s scrub, it is a background task
>         that we start off periodically and view the status of it
>         via zpool status.
>
> 4) On pool destroy (purposely breaks import -d)
>
> 5) On hotsparing, bleach the outgoing disk.
>
> When doing all of these (but particularly 2 through 5) we need a way to
> tell the admin that this has been completed:  I think zpool status is
> probably sufficient initially but we might want to do more.
>
> For case 4 zpool destroy would not return until all the data has been
> bleached - this might take some time so we should provide progress.
>
> Option 3 is needed for the case where we have no policy, ie 1 and 2
> aren''t in effect and we know that soon we need to do a disk
replacement.
>
> With option 5 we would spare in the new disk and start doing a full
> media bleaching on the outgoing disk.  In this case zpool status would
> show that a bleaching is in progress on that disk and that the admin
> should wait until it completes before physical removal.
>
> Instead of just implementing the current NIST algorithm it would be much
> more helpful if we made this exensible, not necessarily fully plugable
> without modifying the ZFS source. The CMRR project at UCSD has a good
> paper on the security/speed tradeoffs [4]. Initially we would provide at
> least the following bleaching methods:
>
> 1) N passes of zeros, default being 1.
> 2) Same algorithm that format(1M) uses today.
> 3) NIST-800-88 compliant method.
> 4) others to be discussed.
>
> One of the tools from the CMRR project specifically takes advantage of
> features of ATA drives, I don''t think we should do that in ZFS
because
> we could be dealing with a pool created from a mix of different drive
> types or we could be dealing with things like iSCSI targets which are
> really ZVOLs "on the other side".
>
>
> Thoughts ?

one Idea that would seem easy for people that aren''t totally paranoid
would
be to pass the group of blocks that contained the deleted file to the top
of  the list of available blocks to be used for new writes, so that at least
no matter what else happens in a few minutes the data will be over written
at least once. Perhaps this should happen with all levels of secure delete,
so that even after the blocks have been overwritten with 6 times with random
data, it would be next to receive a new file. This might even help
performance since the hard disk heads should still be near the blocks of
overwritten data.

Of course the converse might also be nice, if I''m a home user and I
have
been known to make errors it might be nice to have recently deleted blocks
not written over for a while, of course someone would still need to write a
file rescue utility to benefit from this but it could a be tunable option
either per pool or per file system.

James Dickens
uadmin.blogspot.com




References:
>
> [1]
>
>
http://www.sun.com/software/solaris/trustedsolaris/ts_tech_faq/faqs/purge.xml
>
> [2]
> http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf
>
> [3] 09-11-06 update to [2] on page 7.
>
> [4] http://cmrr.ucsd.edu/hughes/subpgset.htm
>
>
> --
> Darren J Moffat
> _______________________________________________
> security-discuss mailing list
> security-discuss at opensolaris.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20061218/b9fdb23b/attachment.html>

>Darren J Moffat wrote:
>> I think we need 5 distinct places to set the policy:
>> 
>> 1) On file delete
>>     This would be a per dataset policy.
>>     The bleaching would happen in a new transaction group
>>     created by the one that did the normal deletion, and would
>>     run only if the    original one completed.  It needs to be done in
>>     such away that the file blocks aren''t on the free list
until
>>     after the bleaching txg is completed.
>> 
>> 2) On ZFS data set destroy
>>     A per pool policy and possibly per dataset with inheritance.
>>     As above for the txg and the free blocks.
>> 
>> 3) On demand for a pool without destroying active data.
>>     This is similar to today''s scrub, it is a background task
>>     that we start off periodically and view the status of it
>>     via zpool status.
>> 
>> 4) On pool destroy (purposely breaks import -d)
>> 
>> 5) On hotsparing, bleach the outgoing disk.
>
>There is one other one that I forgot to mention....
>
>6) When modifying any file you want to bleach the old blocks in a way 
>very simlar to case 1 above.
>
>This and 1) will have some interesting issues around snapshots and clones.

Rather than bleaching which doesn''t always remove all stains, why
can''t
we use a word like "erasing" (which is hitherto unused for filesystem
use
in Solaris, AFAIK)

I''m not sure if erasing outgoing disks is usable; apart from it taking
a
long time, the reason for removing the disks may be that it no longer
responds.


Casper

Casper.Dik at Sun.COM wrote:
> Rather than bleaching which doesn''t always remove all stains, why
can''t
> we use a word like "erasing" (which is hitherto unused for
filesystem use
> in Solaris, AFAIK)
and this method doesn''t remove all stains from the disk anyway it just 
reduces them so they can''t be easily seen ;-)
> I''m not sure if erasing outgoing disks is usable; apart from it
taking a
> long time, the reason for removing the disks may be that it no longer
> responds.
I''m not suggesting bleaching for the user interface just for this 
discussion so that I didn''t use a word already used and I was purposely
avoiding using erase (should have said that!).

-- 
Darren J Moffat

IMO:


 - The hardest problem in the case of bleaching individual files or
   datasets is dealing with snapshots/clones:

    - blocks not shared with parent/child snapshots can be bleached with
      little trouble, of course.

    - But what about shared blocks?

      IMO we have two options:

      a) Warn the user that some blocks remain unbleached;
      b) Give the user an option to force bleaching shared blocks.

      (b) Would break the snapshot read-only-ness, but it''s better than
      forcing the user to destroy clones and snapshots in order to
      really bleach some sensitive content.

   I''d say go for both, (a) and (b).  Of course, (b) may not be easy to
   implement.


 - Unlink is not the right operation to bleach on!

   That''s because we don''t have a fast dnode# -> path
function (ncheck),
   and we don''t have path info for the unlinked file in the snapshots
   and clones that share it (worse, they may have additional hard
   links).

   Instead I would propose a new bleach(1), bleach(2), VOP_BLEACH() and
   fop_bleach() (with emulation in the fop for UFS and filesystems that
   don''t COW).

   The new system call should either appear to truncate the file or to
   overwrite it with zeros.  The latter would allow for bleaching some
   byte ranges, rather than the whole file (ZFS complexity: first COW
   the non-bleached parts of blocks, then bleach the freed blocks).


 - Bleaching vdevs is easier, of course, but the whole vdev has to be
   bleached as by that point we''ve no knowledge of which blocks have
   never been used.


Cheers,

Nico
--

On Mon, Dec 18, 2006 at 11:32:37AM -0600, Nicolas Williams
wrote:
>    The new system call should either appear to truncate the file or to
>    overwrite it with zeros.  The latter would allow for bleaching some
>    byte ranges, rather than the whole file (ZFS complexity: first COW
>    the non-bleached parts of blocks, then bleach the freed blocks).
Also, all meta-data blocks should be COWed no matter what; freed
metat-data blocks could be bleached, but shared ones could not be.  That
means no bleaching directories, and it means that that block checksums
and file sizes can still leak after bleaching.  I think that''s
unfortunate, but not that big a deal.

Nico
--

On Mon, Dec 18, 2006 at 05:44:08PM -0500, Jeffrey Hutzelman
wrote:
> On Monday, December 18, 2006 11:32:37 AM -0600 Nicolas Williams 
> <Nicolas.Williams at Sun.COM> wrote:
> >   I''d say go for both, (a) and (b).  Of course, (b) may not
be easy to
> >   implement.
> 
> Another option would be to warn the user and set a flag on the shared block
> which causes it to be bleached when the last reference goes away.  Of 
> course, one still might want to give the user the option of forcing 
> immediate bleaching of the shared data.
Sure, but if I want something bleached I probably want it bleahced
_now_, not who knows when.
> > - Unlink is not the right operation to bleach on!
> >
> >   That''s because we don''t have a fast dnode# ->
path function (ncheck),
> >   and we don''t have path info for the unlinked file in the
snapshots
> >   and clones that share it (worse, they may have additional hard
> >   links).
> 
> I''m not sure how this matters.  It certainly is desirable to be
able to set
> policy which causes files to be automatically bleached on unlink (or, more 
> properly, when the link count goes to zero, modulo the complexity added by 
> clones and snapshots).
But why set that per-file?  Why not per-dataset/volume?  "Bleach all
blocks when they are freed" automatically means bleaching blocks when
the lask reference is gone (as a result of an unlink of the last file
that had some block, say).
> >   The new system call should either appear to truncate the file or to
> >   overwrite it with zeros.  The latter would allow for bleaching some
> >   byte ranges, rather than the whole file (ZFS complexity: first COW
> >   the non-bleached parts of blocks, then bleach the freed blocks).
> 
> Perhaps the system call should act on a file descriptor, and specify a 
> range to be bleached, with a whence/start/len a la F_SETLK.  This would 
> allow for bleaching byte ranges and also help with the vdev issue (one 
> might imagine an application which knows enough about what is on the device
> to bleach only certain blocks).
Or an iovec-style specification.  But really, how often will one prefer
this to truncate-and-bleach?  Also, the to-be-bleached octet ranges may
not be meaningful in snapshots/clones.  Hmmm.  That convinces me:
truncate-and-bleach or bleach-and-zero, but not bleach individual octet
ranges.

Nico
--

On Mon, Dec 18, 2006 at 05:16:28PM -0600, Nicolas Williams
wrote:
> On Mon, Dec 18, 2006 at 05:44:08PM -0500, Jeffrey Hutzelman wrote:
BTW, Jeff''s posts to zfs-discuss are being rejected with this message:

   You are not allowed to post to this mailing list, and your message
   has been automatically rejected.  If you think that your messages are
   being rejected in error, contact the mailing list owner at
   zfs-discuss-owner at opensolaris.org.

Can we please have our mailing list system not do this?

Nico
--

On Mon, Dec 18, 2006 at 06:46:09PM -0500, Jeffrey Hutzelman
wrote:
> 
> 
> On Monday, December 18, 2006 05:16:28 PM -0600 Nicolas Williams 
> <Nicolas.Williams at Sun.COM> wrote:
> 
> >Or an iovec-style specification.  But really, how often will one prefer
> >this to truncate-and-bleach?  Also, the to-be-bleached octet ranges may
> >not be meaningful in snapshots/clones.  Hmmm.  That convinces me:
> >truncate-and-bleach or bleach-and-zero, but not bleach individual octet
> >ranges.
> 
> Well, consider a file with some structure, like a berkeley db database. 
> The application may well want to bleach each record as it is deleted.
My point is those byte ranges might differe from one version of that
file to another.  Bleaching byte ranges could only affect the current
FS, not any snapshots/clones.  (Of course, if we decide that snapshots
are so read-only that we can''t provide a bleach facility that bleaches
across snapshots, then that''s fine.)

> BTW, Jeff''s posts to zfs-discuss are being rejected with this
message [ ... ]
... while the spam is coming through loud & clear.  ;-)
 
 
This message posted from opensolaris.org

Darren J Moffat wrote:
> ..
> I think we need 5 distinct places to set the policy:
>
> 1) On file delete
>     This would be a per dataset policy.
>     The bleaching would happen in a new transaction group
>     created by the one that did the normal deletion, and would
>     run only if the    original one completed.  It needs to be done in
>     such away that the file blocks aren''t on the free list until
>     after the bleaching txg is completed.
>
> 2) On ZFS data set destroy
>     A per pool policy and possibly per dataset with inheritance.
>     As above for the txg and the free blocks.
>
> 3) On demand for a pool without destroying active data.
>     This is similar to today''s scrub, it is a background task
>     that we start off periodically and view the status of it
>     via zpool status.
>
> 4) On pool destroy (purposely breaks import -d)
>
> 5) On hotsparing, bleach the outgoing disk.

If/when ZFS supports this then it would be nice to also be able
to have Solaris bleach swap on ZFS when it shuts down or reboots.
Although it may be that this option needs to be put into how we
manage swap space and not specifically zomething for ZFS.

Doing this to swap space has been a kernel option on another very
widely spread operating system for at least 2 major OS releases...

Darren

Nicolas Williams wrote:
> On Mon, Dec 18, 2006 at 05:44:08PM -0500, Jeffrey Hutzelman wrote:
>> On Monday, December 18, 2006 11:32:37 AM -0600 Nicolas Williams 
>> <Nicolas.Williams at Sun.COM> wrote:
>>>   I''d say go for both, (a) and (b).  Of course, (b) may
not be easy to
>>>   implement.
>> Another option would be to warn the user and set a flag on the shared
block
>> which causes it to be bleached when the last reference goes away.  Of 
>> course, one still might want to give the user the option of forcing 
>> immediate bleaching of the shared data.
> 
> Sure, but if I want something bleached I probably want it bleahced
> _now_, not who knows when.
I think there are two related things here, given your comments and 
suggestions for a bleach(1) command and VOP/FOP implementation you are 
thinking about a completely different usage method than I am.

How do you /usr/bin/bleach the tmp file that your editor wrote to before 
it did the rename ?  You can''t easily do that - if at all in some
cases.

I''m looking for the systemic solution here not the end user controlled
one.

For comparison what you are suggesting is like doing crypto with 
encrypt(1) it works on pathnames, where as what I''m suggesting is more 
like ZFS crypto it works "inside" ZFS with deep intimate knowledge of 
ZFS and requires zero change on behalf of the user or admin.

While I think having this in the VOP/FOP layer is interesting it isn''t 
the problem I was trying to solve and to be completely honest I''m
really
not interested in solving this outside of ZFS - why make it easy for 
people to stay on UFS ;-)
> But why set that per-file?  Why not per-dataset/volume?  "Bleach all
> blocks when they are freed" automatically means bleaching blocks when
> the lask reference is gone (as a result of an unlink of the last file
> that had some block, say).
I didn''t have anything per file, but exactly what you said.  The policy
was when files are removed, when data sets are removed, when pools are 
removed.

-- 
Darren J Moffat

Jeffrey Hutzelman wrote:
> 
> 
> On Monday, December 18, 2006 05:51:14 PM -0600 Nicolas Williams 
> <Nicolas.Williams at Sun.COM> wrote:
> 
>> On Mon, Dec 18, 2006 at 06:46:09PM -0500, Jeffrey Hutzelman wrote:
>>>
>>>
>>> On Monday, December 18, 2006 05:16:28 PM -0600 Nicolas Williams
>>> <Nicolas.Williams at Sun.COM> wrote:
>>>
>>> > Or an iovec-style specification.  But really, how often will
one
>>> prefer
>>> > this to truncate-and-bleach?  Also, the to-be-bleached octet
ranges
>>> may
>>> > not be meaningful in snapshots/clones.  Hmmm.  That convinces
me:
>>> > truncate-and-bleach or bleach-and-zero, but not bleach
individual
>>> octet
>>> > ranges.
>>>
>>> Well, consider a file with some structure, like a berkeley db
database.
>>> The application may well want to bleach each record as it is
deleted.
>>
>> My point is those byte ranges might differe from one version of that
>> file to another.
> 
> That byte range contains the data the application is trying to bleach in 
> any version of the file which contains the affected block(s).  Obviously 
> if the file has been modified and the data moved to someplace else, then 
> your bleach won''t affect the version(s) of the file before the
change.
> But then, there''s only so much you can do.
I explicitly do NOT want the applications involved in this, the whole 
point of my proposal being the way it was is that it works equally for 
all applications and no application code needs to or can be changed to 
change this behaviour.  Just like doing crypto "in" the filesystem vs 
doing it at the application layer.

-- 
Darren J Moffat

Darren Reed wrote:
> If/when ZFS supports this then it would be nice to also be able
> to have Solaris bleach swap on ZFS when it shuts down or reboots.
> Although it may be that this option needs to be put into how we
> manage swap space and not specifically zomething for ZFS.
> 
> Doing this to swap space has been a kernel option on another very
> widely spread operating system for at least 2 major OS releases...
Which ones ?  I know that MacOS X and OpenBSD both support "encrypted"
swap which for swap IMO is a better way to solve this problem.

You can get that today with OpenSolaris by using the stuff in the loficc 
project.   You will also get encrypted swap when we have ZFS crypto and 
you swap on a ZVOL that is encrypted.

Note though that that isn''t quite the same way as OpenBSD solves the 
encrypted swap problem, and I''m not familiar with the technical details
of what Apple did in MacOS X.

Bleaching is a time consuming task, not something I''d want to do at 
system boot/halt.

-- 
Darren J Moffat

In case it wasn''t clear I am NOT proposing a UI like this:

$ zfs bleach ~/Documents/company-finance.odp

Instead ~/Documents or ~ would be a ZFS file system with a policy set 
something like this:

# zfs set erase=file:zero

Or maybe more like this:

# zfs create -o erase=file -o erasemethod=zero homepool/darrenm

The goal is the same as the goal for things like compression in ZFS, no 
application change it is "free" for the applications.

All of the same reasons for doing crypto outside of a command like 
encrypt(1) apply here too - especially the temp file and rename problems.

--
Darren J Moffat

On Dec 19, 2006, at 08:59, Darren J Moffat wrote:
> Darren Reed wrote:
>> If/when ZFS supports this then it would be nice to also be able
>> to have Solaris bleach swap on ZFS when it shuts down or reboots.
>> Although it may be that this option needs to be put into how we
>> manage swap space and not specifically zomething for ZFS.
>> Doing this to swap space has been a kernel option on another very
>> widely spread operating system for at least 2 major OS releases...
>
> Which ones ?  I know that MacOS X and OpenBSD both support  
> "encrypted" swap which for swap IMO is a better way to solve this
> problem.
>
> You can get that today with OpenSolaris by using the stuff in the  
> loficc project.   You will also get encrypted swap when we have ZFS  
> crypto and you swap on a ZVOL that is encrypted.
>
> Note though that that isn''t quite the same way as OpenBSD solves  
> the encrypted swap problem, and I''m not familiar with the
technical
> details of what Apple did in MacOS X.
there''s an encryption option in the dynamic_pager to write out  
encrypted paging files (/var/vm/swapfile*) .. it gets turned on with  
an environment variable that gets set at boot (what happens when you  
choose secure virtual memory.)  Before this was implemented there was  
a workaround using an encrypted dmg that held the swap files .. but  
that was an incomplete solution.
> Bleaching is a time consuming task, not something I''d want to do
at
> system boot/halt.
particularly if we choose to do a 35 pass Gutmann algorithm .. :)

---
.je

On Dec 18, 2006, at 11:54, Darren J Moffat wrote:
> Casper.Dik at Sun.COM wrote:
>> Rather than bleaching which doesn''t always remove all stains,
why
>> can''t
>> we use a word like "erasing" (which is hitherto unused for  
>> filesystem use
>> in Solaris, AFAIK)
>
> and this method doesn''t remove all stains from the disk anyway it
> just reduces them so they can''t be easily seen ;-)
and if you add the right amount of ammonia is should remove  
everything .. (ahh - fun with trichloramine)

---
.je

On Tue, Dec 19, 2006 at 02:04:37PM +0000, Darren J Moffat
wrote:
> In case it wasn''t clear I am NOT proposing a UI like this:
> 
> $ zfs bleach ~/Documents/company-finance.odp
> 
> Instead ~/Documents or ~ would be a ZFS file system with a policy set 
> something like this:
> 
> # zfs set erase=file:zero
> 
> Or maybe more like this:
> 
> # zfs create -o erase=file -o erasemethod=zero homepool/darrenm
I get it.  This should be lots easier than bleach(1).  Snapshots/clones
are mostly not an issue here.  When a block is truly freed, then it is
wiped.

Clones are an issue here only if they have different settings for this
property than the FS that spawned them (so you might want to disallow
re-setting of this property).

Nico
--

Nicolas Williams wrote:
> On Tue, Dec 19, 2006 at 02:04:37PM +0000, Darren J Moffat wrote:
>> In case it wasn''t clear I am NOT proposing a UI like this:
>>
>> $ zfs bleach ~/Documents/company-finance.odp
>>
>> Instead ~/Documents or ~ would be a ZFS file system with a policy set 
>> something like this:
>>
>> # zfs set erase=file:zero
>>
>> Or maybe more like this:
>>
>> # zfs create -o erase=file -o erasemethod=zero homepool/darrenm
> 
> I get it.  This should be lots easier than bleach(1).  Snapshots/clones
> are mostly not an issue here.  When a block is truly freed, then it is
> wiped.
Yep.
> Clones are an issue here only if they have different settings for this
> property than the FS that spawned them (so you might want to disallow
> re-setting of this property).
I think you are saying it should have INHERITY set to YES and EDIT set 
to NO.  We don''t currently have any properties like that but crypto
will
need this as well - for a very similar reason with clones.


-- 
Darren J Moffat

On Tue, 19 Dec 2006, Jonathan Edwards wrote:
>
> On Dec 18, 2006, at 11:54, Darren J Moffat wrote:
>
>> Casper.Dik at Sun.COM wrote:
>>> Rather than bleaching which doesn''t always remove all
stains, why can''t
>>> we use a word like "erasing" (which is hitherto unused
for filesystem use
>>> in Solaris, AFAIK)
>> 
>> and this method doesn''t remove all stains from the disk anyway
it just
>> reduces them so they can''t be easily seen ;-)
>
> and if you add the right amount of ammonia is should remove everything .. 
> (ahh - fun with trichloramine)
Fluoric acid will dissolve the magnetic film on the platter as well as the 
platter itself. Always keep a PTFE bottle with the stuff in, just in case

;)

On the technical side, I don''t think a new VOP will be needed. This
could
easily be done in VOP_SPACE together with a new per-fs property - bleach 
new block when it''s allocated (aka VOP_SPACE directly, or in a backend 
also called e.g. on allocating writes / filling holes), bleach existing 
block when VOP_SPACE is used to "stamp a hole" into a file, aka a
request
is made to bleach the blocks of an existing file.
I.e. make the implementation behind ftruncate()/posix_fallocate() do the 
per-file bleaching if so desired. And that implementation is VOP_SPACE.

FrankH.

>
> ---
> .je
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Tue, Dec 19, 2006 at 04:37:36PM +0000, Darren J Moffat
wrote:
> I think you are saying it should have INHERITY set to YES and EDIT set 
> to NO.  We don''t currently have any properties like that but
crypto will
> need this as well - for a very similar reason with clones.
What I mean is that if there''s a block that''s shared between
two
filesystems then what do you do if the two filesystems have different
settings for this property?  IMO you shouldn''t allow this to happen.

Frank Hofmann wrote:
> On the technical side, I don''t think a new VOP will be needed.
This
> could easily be done in VOP_SPACE together with a new per-fs property - 
> bleach new block when it''s allocated (aka VOP_SPACE directly, or
in a
> backend also called e.g. on allocating writes / filling holes), bleach 
> existing block when VOP_SPACE is used to "stamp a hole" into a
file, aka
> a request is made to bleach the blocks of an existing file.
> I.e. make the implementation behind ftruncate()/posix_fallocate() do the 
> per-file bleaching if so desired. And that implementation is VOP_SPACE.
That isn''t solving the problem though, it solves a different problem.

The problem that I want to be solved is that as files/datasets/pools are 
deleted (not as they are allocated) they are bleached.  In the cases 
there would not be a call to posix_fallocate() or ftruncate(), instead 
an unlink(2) or a zfs destory or zpool destroy.  Also on hotsparing in a 
disk - if the old disk can still be written to in some way we should do 
our best to bleach it.

-- 
Darren J Moffat

On Tue, 19 Dec 2006, Darren J Moffat wrote:
> Frank Hofmann wrote:
>> On the technical side, I don''t think a new VOP will be needed.
This could
>> easily be done in VOP_SPACE together with a new per-fs property -
bleach
>> new block when it''s allocated (aka VOP_SPACE directly, or in a
backend also
>> called e.g. on allocating writes / filling holes), bleach existing
block
>> when VOP_SPACE is used to "stamp a hole" into a file, aka a
request is made
>> to bleach the blocks of an existing file.
>> I.e. make the implementation behind ftruncate()/posix_fallocate() do
the
>> per-file bleaching if so desired. And that implementation is VOP_SPACE.
>
> That isn''t solving the problem though, it solves a different
problem.
Well, the thread has taken lots of turns already; the "erase just a
file"
task has been mentioned, and someone threw the idea of "VOP_BLEACH"
in.
>
> The problem that I want to be solved is that as files/datasets/pools are 
> deleted (not as they are allocated) they are bleached.  In the cases there
VOP_SPACE() does truncation (free) as well as growth (alloc).
> would not be a call to posix_fallocate() or ftruncate(), instead an
unlink(2)
> or a zfs destory or zpool destroy.  Also on hotsparing in a disk - if the
old
> disk can still be written to in some way we should do our best to bleach
it.
Since VOP_*() requires a filesystem (with "/" specifying "all of
this
fs"), per-zvol or per-vdev "bleaching" needs a different
implementation
vehicle, that''s clear, as you don''t have any handle then that
you could
call it with.

I wouldn''t just brush the VOP_*() approach aside. The world
isn''t pure ZFS
there''s more to it, whether you wish otherwise or no ...

FrankH.


>
> -- 
> Darren J Moffat
>

Frank Hofmann wrote:
>> would not be a call to posix_fallocate() or ftruncate(), instead an 
>> unlink(2) or a zfs destory or zpool destroy.  Also on hotsparing in a 
>> disk - if the old disk can still be written to in some way we should 
>> do our best to bleach it.
> 
> Since VOP_*() requires a filesystem (with "/" specifying
"all of this
> fs"), per-zvol or per-vdev "bleaching" needs a different
implementation
> vehicle, that''s clear, as you don''t have any handle then
that you could
> call it with.
> 
> I wouldn''t just brush the VOP_*() approach aside. The world
isn''t pure
> ZFS there''s more to it, whether you wish otherwise or no ...
I agree, but I think these things are complementary and that isn''t 
solving the problem I was trying to solve since it requires someone make 
explicit calls other than unlink(2) or similar.

-- 
Darren J Moffat

On Mon, 2006-12-18 at 16:05 +0000, Darren J Moffat
wrote:
> 6) When modifying any file you want to bleach the old blocks in a way 
> very simlar to case 1 above.
I think this is the crux of the problem.  If you fail to solve it, you
can''t meaningfully say that all blocks which once contained parts of a
file have been overwritten and instead have to fall back on a "bleach
all unallocated blocks in the pool".

And if you can solve this one, I think you get cases 1 and 2 "for
free".

I think the way to go here is to create a file, dataset, and/or pool
property which turns on "bleach on free"; any blocks freed after this
property set will be appropriately bleached.

Other issues:
 - in some threat models, overwrite by zero is sufficient; in others,
you need multiple passes of overwrite with specific data patterns.

 - If you''re going to immediately reuse a block, do you need to bleach
before reallocation, or is mere overwrite by different allocated data
considered sufficient?

There also may be a reason to do this when confidentiality isn''t
required: as a sparse provisioning hack..

If you were to build a zfs pool out of compressed zvols backed by
another pool, then it would be very convenient if you could run in a
mode where freed blocks were overwritten by zeros when they were freed,
because this would permit the underlying compressed zvol to free *its*
blocks.

						- Bill

On Tue, Dec 19, 2006 at 03:09:03PM -0500, Jeffrey Hutzelman
wrote:
> 
> 
> On Tuesday, December 19, 2006 01:54:56 PM +0000 Darren J Moffat 
> <Darren.Moffat at Sun.COM> wrote:
> 
> >While I think having this in the VOP/FOP layer is interesting it
isn''t
> >the problem I was trying to solve and to be completely honest
I''m really
> >not interested in solving this outside of ZFS - why make it easy for
> >people to stay on UFS ;-)
> 
> Because as great as ZFS is, someday someone is going to run into a problem 
> that it doesn''t solve.  Having the right abstraction to begin with
will
> make that day easier when it comes.
I understand what Darren was proposing now.  He''s talking about wiping
blocks as they are freed.

I initially thought he meant something like a guarantee on file deletion
that the file''s data is gone -- but snapshots and clones are in
conflict
with that, but not with wiping blocks as they are freed.

Now, if we had a bleach(1) operation, then we''d need a bleach(2) and a
VOP_BLEACH and fop_bleach.  But that''s not what Darren is proposing.
> >I didn''t have anything per file, but exactly what you said. 
The policy
> >was when files are removed, when data sets are removed, when pools are
> >removed.
> 
> Well, that''s great for situations where things actually get
removed.  It''s
> not so great for things that get rewritten rather than removed, and it 
> seems nearly useless for vdevs.  I think there''s some benefit to
making the
> functionality directly available to user-mode, but more importantly, 
> there''s a definite advantage to a system in which the user knows
that a
> file was bleached when they removed it, and not decades later when someone 
> gets around to removing a stray snapshot.  That difference can have serious
> legal and/or intelligence implications.
Yes, I think that a bleach operation that forcefully removes a file''s
contents even in all snapshots and clones, could be useful.  But I''m
not
sure that we can get it.

Nico
--

Darren J Moffat wrote:
> I believe that ZFS should provide a method of bleaching a disk or part 
> of it that works without crypto having ever been involved.
I see two use cases here:

1. This filesystem contains sensitive information.  When it is freed, 
make sure it''s really gone.

I believe that the proper way to do this is to introduce a new zfs 
property (say, "bleach") which, when set, will cause any blocks which 
are freed to be bleached.  The value of the property can specify the 
type of bleach to use (ie, the overwrite pattern).

To ease administrative complexity, this property should probably only be 
set at filesystem creation time (eg. ''zfs create -o 
bleach=maximum-strength pool/fs''). Furthermore, ''zfs promote
<fs>''
should not be run if the fs and its origin have different bleach 
settings (because the ownership of the shared blocks would change). 
However, clones do not in general pose any problems for this model 
(because it''s always clear which fs owns each block, and only that fs 
can free it).

2. I''ve deleted some stuff at some point in the past, and now I want to
make sure that it''s really gone.

To do this, we should have a "zpool bleach" command which would 
immediately go and bleach all unallocated blocks.

Note that if both these features are implemented, it might make sense to 
allow changing the ''bleach'' property on a fs after it has been
created,
and have that implicitly kick off a ''zpool bleach'' to ensure
that any
space previously used for that fs is bleached.

Also note that these bleaching features would not change the semantics 
of snapshots, clones, etc.  A block will not be bleached until all 
references to it are removed, and the block is freed.

--matt

On Tue, 2006-12-19 at 16:19 -0800, Matthew Ahrens wrote:
> Darren J Moffat wrote:
> > I believe that ZFS should provide a method of bleaching a disk or part
> > of it that works without crypto having ever been involved.
> 
> I see two use cases here:
I agree with your two, but I think I see a third use case in Darren''s
example: bleaching disks as they are removed from a pool.

We may need a second dimension controlling *how* to bleach..

						- Bill

Bill Sommerfeld wrote:
> On Tue, 2006-12-19 at 16:19 -0800, Matthew Ahrens wrote:
>> Darren J Moffat wrote:
>>> I believe that ZFS should provide a method of bleaching a disk or
part
>>> of it that works without crypto having ever been involved.
>> I see two use cases here:
> 
> I agree with your two, but I think I see a third use case in
Darren''s
> example: bleaching disks as they are removed from a pool.
That sounds plausible too.  (And you could implement it as ''zfs destroy
-r pool; zpool bleach pool''
> We may need a second dimension controlling *how* to bleach..
You mean whether we do single overwrite with zeros, muliple overwrites 
with some crazy government-mandated patterns, etc, right?  That''s what
I
meant by "the value of the property can specify what type of bleach to 
use" (perhaps taking the metaphor a bit too far) for example, ''zfs
set
bleach=<how> <fs>''.  Like other properties, we would
provide "bleach=on"
which would choose a reasonable default.  We''d need something similar 
with ''zpool bleach'' (eg ''zpool bleach [-o
<how>] <pool>'').

--matt

Bill Sommerfeld wrote:
> There also may be a reason to do this when confidentiality isn''t
> required: as a sparse provisioning hack..
> 
> If you were to build a zfs pool out of compressed zvols backed by
> another pool, then it would be very convenient if you could run in a
> mode where freed blocks were overwritten by zeros when they were freed,
> because this would permit the underlying compressed zvol to free *its*
> blocks.
A very interesting observation.  Particularly given that I have just 
created such a configuration - with iSCSI in the "middle".

-- 
Darren J Moffat

Matthew Ahrens wrote:
> Darren J Moffat wrote:
>> I believe that ZFS should provide a method of bleaching a disk or part 
>> of it that works without crypto having ever been involved.
> 
> I see two use cases here:
> 
> 1. This filesystem contains sensitive information.  When it is freed, 
> make sure it''s really gone.
> 
> I believe that the proper way to do this is to introduce a new zfs 
> property (say, "bleach") which, when set, will cause any blocks
which
> are freed to be bleached.  The value of the property can specify the 
> type of bleach to use (ie, the overwrite pattern).
> 
> To ease administrative complexity, this property should probably only be 
> set at filesystem creation time (eg. ''zfs create -o 
> bleach=maximum-strength pool/fs''). Furthermore, ''zfs
promote <fs>''
> should not be run if the fs and its origin have different bleach 
> settings (because the ownership of the shared blocks would change). 
> However, clones do not in general pose any problems for this model 
> (because it''s always clear which fs owns each block, and only that
fs
> can free it).
> 
> 2. I''ve deleted some stuff at some point in the past, and now I
want to
> make sure that it''s really gone.
> 
> To do this, we should have a "zpool bleach" command which would 
> immediately go and bleach all unallocated blocks.
> 
> Note that if both these features are implemented, it might make sense to 
> allow changing the ''bleach'' property on a fs after it has
been created,
> and have that implicitly kick off a ''zpool bleach'' to
ensure that any
> space previously used for that fs is bleached.
> 
> Also note that these bleaching features would not change the semantics 
> of snapshots, clones, etc.  A block will not be bleached until all 
> references to it are removed, and the block is freed.
You have it in a nutshell as far as I''m concerned, nice simplification 
that we only need this at the ZFS level and at the pool level.


-- 
Darren J Moffat

Not to add a cold blanket to this...

This would be mostly a "vanity erase" not really a serious
"security
erase" since it will not over write the remnants of remapped sectors.  
Serious security erase software will unmap sectors and erase both  
locations using special microcode features. While getting access to  
these remnants may be considered exotic measures, it is becoming less  
so with the plenitude vendor specific microcode features, and the  
number of data recovery organizations that use spin stands. Sectors  
can be remapped without the data being completely bad. It may still  
be readable with recoverable errors in the drive.

Jim


On Dec 20, 2006, at 1:41 AM, Darren J Moffat wrote:
> Bill Sommerfeld wrote:
>> There also may be a reason to do this when confidentiality
isn''t
>> required: as a sparse provisioning hack..
>> If you were to build a zfs pool out of compressed zvols backed by
>> another pool, then it would be very convenient if you could run in a
>> mode where freed blocks were overwritten by zeros when they were  
>> freed,
>> because this would permit the underlying compressed zvol to free  
>> *its*
>> blocks.
>
> A very interesting observation.  Particularly given that I have  
> just created such a configuration - with iSCSI in the "middle".
>
> -- 
> Darren J Moffat
> _______________________________________________
> security-discuss mailing list
> security-discuss at opensolaris.org

james hughes wrote:
> Not to add a cold blanket to this...
> 
> This would be mostly a "vanity erase" not really a serious
"security
> erase" since it will not over write the remnants of remapped sectors. 
Indeed and as you said there is other software to deal with this for 
those types of customers that need that.  There are also physical 
destruction methods as well.

This is intended as a defense in depth measure and also a sufficiently 
good measure for the customers that don''t need full compliance with
NIST
like requirements that need degausing or physical destruction.

It is intended to make customers more comfortable about handing disks 
back to their vendor.

Today we need to manually run format(1M)''s analyze/purge for that.

Are you saying that you don''t think this is sufficiently useful that we
should implement this in ZFS or are you just pointing out that for a 
some security policies this is not enough ?

-- 
Darren J Moffat

On Tue, Dec 19, 2006 at 02:04:37PM +0000, Darren J Moffat
wrote:
> In case it wasn''t clear I am NOT proposing a UI like this:
> 
> $ zfs bleach ~/Documents/company-finance.odp
> 
> Instead ~/Documents or ~ would be a ZFS file system with a policy set
something like this:
> 
> # zfs set erase=file:zero
> 
> Or maybe more like this:
> 
> # zfs create -o erase=file -o erasemethod=zero homepool/darrenm
> 
> The goal is the same as the goal for things like compression in ZFS, no
application change it is "free" for the applications.
I like the idea, I really do, but it will be soooo expensive because of
ZFS'' COW model. Not only file removal or truncation will call
bleaching,
but every single file system modification... Heh, well, if privacy of
your data is important enough, you probably don''t care too much about
performance. I for one would prefer encryption, which may turns out to be
much faster than bleaching and also more secure.

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20061220/0934b858/attachment.bin>

On Wed, 20 Dec 2006, Pawel Jakub Dawidek wrote:
> On Tue, Dec 19, 2006 at 02:04:37PM +0000, Darren J Moffat wrote:
>> In case it wasn''t clear I am NOT proposing a UI like this:
>>
>> $ zfs bleach ~/Documents/company-finance.odp
>>
>> Instead ~/Documents or ~ would be a ZFS file system with a policy set
something like this:
>>
>> # zfs set erase=file:zero
>>
>> Or maybe more like this:
>>
>> # zfs create -o erase=file -o erasemethod=zero homepool/darrenm
>>
>> The goal is the same as the goal for things like compression in ZFS, no
application change it is "free" for the applications.
>
> I like the idea, I really do, but it will be soooo expensive because of
> ZFS'' COW model. Not only file removal or truncation will call
bleaching,
> but every single file system modification... Heh, well, if privacy of
> your data is important enough, you probably don''t care too much
about
> performance. I for one would prefer encryption, which may turns out to be
> much faster than bleaching and also more secure.
And this kind of "deep bleaching" would also break if you use
snapshots -
how do you reliably bleach if you need to keep the all of the old data 
around ? You only could do so once the last snapshot is gone. Kind of 
defeating the idea - automatic but delayed indefinitely till operator 
intervention (deleting the last snapshot).

FrankH.

Pawel Jakub Dawidek writes:
> > The goal is the same as the goal for things like compression in ZFS,
no application change it is "free" for the applications.
> 
> I like the idea, I really do, but it will be soooo expensive because of
> ZFS'' COW model. Not only file removal or truncation will call
bleaching,
> but every single file system modification... Heh, well, if privacy of
> your data is important enough, you probably don''t care too much
about
> performance. I for one would prefer encryption, which may turns out to be
> much faster than bleaching and also more secure.
I think the idea here is that since ZFS encourages the use of lots of
small file systems (rather than one or two very big ones), you''d have
just one or two very small file systems with crucial data marked as
needing bleach, while the others would get by with the usual
complement of detergent and switch fabric softener.

Having _every_ file modification result in dozens of I/Os would
probably be bad, but perhaps not if it''s not _every_ modification
that''s affected.

-- 
James Carlson, KISS Network                    <james.d.carlson at
sun.com>
Sun Microsystems / 1 Network Drive         71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

On Dec 20, 2006, at 04:41, Darren J Moffat wrote:
> Bill Sommerfeld wrote:
>> There also may be a reason to do this when confidentiality
isn''t
>> required: as a sparse provisioning hack..
>> If you were to build a zfs pool out of compressed zvols backed by
>> another pool, then it would be very convenient if you could run in a
>> mode where freed blocks were overwritten by zeros when they were  
>> freed,
>> because this would permit the underlying compressed zvol to free  
>> *its*
>> blocks.
>
> A very interesting observation.  Particularly given that I have  
> just created such a configuration - with iSCSI in the "middle".
over ipsec?  wow - how many layers is that before you start talking  
to the real (non-psuedo) block storage device?

---
.je

Jonathan Edwards wrote:
>
> On Dec 20, 2006, at 04:41, Darren J Moffat wrote:
>
>> Bill Sommerfeld wrote:
>>> There also may be a reason to do this when confidentiality
isn''t
>>> required: as a sparse provisioning hack..
>>> If you were to build a zfs pool out of compressed zvols backed by
>>> another pool, then it would be very convenient if you could run in
a
>>> mode where freed blocks were overwritten by zeros when they were
freed,
>>> because this would permit the underlying compressed zvol to free
*its*
>>> blocks.
>>
>> A very interesting observation.  Particularly given that I have just 
>> created such a configuration - with iSCSI in the "middle".
>
> over ipsec?  wow - how many layers is that before you start talking to 
> the real (non-psuedo) block storage device?
>
It''s turtles all the way down.

On Wed, 2006-12-20 at 03:21 -0800, james hughes wrote:
> This would be mostly a "vanity erase" not really a serious
"security
> erase" since it will not over write the remnants of remapped sectors.
Yup.  As usual, your milage will vary depending on your threat model.  

My gut feel is that there''s a cost-benefit sweet spot near a mechanism
which provides for the prompt overwrite of recently deallocated blocks
with either zeros or newly allocated data, with more intensive bleaching
reserved for when disks are taken out of service. 

						- Bill

On Dec 20, 2006, at 1:37 PM, Bill Sommerfeld wrote:
> On Wed, 2006-12-20 at 03:21 -0800, james hughes wrote:
>> This would be mostly a "vanity erase" not really a serious
"security
>> erase" since it will not over write the remnants of remapped
sectors.
>
> Yup.  As usual, your milage will vary depending on your threat model.
>
> My gut feel is that there''s a cost-benefit sweet spot near a
mechanism
> which provides for the prompt overwrite of recently deallocated blocks
> with either zeros or newly allocated data,
What happens when the machine crashes after the blocks are  
deallocated but before they are scrubbed? Is that covered?
> with more intensive bleaching
> reserved for when disks are taken out of service.
If I had a choice of destroying disks or running a program that will  
take hours to complete (with dubious quality), I would (and do)  
choose to destroy the disk.

> 						- Bill
>
>
>
>
>
>
>
>
>
>
>

On Dec 20, 2006, at 5:46 AM, Darren J Moffat wrote:
> james hughes wrote:
>> Not to add a cold blanket to this...
>> This would be mostly a "vanity erase" not really a serious  
>> "security erase" since it will not over write the remnants of
>> remapped sectors.
>
> Indeed and as you said there is other software to deal with this  
> for those types of customers that need that.  There are also  
> physical destruction methods as well.
>
> This is intended as a defense in depth measure and also a  
> sufficiently good measure for the customers that don''t need full  
> compliance with NIST like requirements that need degausing or  
> physical destruction.
Govt, finance, healthcare all require the NIST overwrite...
> It is intended to make customers more comfortable about handing  
> disks back to their vendor.
These are the people that have the tools to get the data back.
> Today we need to manually run format(1M)''s analyze/purge for that.
Most banks do not return the disks, they return the top plate to get  
the warrantee credit and then just keep the disks...
> Are you saying that you don''t think this is sufficiently useful  
> that we should implement this in ZFS or are you just pointing out  
> that for a some security policies this is not enough ?
I think more the former. Lets also discuss who this policy will be  
enough for.

The load on the system may be as large as encrypting the data if you  
purge all files, and if you don''t then you have the  problem of  
finding all former copies of the data.

The complexity of implementation may be on par with encryption.

The caveats that need to be placed in the man pages on when this is  
not enough are problematic, and if the customer doesn''t read it...

It just seems to be a lot of work for not a lot of benefit.

My mind is not make up here, so these discussions are good...
> -- 
> Darren J Moffat
> _______________________________________________
> security-discuss mailing list
> security-discuss at opensolaris.org

Darren Reed wrote:
> Darren,
>
> A point I don''t yet believe that has been addressed in this
> discussion is: what is the threat model?
>
> Are we targetting NIST requirements for some customers
> or just general use by everyday folks?
Even higher level: What problem are you/we trying to solve?

Darren,

A point I don''t yet believe that has been addressed in this
discussion is: what is the threat model?

Are we targetting NIST requirements for some customers
or just general use by everyday folks?

Darrn

Pawel Jakub Dawidek wrote:
> I like the idea, I really do, but it will be soooo expensive because of
> ZFS'' COW model. Not only file removal or truncation will call
bleaching,
> but every single file system modification... Heh, well, if privacy of
> your data is important enough, you probably don''t care too much
about
> performance. 
I''m not sure it will be that slow, the bleaching will be done in a 
separate (new) transaction group in most (probably all) cases anyway so 
it shouldn''t really impact your write performance unless you are very 
I/O bound and already running near the limit.  However this is 
speculation until someone tries to implement this!
> I for one would prefer encryption, which may turns out to be
> much faster than bleaching and also more secure.
At least NIST, under I believe the guidance of the NSA, does not 
consider that encryption and key destruction alone is sufficient in all 
cases.  Which is why I''m proposing this as complementary.

-- 
Darren J Moffat

Frank Hofmann wrote:
> And this kind of "deep bleaching" would also break if you use
snapshots
> - how do you reliably bleach if you need to keep the all of the old data 
> around ? You only could do so once the last snapshot is gone. Kind of 
> defeating the idea - automatic but delayed indefinitely till operator 
> intervention (deleting the last snapshot).
Right that doesn''t break snapshots at all in fact it is working exactly
like snapshots work today anyway.  With user delegation (when is this 
integrating BTW?) the file system operator might be the end user anyway.

-- 
Darren J Moffat

On Thu, Dec 21, 2006 at 03:31:59PM +0000, Darren J Moffat
wrote:
> Pawel Jakub Dawidek wrote:
> >I like the idea, I really do, but it will be soooo expensive because of
> >ZFS'' COW model. Not only file removal or truncation will call
bleaching,
> >but every single file system modification... Heh, well, if privacy of
> >your data is important enough, you probably don''t care too
much about
> >performance. 
> 
> I''m not sure it will be that slow, the bleaching will be done in a
> separate (new) transaction group in most (probably all) cases anyway so 
> it shouldn''t really impact your write performance unless you are
very
> I/O bound and already running near the limit.  However this is 
> speculation until someone tries to implement this!
Yes, bleaching lazily would help with performance.  You might even delay
bleaching for very long periods of time if you want, as long as there''s
an interface by which to request that all outstanding free-but-not-yet-
bleached blocks be bleached immediately and synchronously.
> >I for one would prefer encryption, which may turns out to be
> >much faster than bleaching and also more secure.
> 
> At least NIST, under I believe the guidance of the NSA, does not 
> consider that encryption and key destruction alone is sufficient in all 
> cases.  Which is why I''m proposing this as complementary.
James makes a good argument that this scheme won''t suffice for
customers
who need that level of assurance.  I''m inclined to agree.  For
customers
who don''t need that level of assurance then encryption ought to
suffice.

Nico
--

james hughes wrote:
> 
> On Dec 20, 2006, at 1:37 PM, Bill Sommerfeld wrote:
> 
>> On Wed, 2006-12-20 at 03:21 -0800, james hughes wrote:
>>> This would be mostly a "vanity erase" not really a
serious "security
>>> erase" since it will not over write the remnants of remapped
sectors.
>>
>> Yup.  As usual, your milage will vary depending on your threat model.
>>
>> My gut feel is that there''s a cost-benefit sweet spot near a
mechanism
>> which provides for the prompt overwrite of recently deallocated blocks
>> with either zeros or newly allocated data,
> 
> What happens when the machine crashes after the blocks are deallocated 
> but before they are scrubbed? Is that covered?
I would hope so, and I believe this would fall out from the all ways 
consistent on disk transactional nature of ZFS.
>> with more intensive bleaching
>> reserved for when disks are taken out of service.
> 
> If I had a choice of destroying disks or running a program that will 
> take hours to complete (with dubious quality), I would (and do) choose 
> to destroy the disk.
Not all customers can make that same choice though.  Some times you need 
to give the broken disk back to the vendor as part of the replacement 
otherwise you are buying a new disk.

We aren''t talking about very high security environments here, and even 
in those the NIST recommendations suggest you do something like this AND 
  physically destroy the disk.

This is more targeted at the financial, education and home user worlds 
than military or other high sensitive environments.

It is all about using the appropriate tool given the risks you are 
willing to take for a given cost of physical and time resources.
-- 
Darren J Moffat

Torrey McMahon wrote:
> Darren Reed wrote:
>> Darren,
>>
>> A point I don''t yet believe that has been addressed in this
>> discussion is: what is the threat model?
>>
>> Are we targetting NIST requirements for some customers
>> or just general use by everyday folks?
> 
> Even higher level: What problem are you/we trying to solve?
Solving the erase that data form this disk problem while leaving the 
disk usable.  Without requiring it be done to the whole disk, or even 
whole "disk like thing" that format(1M) can see.

-- 
Darren J Moffat

Darren Reed wrote:
> Darren,
> 
> A point I don''t yet believe that has been addressed in this
> discussion is: what is the threat model?
There are several and this is about providing functionality so that 
customers can choose what they want to use when it is appropriate.

Using format(1M) for whole "disk" erasure isn''t always
suitable or even
possible.
> Are we targetting NIST requirements for some customers
> or just general use by everyday folks?
Both, NIST requires disk overwrite in addition to physical destruction 
in some cases.

-- 
Darren J Moffat

Nicolas Williams wrote:
> James makes a good argument that this scheme won''t suffice for
customers
> who need that level of assurance.  I''m inclined to agree.  For
customers
> who don''t need that level of assurance then encryption ought to
suffice.
Has anyone other than me actually read the current NIST guidelines on 
this ? [ the url was in my original email message ].

The current NIST guidelines, or at least my reading of it, says that 
even if you are using encryption and even if you are going to do 
physical destruction you still need to do something like this.

So this is complementary to encrypting the data - not that we can''t in 
ZFS encryption ALL ZFS metadata (we should be able to encrypt all the 
file system relevant meta data) at least thats my current belief based 
on my knowledge of ZFS.

Maybe doing this in ZFS isn''t necessary and what we have with
format(1M)
  purge/analyze is the correct user interface.

-- 
Darren J Moffat

On Thu, Dec 21, 2006 at 03:47:07PM +0000, Darren J Moffat
wrote:
> Nicolas Williams wrote:
> >James makes a good argument that this scheme won''t suffice for
customers
> >who need that level of assurance.  I''m inclined to agree.  For
customers
> >who don''t need that level of assurance then encryption ought
to suffice.
> 
> Has anyone other than me actually read the current NIST guidelines on 
> this ? [ the url was in my original email message ].
> 
> The current NIST guidelines, or at least my reading of it, says that 
> even if you are using encryption and even if you are going to do 
> physical destruction you still need to do something like this.
I think it''s a bit nuanced.

Pages 15-16 obliquely mention encryption in the description of
"clearing":

"... It must be resistant to keystore recovery attempts executed from
standard input devices and from data scavenging tools.  ..."

I''m not sure how to interpret that in the case of ZFS encryption.  The
actual keys used to encrypt file are not typed in by users, and data
scavenging tools could only get at them if: a) they recovered user
passwords from which master FS keys are derived, b) have access to the
media.

On page 4 (errata), it says that on 9-11-06 (version 10-06) text was
deleted that had once declared encryption insufficient.

So, altogether I would read this as allowing deletion of keys as a
method of clearing.

Since clearing is all we can hope to do in ZFS then I think this should
be sufficient.

Nico
--

One other area where is is useful is when you are in a jurisdiction 
where a court order may require you to produce your encryption keys - 
yes such jurisdictions exist and I don''t want to debate the "human
rights" angle or social engineering aspects of this just state that it 
exists.

For such environments you may not want to use encryption, because you 
could be forced to give up your key, or even if you are you want a 
background method of destroying the cipher text without doing full disk 
destruction.

Think of court cases between companies rather than criminal activity.

--
Darren J Moffat

Darren J Moffat wrote:
> One other area where is is useful is when you are in a jurisdiction 
> where a court order may require you to produce your encryption keys - 
> yes such jurisdictions exist and I don''t want to debate the
"human
> rights" angle or social engineering aspects of this just state that it
> exists.

I think in these cases you want plausable deniability where different
encryption keys produce different view of the disk, none of which
give away that there are any other correct views of the data.

If it is possible to destroy a small piece of the ZFS meta data (key
material) and that makes it thereafter impossible to encrypt data,
sure, but otherwise, bleaching is probably going to take a bit too
long once you hear the knock on the door...

> For such environments you may not want to use encryption, because you 
> could be forced to give up your key, or even if you are you want a 
> background method of destroying the cipher text without doing full 
> disk destruction.
>
> Think of court cases between companies rather than criminal activity.

For corporations there are different requirements, for examples laws
that regulate data retention.  Not only this but you also need to make
sure that the data you want to make "unavailable" never got backed
up or that those backups get wiped...

Darren

On Jan 2, 2007, at 6:48 AM, Darren Reed wrote:
> Darren J Moffat wrote:
>
>> ...
>> Of course.  I didn''t mention it because I thought it was
obvious
>> but this would NOT break the COW or the transactional integrity of  
>> ZFS.
>>
>> One of the possible ways that the "to be bleached" blocks are
>> dealt with in the face of a crash is just like everything else -  
>> they would be in the ZFS Intent Log as "things to do".
>
>
> Do NIST and other specifications that come into play here dictate
> what should be done in these and other situations?
From
	http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf
includes the statement that
	The security goal of the overwriting process is to replace written  
data with random data.

How we achieve this is our problem. I expect that this is a  
subjective analysis if we meet the goals...
> Do they say how this feature must be provided or in which situations
> it is required to be covered in order to meet their criteria.
They talk about "Clearing" vs "Purging" vs
"Destruction"... Clearing
is the lowest level and seems to be useful when repurposing storage  
within an organization. All of these are supposed to be "NSA/CSS  
Approved". I do not see the exact approval requirements in this  
document.
> Or do they just document overwrite patterns, depending on the
> security of the information?
From
	http://csrc.nist.gov/publications/nistpubs/800-88/NISTSP800-88_rev1.pdf
it states
	Studies have shown that most of today?s media can be effectively  
cleared by one overwrite.
> Darren
>
> _______________________________________________
> security-discuss mailing list
> security-discuss at opensolaris.org

On Tue, 2006-12-26 at 14:01 +0300, Victor Latushkin wrote:
> What happens if fatal failure occurs after the txg which frees blocks
> have been written but before before txg doing bleaching will be
> started/completed?
clearly you''d need to store the "unbleached" list
persistently in the
pool.

transactions which freed blocks (by punching holes in the allocation
space map) would instead or additionally move them to the unbleached
list; a separate bleaching task queue would pick blocks off the
unbleached list, bleach them; only once bleaching was complete would
they be removed from the unbleached list.

In the face of a crash, some blocks might get bleached twice.

						- Bill





_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Darren J Moffat wrote:
> Pawel Jakub Dawidek wrote:
>> I like the idea, I really do, but it will be soooo expensive because of
>> ZFS'' COW model. Not only file removal or truncation will call
bleaching,
>> but every single file system modification... Heh, well, if privacy of
>> your data is important enough, you probably don''t care too
much about
>> performance.
>
> I''m not sure it will be that slow, the bleaching will be done in a
> separate (new) transaction group in most (probably all) cases anyway so
> it shouldn''t really impact your write performance unless you are
very
> I/O bound and already running near the limit.  However this is
> speculation until someone tries to implement this!
>
What happens if fatal failure occurs after the txg which frees blocks
have been written but before before txg doing bleaching will be
started/completed?
>> I for one would prefer encryption, which may turns out to be
>> much faster than bleaching and also more secure.
>
> At least NIST, under I believe the guidance of the NSA, does not
> consider that encryption and key destruction alone is sufficient in all
> cases.  Which is why I''m proposing this as complementary.
>
True, dropping the keys leaves lots of encrypted material for determined
cryptoanalytic to analyze, so it should be bleached in some good way.

Victor
_______________________________________________
security-discuss mailing list
security-discuss@opensolaris.org

On Wed, Dec 27, 2006 at 08:45:23AM -0500, Bill Sommerfeld
wrote:
> I think your paranoia is indeed running a bit high if the alternative is
> that some blocks escape bleaching "forever" when they were freed
shortly
> before a crash.
Lazy bg bleaching of freed blocks is not enough if you''re really
paranoid about deleting things that might be cloned.  (See sub-thread
about bleach(2), which is off the table.)
> For a portable system, the risk of theft is highest when the laptop is
> unattended and idle -- and that''s the point where the bleaching
process
> would have time to catch up; most likely, the unbleached list would be
> small or empty..
For portable systems the risk is not the loss of unbleached freed blocks
-- it''s the loss of all those live blocks.  Thus you''d need
encryption.

But encryption''s still not enough if the system is stolen while the
keys
are in memory.

Nico
--
_______________________________________________
security-discuss mailing list
security-discuss@opensolaris.org

On Tue, 2006-12-26 at 13:59 -0500, Torrey McMahon wrote:
> > clearly you''d need to store the "unbleached" list
persistently in the
> > pool.
>
> Which could then be easily referenced to find all the blocks that were
> recently deleted but not yet bleached? Is my paranoia running a bit too
> high?
I think your paranoia is indeed running a bit high if the alternative is
that some blocks escape bleaching "forever" when they were freed
shortly
before a crash.

For a portable system, the risk of theft is highest when the laptop is
unattended and idle -- and that''s the point where the bleaching process
would have time to catch up; most likely, the unbleached list would be
small or empty..

						- Bill

_______________________________________________
security-discuss mailing list
security-discuss@opensolaris.org

Bill Sommerfeld wrote:
> On Tue, 2006-12-26 at 14:01 +0300, Victor Latushkin wrote:
>
>
>> What happens if fatal failure occurs after the txg which frees blocks
>> have been written but before before txg doing bleaching will be
>> started/completed?
>>
>
> clearly you''d need to store the "unbleached" list
persistently in the
> pool.
Which could then be easily referenced to find all the blocks that were
recently deleted but not yet bleached? Is my paranoia running a bit too
high?

_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Quoth Darren J Moffat on Thu, Dec 21, 2006 at 03:31:59PM
+0000:
> Pawel Jakub Dawidek wrote:
> >I like the idea, I really do, but it will be soooo expensive because of
> >ZFS'' COW model. Not only file removal or truncation will call
bleaching,
> >but every single file system modification... Heh, well, if privacy of
> >your data is important enough, you probably don''t care too
much about
> >performance.
>
> I''m not sure it will be that slow, the bleaching will be done in a
> separate (new) transaction group in most (probably all) cases anyway so
> it shouldn''t really impact your write performance unless you are
very
> I/O bound and already running near the limit.  However this is
> speculation until someone tries to implement this!
Bleaching previously used blocks will corrupt files pointed to by older
uberblocks.  I think that means that you''d have to verify that the new
uberblock is readable before you proceed, since part of ZFS''s fault
tolerance is falling back to the most recent good uberblock if the
latest one is corrupt.  I don''t think this makes bleaching unworkable,
but the interplay will require analysis.


David
_______________________________________________
security-discuss mailing list
security-discuss@opensolaris.org

james hughes wrote:
>> This is intended as a defense in depth measure and also a sufficiently
>> good measure for the customers that don''t need full compliance
with
>> NIST like requirements that need degausing or physical destruction.
>
> Govt, finance, healthcare all require the NIST overwrite...
Jim,

Do customers who are in business of wiping out all the deleted
data need an ability to wipe out individual files or could they
live with a periodic barrier-type of operation "wipe-out all
the unallocated but previously used blocks"?

In other words, wouldn''t it be sufficient, and maybe even more
practical, to keep deletion virtually unchanged, and introduce
a barrier-type operation (an explicit command, which could be
just an option of a disk scrubber) to wipe out all the deallocated
but previously instantiated blocks in a given pool (including
wiping out remapped sectors, possibly only in certain hardware
configurations that allow for such an operation).

It seems to me that overwriting file data upon a delete or
copy-on-write update to a file would not only introduce a lot
of complexity to ZFS but also it is likely hurt performance
substantially.

-- Olaf


_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Darren Reed wrote:
> Darren J Moffat wrote:
>
>> ...
>> Of course.  I didn''t mention it because I thought it was
obvious but
>> this would NOT break the COW or the transactional integrity of ZFS.
>>
>> One of the possible ways that the "to be bleached" blocks are
dealt
>> with in the face of a crash is just like everything else - they would
>> be in the ZFS Intent Log as "things to do".
>
>
> Do NIST and other specifications that come into play here dictate
> what should be done in these and other situations?
Not that I have found.
> Do they say how this feature must be provided or in which situations
> it is required to be covered in order to meet their criteria?
The cover what should be done to the data now how it should be implemented.
> Or do they just document overwrite patterns, depending on the
> security of the information?
More like that and they document what type of physical destruction or
other operations need to be done such as degausing etc.


--
Darren J Moffat
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Good point.  Verifying that the new ?berblock is readable isn?t actually
sufficient, since it might become unreadable in the future.  You?d need to wait
for several transaction groups, until the block was unreachable by the oldest
remaining ?berblock, to be safe in this sense.

On the other hand, one could make an argument that since the file had been
deleted, it would be reasonable for its data to be unreadable after a crash.


This message posted from opensolaris.org
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

David Bustos wrote:
> Quoth Darren J Moffat on Thu, Dec 21, 2006 at 03:31:59PM +0000:
>> Pawel Jakub Dawidek wrote:
>>> I like the idea, I really do, but it will be soooo expensive
because of
>>> ZFS'' COW model. Not only file removal or truncation will
call bleaching,
>>> but every single file system modification... Heh, well, if privacy
of
>>> your data is important enough, you probably don''t care too
much about
>>> performance.
>> I''m not sure it will be that slow, the bleaching will be done
in a
>> separate (new) transaction group in most (probably all) cases anyway so
>> it shouldn''t really impact your write performance unless you
are very
>> I/O bound and already running near the limit.  However this is
>> speculation until someone tries to implement this!
>
> Bleaching previously used blocks will corrupt files pointed to by older
> uberblocks.  I think that means that you''d have to verify that the
new
> uberblock is readable before you proceed, since part of ZFS''s
fault
> tolerance is falling back to the most recent good uberblock if the
> latest one is corrupt.  I don''t think this makes bleaching
unworkable,
> but the interplay will require analysis.
Of course.  I didn''t mention it because I thought it was obvious but
this would NOT break the COW or the transactional integrity of ZFS.

One of the possible ways that the "to be bleached" blocks are dealt
with
in the face of a crash is just like everything else - they would be in
the ZFS Intent Log as "things to do".

--
Darren J Moffat
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Darren J Moffat wrote:
> ...
> Of course.  I didn''t mention it because I thought it was obvious
but
> this would NOT break the COW or the transactional integrity of ZFS.
>
> One of the possible ways that the "to be bleached" blocks are
dealt
> with in the face of a crash is just like everything else - they would
> be in the ZFS Intent Log as "things to do".

Do NIST and other specifications that come into play here dictate
what should be done in these and other situations?

Do they say how this feature must be provided or in which situations
it is required to be covered in order to meet their criteria?

Or do they just document overwrite patterns, depending on the
security of the information?

Darren

_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Anton B. Rang wrote:
> Good point.  Verifying that the new ?berblock is readable isn?t actually
sufficient, since it might become unreadable in the future.  You?d need to wait
for several transaction groups, until the block was unreachable by the oldest
remaining ?berblock, to be safe in this sense.
>
> On the other hand, one could make an argument that since the file had been
deleted, it would be reasonable for its data to be unreadable after a crash.
The data would be unreadable because the ZIL would be used and the file
would be marked as deleted.  The bleaching transaction would be in the
ZIL as well.

--
Darren J Moffat
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss