Btrfs devel - Mar 2013 - Impossible or Possible to Securely Erase File on Btrfs?

Hi,

After reading through the btrfs documentation I''m curious to know if 
it''s possible to ever securely erase a file from a btrfs filesystem (or
ZFS for that matter). On non-COW filesystems atop regular HDDs one can 
simply overwrite the file with zeros or random data using dd or some 
other tool and rest assured that the blocks which contained the 
sensitive information have been wiped. However on btrfs it would seem 
any such attempt would write the zeros/random data to a new location, 
leaving the old blocks with the sensitive data intact. Further, since 
specifying NOCOW is only possible for newly created files, there seems 
to be no way to overwrite the appropriate blocks short of deleting the 
associated file and then filling the entire free filesystem space with 
zeros/random data such that the old blocks are eventually overwritten. 
What''s the verdict on this?

Regards,

Kyle
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mon, Mar 18, 2013 at 02:15:17PM -0400, Kyle wrote:
> After reading through the btrfs documentation I''m curious to know
if
> it''s possible to ever securely erase a file from a btrfs
filesystem
> (or ZFS for that matter).
   Not really.

   It gets even worse for SSDs, because the SSD itself can be
effectively CoW, with old pages lurking away in the flash storage
where (with a bit of physical persuasion of the hardware) they can be
read. So you have the same problem on SSDs even with non-CoW
filesystems.
> On non-COW filesystems atop regular HDDs one can simply overwrite
> the file with zeros or random data using dd or some other tool and
> rest assured that the blocks which contained the sensitive
> information have been wiped.
   Assuming that anything that''s modified the file since its creation
hasn''t written a copy and then overwritten it with rename(), of
course...
> However on btrfs it would seem any such attempt would write the
> zeros/random data to a new location, leaving the old blocks with the
> sensitive data intact. Further, since specifying NOCOW is only
> possible for newly created files, there seems to be no way to
> overwrite the appropriate blocks short of deleting the associated
> file and then filling the entire free filesystem space with
> zeros/random data such that the old blocks are eventually
> overwritten. What''s the verdict on this?
   Your analysis is pretty much correct, but should be expanded to a
whole load of other data-leakage paths in generalised storage systems.
Overwriting file contents hasn''t really been a reliable method of
erasing files for many years (if ever).

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
   --- Great oxymorons of the world, no.  3: Military Intelligence ---

On Mar 18, 2013, at 12:57 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> On Mon, Mar 18, 2013 at 02:15:17PM -0400, Kyle wrote:
>> After reading through the btrfs documentation I''m curious to
know if
>> it''s possible to ever securely erase a file from a btrfs
filesystem
>> (or ZFS for that matter).
> 
>   Not really.
> 
>   It gets even worse for SSDs, because the SSD itself can be
> effectively CoW, with old pages lurking away in the flash storage
> where (with a bit of physical persuasion of the hardware) they can be
> read. So you have the same problem on SSDs even with non-CoW
> filesystems.
Encryption is the solution, either dmcrypt, encryptfs, or Opal SED.

With even consumer SSDs now, Opal 2.0 is increasingly common, so data on the SSD
is already encrypted whether locked or not. By default the drive is unlocked, so
the DEK is exposed but data on the chips is encrypted. Unfortunately
I''m not aware of user space tools to manage Opal drives on linux,
it''s a side topic question but if anyone knows of such a tool
I''d like to know about it. I don''t think hdparm does, rather
it manages the ATA  Security features like Secure Erase, Secure Enhanced Erase,
and I think it can also cause DEK regeneration on SED drives which effectively
is encryption erase (removes the existing DEK, creates a new one, thereby
rendering the drive effectively erased in seconds). But I don''t think
it manages KEKs and the locking/unlocking of the drive.

Booting from these Opal compliant SEDs is difficult because it requires firmware
or bootloader support; and then also kernel and user space tools to support to
lock/unlock when the computer is put to sleep/hibernation and then woken. For
that use case it''s necessary to use software encryption, dmcrypt/LUKS
on Linux.

Another possibility for the OP''s use case is encryptfs which encrypts
above btrfs. The file contents are encrypted, but the file names and file system
itself (and metadata) are not. In this case, deletion is functionally equivalent
to having filled the blocks with random data (short of the DEK escaping into the
wild).

Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Quoting Kyle (2013-03-18 14:15:17)
> Hi,
> 
> After reading through the btrfs documentation I''m curious to know
if
> it''s possible to ever securely erase a file from a btrfs
filesystem (or
> ZFS for that matter). On non-COW filesystems atop regular HDDs one can 
> simply overwrite the file with zeros or random data using dd or some 
> other tool and rest assured that the blocks which contained the 
> sensitive information have been wiped. However on btrfs it would seem 
> any such attempt would write the zeros/random data to a new location, 
> leaving the old blocks with the sensitive data intact. Further, since 
> specifying NOCOW is only possible for newly created files, there seems 
> to be no way to overwrite the appropriate blocks short of deleting the 
> associated file and then filling the entire free filesystem space with 
> zeros/random data such that the old blocks are eventually overwritten. 
> What''s the verdict on this?
We don''t do this now for other reasons mentioned in the thread.  The
best path to get here is to use trim, and to find a device that supports
a secure erase trim (I don''t know if this even exists, sorry).

Outside of that, the only way to do this securely is to delete the
files, rotate in new drives, rotate out the old drives, and run a full
drive secure delete (not very practical).

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Would it make sense for btrfs to support a write zeros to empty space
erase? I know it would be slow as it would have to write to all the
free space in the file system but it would be useful.

It''s probably pretty far down the priority list for development though
I expect.

On Tue, Mar 19, 2013 at 6:18 AM, Chris Mason <chris.mason@fusionio.com>
wrote:
> Quoting Kyle (2013-03-18 14:15:17)
>> Hi,
>>
>> After reading through the btrfs documentation I''m curious to
know if
>> it''s possible to ever securely erase a file from a btrfs
filesystem (or
>> ZFS for that matter). On non-COW filesystems atop regular HDDs one can
>> simply overwrite the file with zeros or random data using dd or some
>> other tool and rest assured that the blocks which contained the
>> sensitive information have been wiped. However on btrfs it would seem
>> any such attempt would write the zeros/random data to a new location,
>> leaving the old blocks with the sensitive data intact. Further, since
>> specifying NOCOW is only possible for newly created files, there seems
>> to be no way to overwrite the appropriate blocks short of deleting the
>> associated file and then filling the entire free filesystem space with
>> zeros/random data such that the old blocks are eventually overwritten.
>> What''s the verdict on this?
>
> We don''t do this now for other reasons mentioned in the thread. 
The
> best path to get here is to use trim, and to find a device that supports
> a secure erase trim (I don''t know if this even exists, sorry).
>
> Outside of that, the only way to do this securely is to delete the
> files, rotate in new drives, rotate out the old drives, and run a full
> drive secure delete (not very practical).
>
> -chris
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gareth@cerberos.id.au - www.rockpaperdynamite.wordpress.com
"Dear God, I would like to file a bug report"
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 3/18/2013 3:09 PM, Chris Murphy wrote:
>
> On Mar 18, 2013, at 12:57 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
>
>> On Mon, Mar 18, 2013 at 02:15:17PM -0400, Kyle wrote:
>>> After reading through the btrfs documentation I''m curious
to know if
>>> it''s possible to ever securely erase a file from a btrfs
filesystem
>>> (or ZFS for that matter).
>>
>>    Not really.
>>
>>    It gets even worse for SSDs, because the SSD itself can be
>> effectively CoW, with old pages lurking away in the flash storage
>> where (with a bit of physical persuasion of the hardware) they can be
>> read. So you have the same problem on SSDs even with non-CoW
>> filesystems.
>
> Encryption is the solution, either dmcrypt, encryptfs, or Opal SED.
>
> With even consumer SSDs now, Opal 2.0 is increasingly common, so data on
the SSD is already encrypted whether locked or not. By default the drive is
unlocked, so the DEK is exposed but data on the chips is encrypted.
Unfortunately I''m not aware of user space tools to manage Opal drives
on linux, it''s a side topic question but if anyone knows of such a tool
I''d like to know about it. I don''t think hdparm does, rather
it manages the ATA  Security features like Secure Erase, Secure Enhanced Erase,
and I think it can also cause DEK regeneration on SED drives which effectively
is encryption erase (removes the existing DEK, creates a new one, thereby
rendering the drive effectively erased in seconds). But I don''t think
it manages KEKs and the locking/unlocking of the drive.
>
> Booting from these Opal compliant SEDs is difficult because it requires
firmware or bootloader support; and then also kernel and user space tools to
support to lock/unlock when the computer is put to sleep/hibernation and then
woken. For that use case it''s necessary to use software encryption,
dmcrypt/LUKS on Linux.
>
> Another possibility for the OP''s use case is encryptfs which
encrypts above btrfs. The file contents are encrypted, but the file names and
file system itself (and metadata) are not. In this case, deletion is
functionally equivalent to having filled the blocks with random data (short of
the DEK escaping into the wild).
>
> Chris Murphy
>
I''m doubtful that encryption would be as effective as what Chris Mason 
was suggesting, which is to either find and use a disk with secure erase 
trim or rotate out the drives and securely erase them so that the data 
is destroyed entirely rather than encrypted.

Simply having the data exist on a disk, even if it''s encrypted, still 
leaves it open to various attacks. For instance, multiple side channel 
attacks exist which could reveal the decryption key, permitting all data 
on the disk to be read including deleted but not yet overwritten files. 
Physical access to the machine may not even be required: A remote 
attacker able to exploit the machine and read the disk blocks would be 
able to recover the data as well.

Granted these types of attacks are rare, however ultimately the only way 
to secure unneeded sensitive data against a determined attacker is to 
destroy it outright. For encrypted data, it''s true the decryption keys 
could be destroyed, thus essentially destroying the data (subject to the 
security of the cryptographic algorithm itself) however this would 
destroy the entire filesystem, not securely erase a single file.

Hugo made the point that a multitude of other data-leakage paths exist, 
and while this is true, at a minimum one should be able to destroy 
individual files beyond recovery so that should data leakage occur in 
the future, previously securely erased data is not recoverable. At 
least, in an ideal world.

For the time being I''ll be thankful I''m not a person tasked
with
protecting/erasing such highly sensitive data. Aside from the 
confidence-inspiring crunching of decommissioned hard disks going 
through an industrial shredder, it must be a nightmare.

-Kyle
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mar 18, 2013, at 1:18 PM, Chris Mason <chris.mason@fusionio.com>
wrote:
> 
> We don''t do this now for other reasons mentioned in the thread. 
The
> best path to get here is to use trim, and to find a device that supports
> a secure erase trim (I don''t know if this even exists, sorry).
I''m not finding a reference to "secure erase trim". In any
case I wouldn''t expect it to differ from trim. The requirement for ATA
SECURITY ERASE UNIT is to replace LBA 0 to READ NATIVE MAX contents with either
0s or 1s.

ENHANCED SECURITY ERASE UNIT additional requires data overwrite, including
sectors not in use. But there''s no requirement for what kind of data is
used for overwriting.

Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mar 18, 2013, at 5:00 PM, Kyle <lists@lolwut.org>
wrote:
> 
> I''m doubtful that encryption would be as effective as what Chris
Mason was suggesting, which is to either find and use a disk with secure erase
trim or rotate out the drives and securely erase them so that the data is
destroyed entirely rather than encrypted.
Well, I''ll argue it''s more effective in that it''s
more likely to be used than it is to be cracked in the time frame of the data
mattering if the proper setup is being used.
> 
> Simply having the data exist on a disk, even if it''s encrypted,
still leaves it open to various attacks. For instance, multiple side channel
attacks exist which could reveal the decryption key, permitting all data on the
disk to be read including deleted but not yet overwritten files.
Not if you''re using LUKS with either a high entropy passphrase, or
increasing PBKDF2 iterations, or both. LUKS and at least Opal SEDs
don''t store keys or passphrases in plaintext. If you''re using
plain dmcrypt (without LUKS) then you be concerned.

> Physical access to the machine may not even be required: A remote attacker
able to exploit the machine and read the disk blocks would be able to recover
the data as well.
If it''s an HDD, or the SSD''s garbage collection after trim is
issue is slow, yes anyone who has either file system or block level access can
obtain unencrypted data from a mapped encrypted block device. But if the fall
back position to a remote take over is counting on deleted files being
irrecoverable, well, you still have a remote attacker stealing visible files.
Chances are if the deleted files are that sensitive, the non-deleted files are
too. So I don''t see the gain of going to extraordinary lengths to zero
the data in released blocks for active use drives.
> 
> Granted these types of attacks are rare, however ultimately the only way to
secure unneeded sensitive data against a determined attacker is to destroy it
outright.
This is probably specious. It''s sorta like saying the only way to
secure a computer containing sensitive data is to disconnect it from the
internet. Or from the wall outlet.
> Hugo made the point that a multitude of other data-leakage paths exist, and
while this is true, at a minimum one should be able to destroy individual files
beyond recovery so that should data leakage occur in the future, previously
securely erased data is not recoverable. At least, in an ideal world.
Well at the moment we are SOL on that front with either SSDs or COW file
systems. You''d need to use HDDs and a file system that''ll
support overwriting files with random data and then marking them as deleted.

I think the overwhelming concern is someone gaining physical access to the
drive: theft of a laptop, or disposal of a dead drive (for which ATA security
erase unit commands aren''t going to work). And for that encryption is
quite useful and important.

Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mon, Mar 18, 2013 at 09:18:28PM -0600, Chris Murphy
wrote:
> I''m not finding a reference to "secure erase trim". In
any case I
> wouldn''t expect it to differ from trim.
There''s a ''secure discard'' trim command in linux,

http://lkml.indiana.edu/hypermail/linux/kernel/1007.1/02355.html

Used inside block layer
http://lxr.free-electrons.com/source/block/blk-core.c#L1738
(assumes the underlying device advertises the secure discard capability)

david
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Tue, Mar 19, 2013 at 09:48:56AM +1100, Gareth Pye
wrote:
> Would it make sense for btrfs to support a write zeros to empty space
> erase? I know it would be slow as it would have to write to all the
> free space in the file system but it would be useful.
> 
> It''s probably pretty far down the priority list for development
though I expect.
I have prototyped this some time ago

http://repo.or.cz/w/linux-2.6/btrfs-unstable.git/commit/55ec5c00022be8f2bbed06b99b5f4be5832a5451

It reuses the trim code that already traverses the free space but calls
blkdev_issue_zeroout instead. The overwrite pattern could be extended to
random data, some posion patterns (like leaf that has 0 items and could
be recognized in the code to catch misdirected writes; similarly for
tree node blocks), or the secure discard.

(I haven''t resolved what to do if there''s a mixture of HDDs
and SSDs,
rewriting the free space has different impact on the underlying
devices.)

The existing FITRIM ioctl structure fstrim_range does not contain any
spare bytes so either we need to go with own ioctl or propose FITRIM_V2.

david
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hi, 

just reading chattr manpage.. 

On Monday 18 March 2013 14:15:17 you wrote:
> Hi,
> 
> After reading through the btrfs documentation I''m curious to know
if
> it''s possible to ever securely erase a file from a btrfs
filesystem (or
> ZFS for that matter). On non-COW filesystems atop regular HDDs one can 
> simply overwrite the file with zeros or random data using dd or some 
> other tool and rest assured that the blocks which contained the 
> sensitive information have been wiped. However on btrfs it would seem 
> any such attempt would write the zeros/random data to a new location, 
> leaving the old blocks with the sensitive data intact. Further, since 
> specifying NOCOW is only possible for newly created files, there seems 
> to be no way to overwrite the appropriate blocks short of deleting the 
> associated file and then filling the entire free filesystem space with 
> zeros/random data such that the old blocks are eventually overwritten. 
> What''s the verdict on this?
what would chattr +s do? 

"
When a file with the `s'' attribute set is deleted, its blocks are
zeroed and written back to the disk.  Note: please make sure to read the bugs
and limitations section at the end of this document.
"

Nice spring to all of you! :)
Mark
> 
> Regards,
> 
> Kyle
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
-- 

Marek Otahal :o)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Mar 19, 2013, at 3:06 AM, David Sterba <dsterba@suse.cz> wrote:
> On Mon, Mar 18, 2013 at 09:18:28PM -0600, Chris Murphy wrote:
>> I''m not finding a reference to "secure erase trim".
In any case I
>> wouldn''t expect it to differ from trim.
> 
> There''s a ''secure discard'' trim command in
linux,
> 
> http://lkml.indiana.edu/hypermail/linux/kernel/1007.1/02355.html
"Secure discard is the same as discard except that all copies
of the discarded sectors (perhaps created by garbage collection)
must also be erased."

The erase mechanism seems to be the same "data set management" trim in
ACS-2 in either case.


Chris Murphy--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Am Dienstag, 19. März 2013 schrieb Marek Otahal:
> Hi,
> 
> just reading chattr manpage..
> 
> On Monday 18 March 2013 14:15:17 you wrote:
> > Hi,
> > 
> > After reading through the btrfs documentation I''m curious to
know if
> > it''s possible to ever securely erase a file from a btrfs
filesystem (or
> > ZFS for that matter). On non-COW filesystems atop regular HDDs one can
> > simply overwrite the file with zeros or random data using dd or some
> > other tool and rest assured that the blocks which contained the
> > sensitive information have been wiped. However on btrfs it would seem
> > any such attempt would write the zeros/random data to a new location,
> > leaving the old blocks with the sensitive data intact. Further, since
> > specifying NOCOW is only possible for newly created files, there seems
> > to be no way to overwrite the appropriate blocks short of deleting the
> > associated file and then filling the entire free filesystem space with
> > zeros/random data such that the old blocks are eventually overwritten.
> > What''s the verdict on this?
> 
> what would chattr +s do?
> 
> "
> When a file with the `s'' attribute set is deleted, its blocks are
zeroed
> and written back to the disk.  Note: please make sure to read the bugs
> and limitations section at the end of this document. "
> 
> Nice spring to all of you! :)
Did you read on as suggested?

BUGS AND LIMITATIONS
       The `c'', ''s'',  and `u'' attributes are
not honored by the ext2 and
ext3 filesystems as implemented in the current mainline Linux kernels.


But well question still stands: Does BTRFS honor it?

My bet is: It doesn´t.

Thanks,
-- 
Martin ''Helios'' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html