Two things, mostly related, that I''m trying to find answers to for our security team. Does this scenario make sense: * Create a filesystem at /users/nfsshare1, user uses it for a while, asks for the filesystem to be deleted * New user asks for a filesystem and is given /users/nfsshare2. What are the chances that they could use some tool or other to read unallocated blocks to view the previous user''s data? Related to that, when files are deleted on a ZFS volume over an NFS share, how are they wiped out? Are they zeroed or anything. Same question for destroying ZFS filesystems, does the data lay about in any way? (That''s largely answered by the first scenario.) If the data is retrievable in any way, is there a way to a) securely destroy a filesystem, or b) securely erase empty space on a filesystem. I know in some sense those questions don''t apply in the way they would to, say, ext3, since a filesystem doesn''t have a block until a file is written. Sorry if these questions aren''t worded well. I''ve been in meetings for the last couple hours. - Cameron Hanover chanover at umich.edu "Chaos was the law of nature. Order was the dream of man." --Henry Brooks Adams -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3682 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100205/e25f202e/attachment.bin>
On 2/5/10 3:49 PM -0500 c.hanover wrote:> Two things, mostly related, that I''m trying to find answers to for our > security team. > > Does this scenario make sense: > * Create a filesystem at /users/nfsshare1, user uses it for a while, asks > for the filesystem to be deleted * New user asks for a filesystem and is > given /users/nfsshare2. What are the chances that they could use some > tool or other to read unallocated blocks to view the previous user''s data?Over NFS? none.> Related to that, when files are deleted on a ZFS volume over an NFS > share, how are they wiped out? Are they zeroed or anything. Same > question for destroying ZFS filesystems, does the data lay about in any > way? (That''s largely answered by the first scenario.)In both cases the data is still on disk.> If the data is retrievable in any way, is there a way to a) securely > destroy a filesystem, or b) securely erase empty space on a filesystem.Someone else will have to answer that. -frank
On Fri, Feb 05, 2010 at 03:49:15PM -0500, c.hanover wrote:> Two things, mostly related, that I''m trying to find answers to for our > security team. > > Does this scenario make sense: > * Create a filesystem at /users/nfsshare1, user uses it for a while, > asks for the filesystem to be deleted > * New user asks for a filesystem and is given /users/nfsshare2. What > are the chances that they could use some tool or other to read > unallocated blocks to view the previous user''s data?If the tool isn''t accessing the raw disks, then the answer is "no chance". (There''s no way to access the raw disks over NFS.)> Related to that, when files are deleted on a ZFS volume over an NFS > share, how are they wiped out? Are they zeroed or anything. Same > question for destroying ZFS filesystems, does the data lay about in > any way? (That''s largely answered by the first scenario.)Deleting a file does not guarantee that data blocks are released: snapshots might exist that retain references to the data blocks of a file that is being deleted. Nor are blocks wiped when released.> If the data is retrievable in any way, is there a way to a) securely > destroy a filesystem, or b) securely erase empty space on a > filesystem.When ZFS crypto ships you''ll be able to securely destroy encrypted datasets. Until then the only form of secure erasure is to destroy the pool and then wipe the individual disks. Nico --
>>>>> "ch" == c hanover <chanover at umich.edu> writes:ch> is there a way to a) securely destroy a filesystem, AIUI zfs crypto will include this, some day, by forgetting the key. but for SSD, zfs above a zvol, or zfs above a SAN that may do snapshots without your consent, I think it''s just logically not a solveable problem, period, unless you have a writeable keystore outside the vdev structure. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100205/6fe42cae/attachment.bin>
On Fri, Feb 05, 2010 at 04:41:08PM -0500, Miles Nordin wrote:> >>>>> "ch" == c hanover <chanover at umich.edu> writes: > > ch> is there a way to a) securely destroy a filesystem, > > AIUI zfs crypto will include this, some day, by forgetting the key.Right.> but for SSD, zfs above a zvol, or zfs above a SAN that may do > snapshots without your consent, I think it''s just logically not a > solveable problem, period, unless you have a writeable keystore > outside the vdev structure.IIIRC ZFS crypto will store encrypted blocks in L2ARC and ZIL, so forgetting the key is sufficient to obtain a high degree of security. ZFS crypto over zvols and what not presents no additional problems. However, if your passphrase is guessable then the key might be recoverable even after it''s "forgotten". Nico --
In our particular case, there won''t be snapshots of destroyed filesystems (I create the snapshots, and destroy them with the filesystem). I''m not too sure on the particulars of NFS/ZFS, but would it be possible to create a 1GB file without writing any data to it, and then use a hex editor to access the data stored on those blocks previously? Any chance someone could make any kind of sense of the contents (allocated in the same order they were before, or what have you)? ZFS crypto will be nice when we get either NFSv4 or NFSv3 w/krb5 for over the wire encryption. Until then, not much point. - Cameron Hanover chanover at umich.edu "Our integrity sells for so little, but it is all we really have. It is the very last inch of us, but within that inch, we are free." --Valerie (V for Vendetta) On Feb 5, 2010, at 4:36 PM, Nicolas Williams wrote:> On Fri, Feb 05, 2010 at 03:49:15PM -0500, c.hanover wrote: >> Two things, mostly related, that I''m trying to find answers to for our >> security team. >> >> Does this scenario make sense: >> * Create a filesystem at /users/nfsshare1, user uses it for a while, >> asks for the filesystem to be deleted >> * New user asks for a filesystem and is given /users/nfsshare2. What >> are the chances that they could use some tool or other to read >> unallocated blocks to view the previous user''s data? > > If the tool isn''t accessing the raw disks, then the answer is "no > chance". (There''s no way to access the raw disks over NFS.) > >> Related to that, when files are deleted on a ZFS volume over an NFS >> share, how are they wiped out? Are they zeroed or anything. Same >> question for destroying ZFS filesystems, does the data lay about in >> any way? (That''s largely answered by the first scenario.) > > Deleting a file does not guarantee that data blocks are released: > snapshots might exist that retain references to the data blocks of a > file that is being deleted. Nor are blocks wiped when released. > >> If the data is retrievable in any way, is there a way to a) securely >> destroy a filesystem, or b) securely erase empty space on a >> filesystem. > > When ZFS crypto ships you''ll be able to securely destroy encrypted > datasets. Until then the only form of secure erasure is to destroy the > pool and then wipe the individual disks. > > Nico > -- > >-------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3682 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100205/29e35386/attachment.bin>
On Fri, Feb 05, 2010 at 05:08:02PM -0500, c.hanover wrote:> In our particular case, there won''t be snapshots of destroyed > filesystems (I create the snapshots, and destroy them with the > filesystem).OK.> I''m not too sure on the particulars of NFS/ZFS, but would it be > possible to create a 1GB file without writing any data to it, and then > use a hex editor to access the data stored on those blocks previously?Absolutely not. That is, you can create a 1GB file without writing to it, but it will appear to contain all zeros.> Any chance someone could make any kind of sense of the contents > (allocated in the same order they were before, or what have you)?No. See above.> ZFS crypto will be nice when we get either NFSv4 or NFSv3 w/krb5 for > over the wire encryption. Until then, not much point.You can use NFS with krb5 over the wire encryption _now_. Nico --
On 2/5/10 5:08 PM -0500 c.hanover wrote:> would it be possible to > create a 1GB file without writing any data to it, and then use a hex > editor to access the data stored on those blocks previously?No, not over NFS and also not locally. You''d be creating a sparse file, which doesn''t allocate space on disk for any filesystem (not just zfs). So when you read it back, you get back all 0s. The only way to actually allocate the space on disk is to write to it, and then of course you read back the data you wrote, not what was previously there.
On Feb 5, 2010, at 5:19 PM, Nicolas Williams wrote:>> ZFS crypto will be nice when we get either NFSv4 or NFSv3 w/krb5 for >> over the wire encryption. Until then, not much point. > > You can use NFS with krb5 over the wire encryption _now_. > > Nico > --I know, that''s just something I''m working out the particulars of before we decide if/when we want to offer it in production. I''ve got it working to some extent now. - Cameron Hanover chanover at umich.edu Tact is for people who aren''t witty enough to be sarcastic. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3682 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100205/7631ad8b/attachment.bin>
You might also want to note that with traditional filesystems, the ''shred'' utility will securely erase data, but no tools like that will work for zfs.
On 05/02/2010 21:46, Nicolas Williams wrote:> On Fri, Feb 05, 2010 at 04:41:08PM -0500, Miles Nordin wrote: >>>>>>> "ch" == c hanover<chanover at umich.edu> writes: >> >> ch> is there a way to a) securely destroy a filesystem, >> >> AIUI zfs crypto will include this, some day, by forgetting the key. > > Right. > >> but for SSD, zfs above a zvol, or zfs above a SAN that may do >> snapshots without your consent, I think it''s just logically not a >> solveable problem, period, unless you have a writeable keystore >> outside the vdev structure. > > IIIRC ZFS crypto will store encrypted blocks in L2ARC and ZIL, so > forgetting the key is sufficient to obtain a high degree of security.Correct, the L2ARC and ZIL are encrypted - they have to be they are stored on persistent storage media. The L2ARC is currently encrypted using a per pool ephemeral key, since the L2ARC is not persistent after pool export/reboot anyway. When the L2ARC moves to being persistent after pool/export reboot then it will be encrypted just the same as the dataset normally is. I have an RFE open that we may or may not implement in the future at would allow specifying wither the L2ARC is encrypted or not using the already existing secondarycache property. The only reason for considering that type of deployment is if the L2ARC is stored on media that is secured by other means to meet the deployments threat model.> ZFS crypto over zvols and what not presents no additional problems. > However, if your passphrase is guessable then the key might be > recoverable even after it''s "forgotten".Passphrases are transformed into AES wrapping keys using PKCS#5 PBE, there is a salt (taken from /dev/random) as well so brute force shouldn''t work. However if the password is easily guessed then the wrapping key could be recovered since while the salt is stored in a hidden property that can not be updated from zfs(1) it is visible if you explicitly ask for it. -- Darren J Moffat
>>>>> "nw" == Nicolas Williams <Nicolas.Williams at sun.com> writes: >>>>> "ch" == c hanover <chanover at umich.edu> writes:Trying again: ch> In our particular case, there won''t be ch> snapshots of destroyed filesystems (I create the snapshots, ch> and destroy them with the filesystem). Right, but if your zpool is above a zvol vdev (ex COMSTAR on another box), then someone might take a snapshot of the encrypted zvol. Then after you ``securely delete'''' a filesystem by overwriting various intermediate keys or whatever, they might roll back the zvol snapshot to undelete. Yes, you still need the passphrase to reach what they''ve undeleted, but that''s always true---what''s ``secure delete'''' supposed to mean besides the ability to permanently remove one dataset but not others, even from those who posess the passphrase? Otherwise it would not be a feature. It would just be a suggestion: ``forget your passphrase.'''' nw> ZFS crypto over zvols and what not presents no additional nw> problems. If you are counting on the ability to forget a key by overwriting the block of vdev in which the key''s stored, then doing it over zvol''s is an additional problem. >> but for SSD, Even if you do not have snapshots, SSD''s are CoW internally so they have something like latent snapshots from an attacker''s perspective. That is the point of my zvol example, which you are losing in your ``zvol''s are just like devices, that''s `abstraction,'' I don''t have to think about it.'''' ex., if your data lifecycle includes the idea that the ZFS crypto user will securely delete things from devices before sent back for warranty repair or reallocated to another group, whether it''s SAN LUN''s or SSD''s or zvol''s or anythnig that has a copy-on-write character, then from now on there is no such thing as overwriting. There is only forgetting passphrases. This is both the case for using crypto in the first place (overwriting blocks is no longer useful. Devices no longer offer any command that can really erase them.), and also the limitation of any ``Secure delete'''' feature. Example, my Chinese friend gives me a USB token and tells me the passphrase. It has ''blah'' stored on it. I create zfs filesystem ''blergh'', write secret stuff to it, then ``securely delete'''' it. I return the token to my friend without fear the contents of ''blergh'' could escape because you''ve promised I''ve ``securely'''' deleted it. He takes the token to its manufacturer, loads diagnostic firmware, rolls back the USB key to an earlier state using its CoW wear-leveling feature, and recovers the ``securely deleted'''' dataset. so in these cases ``secure delete'''' is meaningless. USB tokens are common, and I don''t know what is the use case of a ``secure delete'''' feature rather than simply ``using passphrasese'''', if not this one. zfs crypto overall is not meaningless, but it depends on the passphrase and is granular at whatever is protected by that passphrase, no smaller, once CoW underneath. If you have, (1) ability to change the passphrase whenver you like, and (2) the passphrase can be not just a string a user types but it can include a block of data read off a token, like LUKS, then with a little bit of care you can have back secure erase over CoW backing store. It depends on your ability to securely destroy the old block of key material on this token when you change the passphrase and be sure no one''s saved an old copy of it. That''s what I meant by keystore outside the vdev structure. Another scenario requiring something like secure delete which is complicated by SSD''s and zvol''s underneath is to protect laptops crossing borders. You may wish to make known that you routinely revoke owners'' access to their laptop drives prevent customs agents from trying to harrass/detain people into handing over their passphrases. You might do this by changing the passphrase before travel then delivering the new passphrase to yourself over VPN once you''ve passed customs. Then, you can safely give the old passphrase, which is all you know. If the laptop contains an SSD then the old passphrase is probably still useful to a customs agent who can extract dirty blocks from beneath the SSD-fs, so you lose. For the second scenario, the holy grail feature would be to have two zpools on one vdev, encrypted with different keys. zpool A will have a ''balloon'' dataset reserving the blocks used by zpool B. zpool A will have an encrypted ueberblock free of magic numbers and in a nonstandard location, and will be the one with secret data on it. zpool B will be normal and contain no holygrail features, should be preloaded by you with an earlier snapshot of zpool A. If you were to start using zpool B it would quietly overwrite and corrupt parts of zpool A. so, the process would be: zpool create B ssd <load with nonsecret but bootable stuff> zpool export B zpool create -holygrailfeature -o tokenfile=/tmp/A-token A ssd <automatically makes balloon dataset reserving used blocks of B> <possibly stores B''s txg counter to warn you from importing A if B''s been written since A was created> cp /tmp/A-token /vpn <-- part of key and nonstandard blockno of ueberblock You give the key to zpool B under duress. Either you can go to great lengths and arrange for zpool B to appear well-used before departing, or else you can try to claim ``we travel with stock installs, then receive an incremental stream once we pass customs'''' to justify the oldness of the B duress pool. The overall B/A system functions for other kinds of duress as well: your passphrase might include a physical token, which you can destroy if you become afraid, but now you can do this without making the laptop entirely worthless for the rest of the trip since you can still use the duress pool. Of course if that were the only goal there are easier ways to achieve it, but I think this one''s less mistake/leak prone in practice. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100208/a4a5f459/attachment.bin>
On Mon, Feb 08, 2010 at 03:41:16PM -0500, Miles Nordin wrote:> ch> In our particular case, there won''t be > ch> snapshots of destroyed filesystems (I create the snapshots, > ch> and destroy them with the filesystem). > > Right, but if your zpool is above a zvol vdev (ex COMSTAR on another > box), then someone might take a snapshot of the encrypted zvol. Then > after you ``securely delete'''' a filesystem by overwriting various > intermediate keys or whatever, they might roll back the zvol snapshot > to undelete. > > Yes, you still need the passphrase to reach what they''ve undeleted, > but that''s always true---what''s ``secure delete'''' supposed to mean > besides the ability to permanently remove one dataset but not others, > even from those who posess the passphrase? Otherwise it would not be > a feature. It would just be a suggestion: ``forget your passphrase.''''Correct. Secure erasure through "forgetting the keys" really does depend on "forgetting the keys", which does include "forgetting the passphrase". The only way to avoid that would be to store the wrapped keys in local keystores (i.e., a TPM or a smartcard) that do support secure erasure, so that "forgetting the keys" can be done without having to forget passphrases.> nw> ZFS crypto over zvols and what not presents no additional > nw> problems. > > If you are counting on the ability to forget a key by overwriting the > block of vdev in which the key''s stored, then doing it over zvol''s is > an additional problem.True, but this could happen regardless of whether the underlying storage is a zvol or not. I stand by the statement that "ZFS crypto over zvols and what not presents no additional problems". Nico --