zfs discuss - Nov 2007 - Suggestion/Request: ZFS-aware rm command

Hello again,

Some of you may have read my earlier post about wanting to recover partial ZFS
space.  It seems as though that isn''t possible given the current
implementation... so, I would like to suggest the following
"enhancement":

A zfs-aware rm command (ie. zfs rm).  The idea here is that we would be able to
remove a file (or directory) from a zfs filesystem and all snapshots that the
file exists in.  This would allow the ability to recover space when part of a
zfs pool is moved/deleted, but the bulk of the data in the snapshots is still
relevant.

I know, I know -- snapshots are there to protect us from messing up, but a
specific command that would allow us to "force" the removal
(unlinking) of certain structures within the filesystem and its associated
snapshots would be quite useful.

Take the example that bit me...

I had a filesystem that had a subdirectory that grew too big and had to be moved
to another pool.  Since the snapshots contained all of that data, even though
the directory was moved, I was unable to recover the space (almost 300GB)
without deleting all of the snapshots.

The problem with deleting all of the snapshots is that I would lose the ability
to recover the other data within that filesystem.  The problem with
"sending" the snapshots elsewhere before deleting them is that at
almost 500GB each, I simply didn''t have that kind of space available.

If I had the ability to forcefully delete the directory from the filesystem and
its snapshots, I would have been able to move my data around without sacrificing
the recoverability of the other data.

Maybe something like this:
    zfs rm -f myfile

Seems like this would be pretty easy if we are really just talking about
unlinking pointers to the specific data, but I''ll let those more
intimate with the code speak to that.

Thanks,
Michael
 
 
This message posted from opensolaris.org

I don''t currently have a way to test this, but did you try:
    make a clone of the snapshot
    in the clone, remove the directories
    make a snapshot of the clone
    destroy the clone
    destroy the old snapshot

In my mind, this should work, given no other dependencies exist.
Then again, a mind is a terrible place... :-)
 -- richard

Michael McKnight wrote:
> Hello again,
>
> Some of you may have read my earlier post about wanting to recover partial
ZFS space.  It seems as though that isn''t possible given the current
implementation... so, I would like to suggest the following
"enhancement":
>
> A zfs-aware rm command (ie. zfs rm).  The idea here is that we would be
able to remove a file (or directory) from a zfs filesystem and all snapshots
that the file exists in.  This would allow the ability to recover space when
part of a zfs pool is moved/deleted, but the bulk of the data in the snapshots
is still relevant.
>
> I know, I know -- snapshots are there to protect us from messing up, but a
specific command that would allow us to "force" the removal
(unlinking) of certain structures within the filesystem and its associated
snapshots would be quite useful.
>
> Take the example that bit me...
>
> I had a filesystem that had a subdirectory that grew too big and had to be
moved to another pool.  Since the snapshots contained all of that data, even
though the directory was moved, I was unable to recover the space (almost 300GB)
without deleting all of the snapshots.
>
> The problem with deleting all of the snapshots is that I would lose the
ability to recover the other data within that filesystem.  The problem with
"sending" the snapshots elsewhere before deleting them is that at
almost 500GB each, I simply didn''t have that kind of space available.
>
> If I had the ability to forcefully delete the directory from the filesystem
and its snapshots, I would have been able to move my data around without
sacrificing the recoverability of the other data.
>
> Maybe something like this:
>     zfs rm -f myfile
>
> Seems like this would be pretty easy if we are really just talking about
unlinking pointers to the specific data, but I''ll let those more
intimate with the code speak to that.
>
> Thanks,
> Michael
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

is it possible to delete files in the snapshot? (.zfs directory?)
 
 
This message posted from opensolaris.org

Hi All:

Actually, I am running into the same case (need to remove certain files from a
snapshot), let me give you a good example of why we need to do it.

In our case, we use zfs snapshots to store online backups of home file systems
(that is, readily accessable to employees if they need to recover a file or
directory).  Let say employee (we will call him Mr. Scumbags), has downloaded
and saved a file that is objectionable (say it details the losing streak of the
Chicago Cubs, or some other such very nasty information :)).  After appropriate
discipline (40 lashses, or whaterver else your company deems appropriate), you
(as the honorable system administrator) delete the file from his home directory.
HOWEVER, we also need to delete the files from the backups (the zfs snapshots).
How can we do it?

thanks,

Paul
 
 
This message posted from opensolaris.org

Hi Richard:

I just tried your suggestion, unfortunately it doesn''t work. 
Basically:
make a clone of the snapshot - works bine
in the clone, remove the directories - works fine
make a snapshot of the clone - works fine
destroy the clone - fails, because  ZFS reports that the "filesystem has
children"

Any other suggestions? 
(the only option I can think of, is to use clones instead of snapshots in the
future, just so that I can delete files in the clones in case I ever need to)

thanks,

Paul
 
 
This message posted from opensolaris.org

2007/11/13, Paul Jochum <pjochum at gmail.com>:
> (the only option I can think of, is to use clones instead of snapshots in
the future, just so that I can delete files in the clones in case I ever need
to)
But to create a clone you''ll need a snapshot so I think the problem
will still be there...

Paul Jochum wrote:
> Hi Richard:
> 
> I just tried your suggestion, unfortunately it doesn''t work. 
Basically:
> make a clone of the snapshot - works bine
> in the clone, remove the directories - works fine
> make a snapshot of the clone - works fine
> destroy the clone - fails, because  ZFS reports that the "filesystem
has children"
Have you looked at the "promote" command for zfs(1) ?


-- 
Darren J Moffat

> But to create a clone you''ll need a snapshot so I
> think the problem
> will still be there...
This might be a way around this problem though.  Deleting files from snapshots
sounds like a messy approach in terms of the architecture, but deleting files
from clones would be fine.

So what''s needed is a way to separate a clone from it''s
snapshot and make it standalone.  Once that''s done the original
snapshot can be deleted, and files can be deleted from the clone.

You might loose a few of the benefits of snapshots with this approach, but it
would seem a reasonable way of doing this.  Can ideas like this be raised with
the people developing zfs?
 
 
This message posted from opensolaris.org

I agree, being able to delete the snapshot that a clone is attached to would be
a nice feature.  Until we get that, this is what I have done (in case this helps
anyone else):

1) snapshot the filesystem
2) clone the snapshot into a seperate pool
3) only nfs mount the seperate pool with clones

That way, if I need to delete a file from the backups, I can delete it from the
clones.  Since users only have access to the clones, they would not see the
deleted files.  On the other hand, the deleted files still exist in the
snapshots, so I am never ''corrupting'' my backups.  Note, this
only works for limited situations like mine I believe, not sure if there is a
way to prevent users from having access to their .zfs/snapshot directory when
they have local access to the zfs filesystem.

Paul
 
 
This message posted from opensolaris.org