zfs discuss - Jul 2008 - How to delete hundreds of emtpy snapshots

I got overzealous with snapshot creation. Every 5 mins is a bad idea.
Way too many.

What''s the easiest way to delete the empty ones?

zfs list takes FOREVER

Hi there,

Yes 5 minutes is too often.

I would, if it is easy and you have enough room, and don''t want any of
the snapshots, create another temp zfs file system and copy your data  
there and destroy the current zfs file system. Then recreate it and  
copy the data back. I don''t know if there is a better way ... If there
is I would be interested to know incase I ever actually need to fix  
something like this :)

Ta,
---
Cooper Ry Lees
UNIX Evangelist
Information Management Services (IMS)
Australian Nuclear Science and Technology Organisation (ANSTO)
[p] +61 2 9717 3853
[e] crl at ansto.gov.au

On 17/07/2008, at 3:16 PM, Joe S wrote:
> I got overzealous with snapshot creation. Every 5 mins is a bad idea.
> Way too many.
>
> What''s the easiest way to delete the empty ones?
>
> zfs list takes FOREVER
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

I don''t know if there''s any easy way to delete them, but I can
offer a suggestion for the future.

Tim Foster has a ZFS automatic snapshot script that would be worth taking a look
at:
http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_10

That script lets you fairly easily set up snapshots so you have say:
12x 5 minute snapshots
24x hourly snapshots
7x daily snapshots
5x weekly snapshots
12x monthly snapshots

It''s a grand total of 60 snapshots, you still get backups every 5
minutes for anything you''ve done recently, but you can go back a whole
year if you really need to find something.
 
 
This message posted from opensolaris.org

zfs list is mighty slow on systems with a large number of objects, but there is
no foreseeable plan that I''m aware of to solve that
"problem".

Never the less, you need to do a zfs list, therefore, do it once and work from
that.

zfs list > /tmp/zfs.out
for i in `grep mydataset@ /tmp/zfs.out`; do zfs destroy $i; done


As for 5 minute snapshots.... this is NOT a bad idea.  It is, however, complex
to manage.  Thus, you need to employ tactics to make it more digestible.

You need to ask  yourself first why you want 5 min snaps. Is it replication?  If
so, create it, replicate it, destroy all but the last snapshot or even rotate
them.  Or, is it fallback in case you make a mistake?  Then just keep around the
last 6 snapshots or so.

zfs rename & zfs destroy are your friends.... use them wisely. :)

If you want to discuss exactly what your trying to facilitate I''m sure
we can come up with some more concrete ideas to help you.

benr.
 
 
This message posted from opensolaris.org

Hi Joe,

I use this script to delete my empty snapshots :

#!/bin/bash

NOW=$(date +%Y%m%d-%H%M%S)

POOL=tank

zfs snapshot -r ${POOL}@${NOW}

FS_WITH_SNAPSHOTS=$(zfs list -t snapshot | grep ''@'' | cut -d
''@'' -f 1 | uniq)

for fs in $FS_WITH_SNAPSHOTS ; do

        EMPTY_SNAPSHOTS=$(zfs list -t snapshot | grep -v "@$NOW" |
grep "${fs}@" | awk '' $2 == "0" { print $1
}'' )

        for snapshot in $EMPTY_SNAPSHOTS ; do
                echo "Destroying empty snapshot $snapshot"
                zfs destroy $snapshot
        done
done

Hope that helps,

Sylvain




On Thu, Jul 17, 2008 at 7:16 AM, Joe S <js.lists at gmail.com>
wrote:
> I got overzealous with snapshot creation. Every 5 mins is a bad idea.
> Way too many.
>
> What''s the easiest way to delete the empty ones?
>
> zfs list takes FOREVER
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

Sylvain Dusart schrieb:
> Hi Joe,
> 
> I use this script to delete my empty snapshots :
> 
> #!/bin/bash
> 
> NOW=$(date +%Y%m%d-%H%M%S)
> 
> POOL=tank
> 
> zfs snapshot -r ${POOL}@${NOW}
> 
> FS_WITH_SNAPSHOTS=$(zfs list -t snapshot | grep ''@'' | cut
-d ''@'' -f 1 | uniq)
> 
> for fs in $FS_WITH_SNAPSHOTS ; do
> 
>         EMPTY_SNAPSHOTS=$(zfs list -t snapshot | grep -v "@$NOW"
|
> grep "${fs}@" | awk '' $2 == "0" { print $1
}'' )
> 
>         for snapshot in $EMPTY_SNAPSHOTS ; do
>                 echo "Destroying empty snapshot $snapshot"
>                 zfs destroy $snapshot
>         done
> done
> 
> Hope that helps,
This script is broken - because the "zfs list" output is broken. You 
might end up deleting snapshots with data in it.

Data which is in more than one snapshot will not get counted in the "zfs 
list" output at all.

Let''s try it with an example:

# zfs create export/test
# mkfile 10m /export/test/bla
# zfs snapshot export/test at one
# zfs snapshot export/test at two
# rm /export/test/bla

Now both snapshots should contain (the same) 10MB of valuable data (the 
file /export/test/bla just removed). But:

# zfs list -r export/test
NAME              USED  AVAIL  REFER  MOUNTPOINT
export/test      24.5K   171G  24.5K  /export/test
export/test at one      0      -  24.5K  -
export/test at two      0      -  24.5K  -

... indicates two empty snapshots. But after removing one of the 
snapshots ...

# zfs destroy export/test at one

... the data magically shows up on the second snapshot (in my case not 
10MB, because the compression setting was inherited from the parent zfs)

# zfs list -r export/test
NAME              USED  AVAIL  REFER  MOUNTPOINT
export/test        47K   171G  24.5K  /export/test
export/test at two  22.5K      -  24.5K  -


Your script above would have deleted both snapshots. One possible 
solution would be to re-evaluate the "zfs list" output after each 
snapshot deleted. This way the last one of data-identical snapshots 
would be preserved. Something like:

while :; do
   EMPTYSNAPSHOT=$(zfs list -Ht snapshot -o name,used | \
                     awk ''$2 == 0 { print $1; exit }'')
   [ -n "$EMPTYSNAPSHOT" ] || break
   zfs destroy $EMPTYSNAPSHOT
done


Daniel

On Thu, 17 Jul 2008, Ben Rockwood wrote:
> zfs list is mighty slow on systems with a large number of objects, 
> but there is no foreseeable plan that I''m aware of to solve that 
> "problem".
>
> Never the less, you need to do a zfs list, therefore, do it once and 
> work from that.
If the snapshots were done from a script then their names are easily 
predictable and similar logic can be used to re-create the existing 
names.  This avoids the need to do a ''zfs list''.

Bob

In one of my prior experiments, I included the names of the snapshots I 
created in a plain text file.

I used this file, and not the zfs list output to determine which 
snapshots I was going to remove when it came time.

I don''t even remember *why* I did that in the first place, but it 
certainly made things easier when it came time to clean up a whole bunch 
of stuff...

(And was not impacted by zfs list being non-snappy...)

The snapshot naming scheme meant that it was dead easy to work out which 
to remove / keep...

Right now, I don''t have a system (that box was killed in a dreadful xen
experiment :) so I''ll be watching this thread with renewed interest to 
see who else is doing what...

Nathan.

Bob Friesenhahn wrote:
> On Thu, 17 Jul 2008, Ben Rockwood wrote:
> 
>> zfs list is mighty slow on systems with a large number of objects, 
>> but there is no foreseeable plan that I''m aware of to solve
that
>> "problem".
>>
>> Never the less, you need to do a zfs list, therefore, do it once and 
>> work from that.
> 
> If the snapshots were done from a script then their names are easily 
> predictable and similar logic can be used to re-create the existing 
> names.  This avoids the need to do a ''zfs list''.
> 
> Bob
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

> I got overzealous with snapshot creation. Every 5 mins is a bad idea. Way
too many.
> What''s the easiest way to delete the empty ones?
> zfs list takes FOREVER
You might enjoy reading:

ZFS snapshot massacre
http://blogs.sun.com/chrisg/entry/zfs_snapshot_massacre.

(Yes, the "." is part of the URL (NMF) - so add it or you''ll
404).

Rob
 
 
This message posted from opensolaris.org

Also http://blogs.sun.com/chrisg/entry/a_faster_zfs_snapshot_massacre which I
run every night.  Lots of snapshots are not a bad thing it is keeping them for a
long time that takes space.  I''m still snapping every 10 minutes and it
is great.

The thing I discovered was that I really wanted to be able to find distinct
verstons fo a file so that I could see which one was the version I wanted to get
back. To that end I wrote
http://blogs.sun.com/chrisg/entry/zfs_versions_of_a_file and filed this RFE to
help with this:

http://bugs.opensolaris.org/view_bug.do?bug_id=6719101
 
 
This message posted from opensolaris.org