If I do something like this: zfs snapshot tank at today zfs send tank at today > tank.backup sleep 86400 zfs rename tank at today tank at yesterday zfs snapshot tank at today zfs send -I tank at yesterday tank at today > tank.incr Am I going to be able to restore the streams? Or is it going to be confused because I renamed the snapshot? (In case you can''t tell, I''m trying to come up with a reasonable zfs backup strategy to replace what I used to do with ufsdump.) Thanks!
Bill Shannon wrote:> If I do something like this: > > zfs snapshot tank at today > zfs send tank at today > tank.backup > sleep 86400 > zfs rename tank at today tank at yesterday > zfs snapshot tank at today > zfs send -I tank at yesterday tank at today > tank.incr > > Am I going to be able to restore the streams? > Or is it going to be confused because I renamed > the snapshot?I know this isn''t answering the question but rather than using "today" and "yesterday" why not not just use dates ? I do this: zfs snapshot -R tank@`date +%F` Or sometimes even `date +%F:%T` It may also be worth you having a look at Tim Fosters ZFS auto snapshot and auto backup services: http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_10 http://blogs.sun.com/timf/entry/zfs_automatic_for_the_people -- Darren J Moffat
Darren J Moffat wrote:> I know this isn''t answering the question but rather than using "today" > and "yesterday" why not not just use dates ?Because then I have to compute yesterday''s date to do the incremental dump. I don''t suppose I can create symlinks to snapshots in order to give them multiple names?> It may also be worth you having a look at Tim Fosters ZFS auto snapshot > and auto backup services: > > http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_10 > > http://blogs.sun.com/timf/entry/zfs_automatic_for_the_peopleThanks for the pointers!
On Thu, Mar 06, 2008 at 10:34:07PM -0800, Bill Shannon wrote:> Darren J Moffat wrote: > > I know this isn''t answering the question but rather than using "today" > > and "yesterday" why not not just use dates ? > > Because then I have to compute yesterday''s date to do the incremental > dump.Not if you set a ZFS property with the date of the last backup. -- albert chin (china at thewrittenword.com)
> Because then I have to compute yesterday''s date to do theincremental dump. snaps=15 today=`date +%j` # to change the second day of the year from 002 to 2 today=`expr $today + 0` nuke=`expr $today - $snaps` yesterday=`expr $today - 1` if [ $yesterday -lt 1 ] ; then yesterday=365 fi if [ $nuke -lt 1 ] ; then nuke=`expr 365 + $nuke` fi zfs destroy -r root@$nuke zfs snapshot -r root@$today zfs destroy -r z@$nuke zfs snapshot -r z@$today zfs send -i z/named@$yesterday z/named@$today | bzip2 -c |\ ssh host.com "bzcat | zfs recv -v -F -d z" zfs send -i z/local@$yesterday z/local@$today | bzip2 -c |\ ssh host.com "bzcat | zfs recv -v -F -d z" zfs send -i z/cgp@$yesterday z/cgp@$today | bzip2 -c |\ ssh host.com "bzcat | zfs recv -v -F -d z"
On Fri, 7 Mar 2008, Rob Logan wrote:> > zfs send -i z/named@$yesterday z/named@$today | bzip2 -c |\ > ssh host.com "bzcat | zfs recv -v -F -d z" > zfs send -i z/local@$yesterday z/local@$today | bzip2 -c |\ > ssh host.com "bzcat | zfs recv -v -F -d z" > zfs send -i z/cgp@$yesterday z/cgp@$today | bzip2 -c |\ > ssh host.com "bzcat | zfs recv -v -F -d z"Since I see ''bzip2'' mentioned here (a rather slow compressor), I should mention that based on a recommendation from a friend, I gave a compressor called ''lzop'' (http://www.lzop.org/) a try due to its reputation for compression speed. Compressing zfs send was causing it to take much longer. Testing with ''lzop'' showed that it was 2.5X faster than gzip on the Opteron CPU and that the compression was just a bit worse than gzip''s default compression level. It seems that some assembly language is used for x86 and Opteron. I did not test the relative speed differences on SPARC. The benefit from a compressor depends on the speed of the pipe and the speed of the filesystem. If CPU and/or network is the bottleneck, then LZO compression may be the solution. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Mar 7, 2008, at 8:55 AM, Bob Friesenhahn wrote:> On Fri, 7 Mar 2008, Rob Logan wrote: > Since I see ''bzip2'' mentioned here (a rather slow compressor), I > should mention that based on a recommendation from a friend, I gave a > compressor called ''lzop'' (http://www.lzop.org/) a try due to its > reputation for compression speed. Compressing zfs send was causing it > to take much longer. Testing with ''lzop'' showed that it was 2.5XI turned on compression in the ZFS filesystems and never looked back. Because it''s multi-threaded I have seen zero negative performance impact. In fact, I am unable to really make my ZFS fileservers notice they are in use CPU-wise. The added benefit is that zfs send/recv is already compressed, removing the need for a costly intermediate single-threaded compressor of any kind. --Randy Randy Bias, chief tactician, neoTactics, Inc. (877) NEO-TKTX, randyb at neotactics.com
Randy Bias wrote:> On Mar 7, 2008, at 8:55 AM, Bob Friesenhahn wrote: >> On Fri, 7 Mar 2008, Rob Logan wrote: >> Since I see ''bzip2'' mentioned here (a rather slow compressor), I >> should mention that based on a recommendation from a friend, I gave a >> compressor called ''lzop'' (http://www.lzop.org/) a try due to its >> reputation for compression speed. Compressing zfs send was causing it >> to take much longer. Testing with ''lzop'' showed that it was 2.5X > > I turned on compression in the ZFS filesystems and never looked back. > Because it''s multi-threaded I have seen zero negative performance > impact. In fact, I am unable to really make my ZFS fileservers notice > they are in use CPU-wise. > > The added benefit is that zfs send/recv is already compressed, > removing the need for a costly intermediate single-threaded compressor > of any kind.See the earlier thread on this. zfs send/recv are NOT already compressed (nor will they be encrypted by the ZFS Crypto project either), they are in the clear. -- Darren J Moffat
On Fri, Mar 07, 2008 at 01:52:45AM -0500, Rob Logan wrote:> > Because then I have to compute yesterday''s date to do the > > incremental dump. > > snaps=15 > today=`date +%j` > # to change the second day of the year from 002 to 2 > today=`expr $today + 0`Er, can''t this be confused with octal? Hmm, guess not.> nuke=`expr $today - $snaps` > yesterday=`expr $today - 1` > > if [ $yesterday -lt 1 ] ; then > yesterday=365 > fiEr, what if the year before was a leap year? Anyways, yes, you can keep symlinks to snapshots (.zfs/snapshot/<snapshot-name>). Also, you don''t have to compute yesterday''s date. You can just have a convention for naming these snapshots and do this: yesterday''s snapshot name = zfs list -r <dataset>|grep <convention>|head -1 create today''s snapshot delete yesterday''s.
> > zfs send -i z/cgp@$yesterday z/cgp@$today | bzip2 -c |\ > > ssh host.com "bzcat | zfs recv -v -F -d z" > > Since I see ''bzip2'' mentioned here (a rather slow compressor), I > should mention that based on a recommendation from a friend, I gave a > compressor called ''lzop'' (http://www.lzop.org/) a try due to its > reputation for compression speed.If you have several cores, I can recommend bzip2smp instead of "bzip2 -c"; see http://bzip2smp.sourceforge.net/ for more info. I am using it on my T1000 with good results, performance scales linear if you use only one thread per core (more threads will cause lots of cache misses). Regards -- Volker -- ------------------------------------------------------------------------ Volker A. Brandt Consulting and Support for Sun Solaris Brandt & Brandt Computer GmbH WWW: http://www.bb-c.de/~vab/ Am Wiesenpfad 6, 53340 Meckenheim Email: vab at bb-c.de Handelsregister: Amtsgericht Bonn, HRB 10513 Schuhgr??e: 45 Gesch?ftsf?hrer: Rainer J. H. Brandt und Volker A. Brandt
Bill Shannon wrote:> If I do something like this: > > zfs snapshot tank at today > zfs send tank at today > tank.backup > sleep 86400 > zfs rename tank at today tank at yesterday > zfs snapshot tank at today > zfs send -I tank at yesterday tank at today > tank.incr > > Am I going to be able to restore the streams? > Or is it going to be confused because I renamed > the snapshot?Since no one could answer my question, I decided to do the experiment... Based on my experiments, it seems that I can still restore the streams even with the rename, but I need to rename the restored snapshot, e.g.: zfs recv -F test < tank.backup zfs rollback -r tank at today zfs rename tank at today tank at yesterday zfs recv test < tank.incr