zfs discuss - Nov 2006 - How to backup/clone all filesystems *and* snapshots in a zpool?

Suppose I have a server that is used as a backup system fom many other
("live") servers. It uses ZFS snapshots to enable people to recover
files from any date a year back (or so).

Now, I want to backup this backup server to some kind of external stable storage
in case disaster happens and this ZFS-backup server''s disks gets
corrupted.

If I just backup the normal "current" filesystem on this backup server
then I can always restore that and return to some known point - however all the
snapshots are lost so after that crash my users won''t be able to get
back old files... I could restore multiple backups from various dates, but that
will use up a lot of disk space.

Is there some way to "dump" all information from a ZFS filesystem? I
suppose I *could* backup the raw disk devices that is used by the zpool but
that''ll eat up a lot of tape space...

Any suggestions?
 
 
This message posted from opensolaris.org

On 16/11/06, Peter Eriksson <peter at ifm.liu.se> wrote:
> Is there some way to "dump" all information from a ZFS
filesystem? I suppose I *could* backup the raw disk devices that is used by the
zpool but that''ll eat up a lot of tape space...
If you want to have another copy somewhere, use zfs send/recv.

-- 
Rasputin :: Jack of All Trades - Master of Nuns
http://number9.hellooperator.net/

Hi Peter,

On Thu, 2006-11-16 at 06:15 -0800, Peter Eriksson wrote:
> If I just backup the normal "current" filesystem on this backup
server
> then I can always restore that and return to some known point -
> however all the snapshots are lost so after that crash my users
won''t
> be able to get back old files... I could restore multiple backups from
> various dates, but that will use up a lot of disk space.
Have you looked at the incremental zfs send/receive functionality ? It
allows you to take a backup archive of just the chances to a filesystem
between any two snapshots.

You could then take a full backup at a point far back in time, then take
incremental backups after that for each snapshot. There''s no automatic
way to do that at the moment, but it should be easily scriptable.

The risk though, is that if you take a full backup, then 1...n
incremental backups after that, loosing an incremental backup somewhere
in the middle of your range will make it impossible to restore the
subsequent incremental backups [ see example below ].

Does that help at all ?

	cheers,
			tim

[1] Here''s an example:

Take a file system

# zfs list extra
NAME    USED  AVAIL  REFER  MOUNTPOINT
extra    85K  9.60G  24.5K  /extra

Add some data, and take an original full backup:

# mkfile 10m file1
# zfs snapshot extra at original
# zfs send extra at original > /tmp/extra-original.dat
# ls -alh /tmp/extra-original.dat
-rw-r--r--   1 root     root         10M Nov 16 14:32 /tmp/extra-original.dat

Now take some incremental backups, showing that they''re only containing
the differences between data:

# mkfile 1m file2
# zfs snapshot extra at recent
# zfs send -i original extra at recent > /tmp/extra-recent.dat
# ls -alh /tmp/extra-recent.dat
-rw-r--r--   1 root     root        1.0M Nov 16 14:33 /tmp/extra-recent.dat
# mkfile 2m file3
# zfs snapshot extra at latest
# zfs send -i recent extra at latest > /tmp/extra-latest.dat
# ls -alh /tmp/extra-latest.dat
-rw-r--r--   1 root     root        2.0M Nov 16 14:34 /tmp/extra-latest.dat

Now destroy the filesystem (since it''s top level, this just empties it)

# zfs destroy -r extra

And then try to receive the saved datasets, out of order - simulating us
loosing the extra at recent backup.

# zfs receive extra/restore < /tmp/extra-original.dat
# zfs receive -d extra/restore < /tmp/extra-latest.dat
cannot receive: most recent snapshot does not match incremental source

Okay, restore that first, and we''re okay.

# zfs receive -d extra/restore < /tmp/extra-recent.dat
# zfs receive -d extra/restore < /tmp/extra-latest.dat
#



-- 
Tim Foster, Sun Microsystems Inc, Solaris Engineering Ops
http://blogs.sun.com/timf

Tim Foster wrote:
> Hi Peter,
> 
> On Thu, 2006-11-16 at 06:15 -0800, Peter Eriksson wrote:
>> If I just backup the normal "current" filesystem on this
backup server
>> then I can always restore that and return to some known point -
>> however all the snapshots are lost so after that crash my users
won''t
>> be able to get back old files... I could restore multiple backups from
>> various dates, but that will use up a lot of disk space.
> 
> Have you looked at the incremental zfs send/receive functionality ? It
> allows you to take a backup archive of just the chances to a filesystem
> between any two snapshots.
> 
> You could then take a full backup at a point far back in time, then take
> incremental backups after that for each snapshot. There''s no
automatic
> way to do that at the moment, but it should be easily scriptable.
FYI, I am working on a seamless way to send all the incrementals (send 
-I) and to send all the filesystems (send -r), as well as preserve the 
property settings on your filesystems (recv -p).

--matt