zfs discuss - Jul 2007 - Pseudo file system access to snapshots?

As I have been grappling with how I will manage ZFS backups using
existing enterprise backup tools (e.g. Netbackup), it seems as though
two approaches continue to dominate:

1) Just use the POSIX interface like is used for UFS,
   VxFS, etc.  This has key the disadvantage that it
   is not efficient (space, time, performance, etc.)
   during backup and restore could be impossible
   due to the space inefficiency.  Other disadvantages
   exist as well.
2) Use "zfs send" (but not receive) to do disk-to-disk
   backups, then back up the "zfs send" images to tape.
   This is also inefficient due to extra space, time,
   etc. but you ACL''s, snapshots, clones, etc. seem
   as though they will be preserved on restore.  The
   interface to the backup software will require some
   scripting, much like anything else that requires
   a quiesce before backup.

For a while I was thinking that "zfs send" data streams would be nice
to work with NDMP.  However, this solution will only play well with
the commercial products that have been going after the storage
appliance market for quite some time.  I''m not aware of free tools
that speak NDMP.

Perhaps a better approach is to create a pseudo file system that looks like:

<mntpt>/pool
           /@@
           /@today
           /@yesterday
           /fs
              /@@
              /@2007-06-01
           /otherfs
                   /@@

As you might imagine, reading from pool/@today would be equivalent to
"zfs send pool at today".  Some sort of notation (pool/@yesterday at
today
?) would be needed to represent "zfs send -i pool at yesterday
pool at today".  Reading from pool/fs/@@ would be equivalent to "zfs
snapshot pool/fs@<timestamp>; zfs send pool/fs@<timestamp>".

Writing to a particular path would have the same effect as zfs receive.

Is this something that is maybe worth spending a few more cycles on,
or is it likely broken from the beginning?

Mike

-- 
Mike Gerdts
http://mgerdts.blogspot.com/

Mike Gerdts wrote:
> Perhaps a better approach is to create a pseudo file system that looks
like:
> 
> <mntpt>/pool
>            /@@
>            /@today
>            /@yesterday
>            /fs
>               /@@
>               /@2007-06-01
>            /otherfs
>                    /@@
How is this different from cd <mntpt>/.zfs/snapshot/   ?

On 7/11/07, Darren J Moffat <darrenm at opensolaris.org>
wrote:
> Mike Gerdts wrote:
> > Perhaps a better approach is to create a pseudo file system that looks
like:
> >
> > <mntpt>/pool
> >            /@@
> >            /@today
> >            /@yesterday
> >            /fs
> >               /@@
> >               /@2007-06-01
> >            /otherfs
> >                    /@@
>
> How is this different from cd <mntpt>/.zfs/snapshot/   ?
>
<mntpt>/.zfs/snapshot provides file-level access to the contents of
the snapshot.  If you back those up, then restore every snapshot, you
will potentially be using way more disk space.

What I am proposing is that "cat mntpt/pool/@snap1" delivers a data
stream corresponding to the output of "zfs send" and that "cat
mntpnt/pool/@snap1 at snap2" delivers a data stream corresponding to
"zfs
send -i snap1 snap2".

This would allow existing backup tools to perform block level
incremental backups.  Assuming that writing to the various files is
the equivalent of the corresponding "zfs receive" commands, it
provides for block level restores that preserve space efficiency as
well.

Why?

Suppose I have a server with 50 full root zones on it.  Each zone has
a zonepath at /zones/<zonename> that is about 8 GB.  This implies that
I need 400 GB just for zone paths.  Using ZFS clones, I can likely
trim that down to far less than 100 and probably less than 20.  I
can''t trim it down that far if I don''t have a way to restore
the
system.

This "restore problem" is my key worry in deploying ZFS in the area
where I see it as most beneficial.  Another solution that would deal
with the same problem is block-level deduplication.  So far my queries
in this area have been met with silence.

Hmmm... I just ran into another snag with this.  I had been assuming
that clones and snapshots were more closely related.  But when I tried
to send the differences between the source of a clone and a snapshot
within that clone I got this message:

incremental source must be in same filesystem
usage:
        send [-i <snapshot>] <snapshot>

Mike

-- 
Mike Gerdts
http://mgerdts.blogspot.com/

> This "restore problem" is my key worry in deploying ZFS in the
area
> where I see it as most beneficial.  Another solution that would deal
> with the same problem is block-level deduplication.  So far my queries
> in this area have been met with silence.
I must have missed your messages on deduplication.  But did you see this 
thread on it?  "zfs space efficiency", 6/24 - 7/7?

We''ve been thinking about ZFS dedup for some time, and want to do it
but have
other priorities at the moment.
> Hmmm... I just ran into another snag with this.  I had been assuming
> that clones and snapshots were more closely related.  But when I tried
> to send the differences between the source of a clone and a snapshot
> within that clone I got this message:
I''m not sure what you mean by "more closely related".  The
only reason we
don''t support that is because we haven''t gotten around to
adding the special
cases and error checking for it (and I think you''re the first person to
notice
its omission).  But it''s actually in the works now so stay tuned for an
update
in a few months.

--matt

On 7/11/07, Matthew Ahrens <Matthew.Ahrens at sun.com>
wrote:
> > This "restore problem" is my key worry in deploying ZFS in
the area
> > where I see it as most beneficial.  Another solution that would deal
> > with the same problem is block-level deduplication.  So far my queries
> > in this area have been met with silence.
>
> I must have missed your messages on deduplication.
That''s OK...  I think that I probably spill my guts adequately on the
topic at:

http://mail.opensolaris.org/pipermail/zfs-discuss/2006-August/034065.html
http://mail.opensolaris.org/pipermail/zfs-discuss/2007-April/040034.html
http://mail.opensolaris.org/pipermail/storage-discuss/2007-May/002711.html

The only previous thread that seemed to have much discussion was

http://mail.opensolaris.org/pipermail/zfs-discuss/2007-January/
> But did you see this
> thread on it?  "zfs space efficiency", 6/24 - 7/7?
I stopped reading the "zfs space efficiency" thread just before it got
interesting.  :)

The dedup methods suggested there are quite consistent with those that
I had previously suggested.
>
> We''ve been thinking about ZFS dedup for some time, and want to do
it but have
> other priorities at the moment.
This is very good to hear.
> > Hmmm... I just ran into another snag with this.  I had been assuming
> > that clones and snapshots were more closely related.  But when I tried
> > to send the differences between the source of a clone and a snapshot
> > within that clone I got this message:
>
> I''m not sure what you mean by "more closely related". 
The only reason we
> don''t support that is because we haven''t gotten around to
adding the special
> cases and error checking for it (and I think you''re the first
person to notice
> its omission).  But it''s actually in the works now so stay tuned
for an update
> in a few months.
I was trying to say (but didn''t) that I thought that there should be
nearly (very important word) no differences between finding the
differences from a clone''s origin to a snapshot of the clone and
finding the differences between snapshots between snapshots of the
same file system.

I''m also glad to see this is in the works.  Most of my use cases for
ZFS involve use of clones.  Lack of space-efficient backups and
especially restores makes me wait to use ZFS outside of the lab.

Mike

-- 
Mike Gerdts
http://mgerdts.blogspot.com/