zfs discuss - Dec 2007 - Question - does a snapshot of root include child filesystems?

The title says it all really, we''ll be creating one big zpool here,
with many sub filesystems for various systems.  Am I right in thinking that we
can use snapshots of the root filesystem to take a complete backup of
everything?
 
 
This message posted from opensolaris.org

On Wed, 19 Dec 2007, Ross wrote:
> The title says it all really, we''ll be creating one big zpool
here, with
> many sub filesystems for various systems.  Am I right in thinking that 
> we can use snapshots of the root filesystem to take a complete backup of 
> everything?
I believe what you''re looking for is ''zfs snapshot
-r''


Regards,
markm

Ross wrote:
> The title says it all really, we''ll be creating one big zpool
here, with many sub filesystems for various systems.  Am I right in thinking
that we can use snapshots of the root filesystem to take a complete backup of
everything?
>   
This question doesn''t really make sense.  But it looks like
you''re
wanting to use the -r option (recursive) for zfs snapshot.
 -- richard

The recursive option creates a separate snapshot for every child filesystem,
making backup management more difficult if there are many child filesystems. 
The ability to natively create a single snapshot/backup of an entire ZFS
hierarchy of filesystems would be a very nice thing indeed.
 
 
This message posted from opensolaris.org

Aye, that''s what I was hoping for.  Since ZFS seems to support nesting
filesystems, I was hoping that each filesystem would contain the others, so a
snapshot of the root filesystem would also be a snapshot of the others.

I''m still not sure if that''s the case or not, it could be that
-r just means you have individual snapshots as well (handy for restores).

Going to do some testing now I have zfs running, if it comes to it I''m
hoping I can use -r and script a backup of all the filesystems.  Scripting zfs
send from that shouldn''t be too hard, I''m just hoping
it''s not going to be too complex to script the zfs receive side of it.

I guess the ideal for me would be for filesystems to include child filesystems
in their snaphots and for zfs snapshot -r to recursively snapshot each
filesystem, starting from the topmost branches of the tree and working
backwards.

That would mean you could backup the entire system by just copying one snapshot,
but you would have contained within that individual snapshots for each child
filesystem.

Apologies for my inconsistent (or just plain wrong) use of terminology. 
I''ve only started using ZFS this week, and hadn''t even touched
a Unix system before that!
 
 
This message posted from opensolaris.org

> I''ve only started using ZFS this week, and hadn''t even
touched a Unix
welcome to ZFS... here is a simple script you can start with:

#!/bin/sh

snaps=15
today=`date +%j`
nuke=`expr $today - $snaps`
yesterday=`expr $today - 1`

if [ $yesterday -lt 0 ] ; then
   yesterday=365
fi

if [ $nuke -lt 0 ] ; then
   nuke=`expr 365 + $nuke`
fi

zfs destroy  -r root@$nuke
zfs snapshot -r root@$today
zfs destroy  -r z@$nuke
zfs snapshot -r z@$today

zfs send -i z/local@$yesterday  z/local@$today  | bzip2 -c | ssh  
some.host.com "bzcat | zfs recv -v -F -d z"
zfs send -i z/cgp@$yesterday    z/cgp@$today    | bzip2 -c | ssh  
some.host.com "bzcat | zfs recv -v -F -d z"
zfs send -i z/apache@$yesterday z/apache@$today | bzip2 -c | ssh  
some.host.com "bzcat | zfs recv -v -F -d z"

Ross wrote:
> Aye, that''s what I was hoping for.  Since ZFS seems to support
nesting filesystems, I was hoping that each filesystem would contain the others,
so a snapshot of the root filesystem would also be a snapshot of the others.
>   
IMHO, a file system is created only when you need to change its policies.
For example, I might want a read-only file system and a compressed file
system, so I would make two file systems.  If you want the same policies
for each file system, you''d be better off using directories. 
Continuing
the policies decision, one of the policies may be backup frequency (I
really don''t care to backup my music every day, but I would like to
backup my home directory every day).
> I''m still not sure if that''s the case or not, it could be
that -r just means you have individual snapshots as well (handy for restores).
>
> Going to do some testing now I have zfs running, if it comes to it
I''m hoping I can use -r and script a backup of all the filesystems. 
Scripting zfs send from that shouldn''t be too hard, I''m just
hoping it''s not going to be too complex to script the zfs receive side
of it.
>   
Good idea, scripting tends to eliminate fumble-fingers failure modes :-)
> I guess the ideal for me would be for filesystems to include child
filesystems in their snaphots and for zfs snapshot -r to recursively snapshot
each filesystem, starting from the topmost branches of the tree and working
backwards.
>   
There are many other ways to copy a directory structure, including cp 
(copy).
> That would mean you could backup the entire system by just copying one
snapshot, but you would have contained within that individual snapshots for each
child filesystem.
>
> Apologies for my inconsistent (or just plain wrong) use of terminology. 
I''ve only started using ZFS this week, and hadn''t even touched
a Unix system before that!
>   
Welcome!  I hope you find Unix to be a very flexible system.
 -- richard

Looking into this further, it looks like this be available in build 77, with
"zfs snapshot -r" a new "zfs send -r" feature:

http://www.opensolaris.org/jive/thread.jspa?messageID=163689
http://www.opensolaris.org/os/community/arc/caselog/2007/574/
http://bugs.opensolaris.org/view_bug.do?bug_id=6421958
 
 
This message posted from opensolaris.org

Flexible is right.  I knew Unix / Linux traditionally make it possible to do a
lot from the command line, but having never used them I''d just assumed
they were basically very similar to windows, but with a whole load of extra
overhead because of all the commands you had to learn.

But after just one day with Solaris I''ve realised Unix is very, very
different to windows, and "flexible" means an awful lot more in Unix
than it does in Windows.

Just seeing picker''s script is mind boggling for a windows admin. 
I''d wondered why ZFS didn''t have any compression options for
zfs send, now I know why:  There''s just no need in a Unix system.  In
windows, if ZFS didn''t support compression that would be it, in Unix,
streams mean there''s no need for every program to re-invent features
like that, and I can use whatever compression program I think is best.

And the ability to use SSH (or X-windows when I feel up to it) to remotely
access a box is a godsend.  Windows might have RDP these days, but now I know
what X-Server can do, I can see it''s a poor imitation.  I''d
originally bought a USB KVM switch to go with my new home server, but now
I''m finding it''s essentially redundant.  I can manage
everything on the Solaris box via Putty without ever leaving Windows.

It''s so far taken about a day to work out ZFS, Samba (and SWAT), vi,
bash and SSH.  They''re all far easier than I expected (man is stunning,
windows has nothing like this level of documentation).  I thought it would take
me most of christmas to find out if this would work, but in one evening
I''m done.  I''ve now got a home media server up and running and
iTunes has been migrated over to it.

Many thanks for the help to everyone who replied, it''s very much
appreciated.
 
 
This message posted from opensolaris.org

Ross wrote:
> It''s so far taken about a day to work out ZFS, Samba (and SWAT),
vi,
> bash and SSH.
Try b78 - it''s got the new native CIFS (SMB) server. And it''s
integrated
with zfs, so you can create a windows-accessible share as easily as:

zfs create -o sharesmb=name=sharename poolname/zfsname

R.

Ross wrote:
> Flexible is right.  I knew Unix / Linux traditionally make it possible to
do a lot from the command line, but having never used them I''d just
assumed they were basically very similar to windows, but with a whole load of
extra overhead because of all the commands you had to learn.
>
> But after just one day with Solaris I''ve realised Unix is very,
very different to windows, and "flexible" means an awful lot more in
Unix than it does in Windows.
>   
Yes, this is a philosophical difference between UNIX and Windows (nee VMS).
The saying goes something like this: with VMS you would spend an hour trying
to figure out what options to choose to get something done.  With UNIX you
spend an hour figuring out what commands to string together to get something
done.
> Just seeing picker''s script is mind boggling for a windows admin. 
I''d wondered why ZFS didn''t have any compression options for
zfs send, now I know why:  There''s just no need in a Unix system.  In
windows, if ZFS didn''t support compression that would be it, in Unix,
streams mean there''s no need for every program to re-invent features
like that, and I can use whatever compression program I think is best.
>
>   
Welcome to the UNIX world :-)
 -- richard

On Thu, 20 Dec 2007, Richard Elling wrote:
> Ross wrote:
>> Aye, that''s what I was hoping for.  Since ZFS seems to support
nesting filesystems, I was hoping that each filesystem would contain the others,
so a snapshot of the root filesystem would also be a snapshot of the others.
>>
>
> IMHO, a file system is created only when you need to change its policies.
> For example, I might want a read-only file system and a compressed file
> system, so I would make two file systems.  If you want the same policies
> for each file system, you''d be better off using directories. 
Continuing
> the policies decision, one of the policies may be backup frequency (I
> really don''t care to backup my music every day, but I would like
to
> backup my home directory every day).
[Hi Richard]

Mostly agreed .... but....   { There''s always a "but" right?
:)  }

The issue is that one can''t see into the future.  And since ZFS 
behavior is largely tied to the level of the ZFS filesystem, it makes 
sense to "err" on the side of creating more filesystems than 
necessary, rather than the opposite (using sub-directories).  To 
clarify: today your advice is accurate because we know - just to list 
a few - of certain ZFS characteristics like, compression, snapshots 
etc.; but we *don''t* know what new ZFS features/facilities might be 
introduced in the near *and* far terms.  So my recommendation is to 
"err" on the side of creating "extra" filesystems.  With
this
recommendation its easy to collapse extra filesystem into 
sub-directories - but not the opposite way around.  Especially when 
you consider snapshots.

And ZFS is still very "young" on the overall development lifecycle; 
who knows what clever features/facilities will be dreamed up next. 
Bear in mind, that if someone had told me I''d be running the ZFS 
features facilites that are already available today (either through 
Solaris or OpenSolaris b78 - including ZFS boot) I would have thought 
that they were on drugs!  :)

..... snip .....

Regards,

Al Hopper  Logical Approach Inc, Plano, TX.  al at logical-approach.com
            Voice: 972.379.2133 Fax: 972.379.2134  Timezone: US CDT
OpenSolaris Governing Board (OGB) Member - Apr 2005 to Mar 2007
http://www.opensolaris.org/os/community/ogb/ogb_2005-2007/
Graduate from "sugar-coating school"?  Sorry - I never attended! :)

True, we''re already planning to take advantage of ZFS quotas at work. 
That alone will mean 100+ filesystems as we''ll have one for each user. 
While there are some disadvantages to the quota management I''m used to,
overall management of ZFS quotas looks very simple, and the ability to roll back
things like individual user profiles is going to be very nice.

And we''re planning on the same approach on the network, with each
separate element given it''s own filesystem.  While it gives us tons of
flexibility and a lot of peace of mind when it comes to snapshots, it means
we''re going to have in the order of 150 filesystems to backup.

However, since everything will be stored within one pool, it''s looking
very much like "zfs snapshot -r" and "zfs send -r" will let
us backup our entire network in one go.  I''m buring build 78 to CD as
we speak to see just how this works.
 
 
This message posted from opensolaris.org