zfs discuss - Oct 2008 - Best practice recommendations for ZFS + Virtualbox

Greetings.

I''m currently looking into creating a better solution for my
combination of Sun xVM Virtualbox and ZFS.

I have two 500g sata drives configured into a zpool. I''ve used
virtualbox for awhile, as well as zfs, so I am familiar with their
functionalities. My main question, is more of a theory or ''best
practice'' based concern with the two entities.

Right now I have a zfs filesystem that holds all of my virtualbox vdi files, as
well as the machine files. This really isn''t a space saving situation,
because the use of zfs clones/snapshots encompasses the entire zfs filesystem
(tank/xvm), so I was wondering if any of you had a better solution.

I usually create ''gold'' images of virtual machines after
successful OS installation + vbox tools. I would then use the virtualbox
clonevdi commands to create more vdi files when I''d like to test
different situations. I don''t like the virtualbox snapshots as much as
fresh vdi files.

As of right now, I was thinking of creating a ''gold image'' zfs
filesystem (tank/xvm/VDI/gold), and then the snapshot+clone techniques  to
create something like: tank/xvm/VDI/clones. The problem is that if I ever add a
new ''gold vdi file'', it does not effect the clones, so I
won''t be saving the disk space when I clone that vdi file. I am certain
I''ll be testing more OS''s than the current ones, so
scalability is a concern.

Anyone have a similar setup, but with better results, or maybe something I could
use as an idea?

Thanks in advance,


                Steve
--
This message posted from opensolaris.org

>>>>> "s" == Steve  <spc1691 at rit.edu> writes:
     s> the use of zfs
     s> clones/snapshots encompasses the entire zfs filesystem

I use one ZFS filesystem per VDI file.  It might be better to use
vmdk''s and zvol''s, but right now that''s not what I
do.

I also often copy ExPee VDI''s onto physical hardware using xxcopy, the
MS iSCSI initiator, and FreeNAS, so I use the VDI''s in ZFS act as
bootable-machine backups for this ghetto-ghosting scheme.  There is
this retarded dance you have to do with WPA.DBL---write me if you ever
need to do this.

     s> virtualbox clone

the ZFS snapshot/clone commands are a lot faster and less
mistake-prone than the virtualbox features, and they support a full
clone tree rather than just a linear progression of snapshots.  And
that ``delete last two snapshots'''' button in vbox is idiotic.

After cloning a VDI you need to incant:

VBoxManage internalcommands setvdiuuid <cloned filename>

to mark a new random uuid onto it, then you can add it with the vdi
wizard or whatever you call it.

     s> if I ever add a new ''gold vdi file'', it does not
effect the
     s> clones, [...] I''ll be testing more OS''s than the
current ones,
     s> so scalability

what?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 304 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081015/19d46148/attachment.bin>

s> if I ever add a new ''gold vdi file'', it does not effect
the
s> clones, [...] I''ll be testing more OS''s than the current
ones,
s> so scalability

what?


----

What I meant is that if I have a zfs filesystem of a bunch of gold images
(VDIs), if I would zfs  snapshot/clone the filesystem. If I add another gold
image to the original (parent the word??) filesystem, the clones would not be
effected in the same way as a normal zfs snapshot of the gold image filesystem
would be; which makes sense.

I think I''m going to go with creating one zfs filesystem per VDI, then
cloning then as needed. I was just wondering if there was a better large scale
way to scale a situation like this. Often talking with other people with similar
knowledge makes one think of ideas you normally wouldn''t.

Thanks Miles.
--
This message posted from opensolaris.org

on Wed Oct 15 2008, Miles Nordin <carton-AT-Ivy.NET> wrote:
>>>>>> "s" == Steve  <spc1691 at rit.edu>
writes:
>
>      s> the use of zfs
>      s> clones/snapshots encompasses the entire zfs filesystem
>
> I use one ZFS filesystem per VDI file.  It might be better to use
> vmdk''s and zvol''s, but right now that''s not what
I do.
>
> I also often copy ExPee VDI''s onto physical hardware using xxcopy,
the
> MS iSCSI initiator, and FreeNAS, so I use the VDI''s in ZFS act as
> bootable-machine backups for this ghetto-ghosting scheme.  There is
> this retarded dance you have to do with WPA.DBL---write me if you ever
> need to do this.
>
>      s> virtualbox clone
>
> the ZFS snapshot/clone commands are a lot faster and less
> mistake-prone than the virtualbox features, and they support a full
> clone tree rather than just a linear progression of snapshots.  
I was going to do the same thing... but I''ve been wondering how to deal
with backups to my Amazon s3 storage area.  Does zfs send avoid
duplicating common data in clones and snapshots?

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

>>>>> "da" == David Abrahams <dave at
boostpro.com> writes:
    da> how to deal with backups to my Amazon s3 storage area.  Does
    da> zfs send avoid duplicating common data in clones and
    da> snapshots?

how can you afford to use something so expensive as S3 for backups?
Anyway ''zfs send'' does avoid duplication but you must never
store a
''zfs send'' stream.  They''re not robust like
''tar'' and ''cpio'' streams.
A bit flip will ruin the entire stream, both before and after the bit
flip, while tar/cpio will just search for the next file header and
lose very little.  Also, correctly restoring them depends on a whole
mess of kernel code which is not well-checked for inter-version
compatibility.  And there is no standalone stream-testing tool.  

For zpools it seems well-tested that later ZFS code can import earlier
zpools, and also x86/SPARC zpools/kernels work together, but neither
has been consistently true of the ''zfs send'' format.  You can
only use
''zfs send'' inside a pipe, where you can try again or give up
if it
doesn''t work.

I asked for access to the si wiki so I could write a clearer ''zfs
send'' warning than the rather mild one that''s up there now,
but I got
no response from siaccount at solarisinternals.com.

It sounds silly, but you''d actually be much better off making a
compressed zpool on top of an ''mkfile'' vdev, fill it with
data, export
it, and send that to s3.  I don''t know of any proper stream storage
format which captures the snapshot/clone tree and also has the
relevant characteristics of tarballs: robust to endiness, kernel
versions, and bit flips, and validateable without restoring.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 304 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081017/0cadcbfb/attachment.bin>

on Fri Oct 17 2008, Miles Nordin <carton-AT-Ivy.NET> wrote:
>>>>>> "da" == David Abrahams <dave at
boostpro.com> writes:
>
>     da> how to deal with backups to my Amazon s3 storage area.  Does
>     da> zfs send avoid duplicating common data in clones and
>     da> snapshots?
>
> how can you afford to use something so expensive as S3 for backups?
Is there a cheaper alternative that will securely and persistently store
a copy of my data offsite?
> Anyway ''zfs send'' does avoid duplication but you must
never store a
> ''zfs send'' stream.  They''re not robust like
''tar'' and ''cpio'' streams.
> A bit flip will ruin the entire stream, both before and after the bit
> flip, while tar/cpio will just search for the next file header and
> lose very little.  Also, correctly restoring them depends on a whole
> mess of kernel code which is not well-checked for inter-version
> compatibility.  
I''m on zfs-fuse, so it''s probably not in the kernel... but
that probably
doesn''t matter much.
> And there is no standalone stream-testing tool.  
OK, then...
> For zpools it seems well-tested that later ZFS code can import earlier
> zpools, and also x86/SPARC zpools/kernels work together, but neither
> has been consistently true of the ''zfs send'' format.  You
can only use
> ''zfs send'' inside a pipe, where you can try again or give
up if it
> doesn''t work.
I understand.
> I asked for access to the si wiki so I could write a clearer ''zfs
> send'' warning than the rather mild one that''s up there
now, but I got
> no response from siaccount at solarisinternals.com.
>
> It sounds silly, but you''d actually be much better off making a
> compressed zpool on top of an ''mkfile'' vdev, fill it with
data, export
> it, and send that to s3.  
Hmm... not much good for incremental backups, though.  Also, it will
replicate all the redundant data in snapshots and clones unless I''m
misunderstanding you.
> I don''t know of any proper stream storage
> format which captures the snapshot/clone tree and also has the
> relevant characteristics of tarballs: robust to endiness, kernel
> versions, and bit flips, and validateable without restoring.
tar actually adds redundancy to the degree that it could detect and
correct bit flips?  This is news to me, and I can''t find anything of
the
sort on its manpage.  Where can I find out more?

If that''s true, I suppose one could tar a zfs send stream.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com

>>>>> "da" == David Abrahams <dave at
boostpro.com> writes:
    da> Is there a cheaper alternative that will securely and
    da> persistently store a copy of my data offsite?

rented dedicated servers with disks in them?  I have not shopped for
this, but for backups it just needs to not lose your data at the same
time as the primary copy, not the same quality level as needing to
really for certain save the data no matter what.

    da> tar actually adds redundancy to the degree that it could
    da> detect and correct bit flips?

no, of course not.  but the following formats:

 * tar/cpio
 * zpool on disk
 * other filesystems (ufs, xfs, ext3, hfs+, iso9660, ...)

All have the characteristic that a flipped bit will usually not affect
much---almost always just one file, or sometimes a little more if it''s
metadata.  Maybe a really small number of bits are extremely special
because of filesystem bugs, but experience and hard lessons keeps
these minimal.

OTOH, If one bit of a zfs send stream is flipped, you are guaranteed
to lose the ENTIRE stream.

to review, the problems with storing zfs send output are:

 * the one we just discussed
 * endiness/zfs-version unportability
 * lack of validation tool like ''tar t'' or ''zpool
scrub''

tar files and (mostly) zpool-inside-mkfile do not have these problems.

    da> Hmm... not much good for incremental backups, though.

right, that''s a big problem.  esp with slow zfs send and S3''s
copyin/copyout fees.  I''ve no answer.

    da>   Also, it will replicate all the redundant data in snapshots
    da> and clones

no, zfs send | zfs recv will recreate the same clone tree on the
mkfile-zpool.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 304 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081017/a8da6254/attachment.bin>