Steve
2008-Oct-15 23:29 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
Greetings. I''m currently looking into creating a better solution for my combination of Sun xVM Virtualbox and ZFS. I have two 500g sata drives configured into a zpool. I''ve used virtualbox for awhile, as well as zfs, so I am familiar with their functionalities. My main question, is more of a theory or ''best practice'' based concern with the two entities. Right now I have a zfs filesystem that holds all of my virtualbox vdi files, as well as the machine files. This really isn''t a space saving situation, because the use of zfs clones/snapshots encompasses the entire zfs filesystem (tank/xvm), so I was wondering if any of you had a better solution. I usually create ''gold'' images of virtual machines after successful OS installation + vbox tools. I would then use the virtualbox clonevdi commands to create more vdi files when I''d like to test different situations. I don''t like the virtualbox snapshots as much as fresh vdi files. As of right now, I was thinking of creating a ''gold image'' zfs filesystem (tank/xvm/VDI/gold), and then the snapshot+clone techniques to create something like: tank/xvm/VDI/clones. The problem is that if I ever add a new ''gold vdi file'', it does not effect the clones, so I won''t be saving the disk space when I clone that vdi file. I am certain I''ll be testing more OS''s than the current ones, so scalability is a concern. Anyone have a similar setup, but with better results, or maybe something I could use as an idea? Thanks in advance, Steve -- This message posted from opensolaris.org
Miles Nordin
2008-Oct-16 00:12 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
>>>>> "s" == Steve <spc1691 at rit.edu> writes:s> the use of zfs s> clones/snapshots encompasses the entire zfs filesystem I use one ZFS filesystem per VDI file. It might be better to use vmdk''s and zvol''s, but right now that''s not what I do. I also often copy ExPee VDI''s onto physical hardware using xxcopy, the MS iSCSI initiator, and FreeNAS, so I use the VDI''s in ZFS act as bootable-machine backups for this ghetto-ghosting scheme. There is this retarded dance you have to do with WPA.DBL---write me if you ever need to do this. s> virtualbox clone the ZFS snapshot/clone commands are a lot faster and less mistake-prone than the virtualbox features, and they support a full clone tree rather than just a linear progression of snapshots. And that ``delete last two snapshots'''' button in vbox is idiotic. After cloning a VDI you need to incant: VBoxManage internalcommands setvdiuuid <cloned filename> to mark a new random uuid onto it, then you can add it with the vdi wizard or whatever you call it. s> if I ever add a new ''gold vdi file'', it does not effect the s> clones, [...] I''ll be testing more OS''s than the current ones, s> so scalability what? -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081015/19d46148/attachment.bin>
Steve
2008-Oct-16 01:37 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
s> if I ever add a new ''gold vdi file'', it does not effect the s> clones, [...] I''ll be testing more OS''s than the current ones, s> so scalability what? ---- What I meant is that if I have a zfs filesystem of a bunch of gold images (VDIs), if I would zfs snapshot/clone the filesystem. If I add another gold image to the original (parent the word??) filesystem, the clones would not be effected in the same way as a normal zfs snapshot of the gold image filesystem would be; which makes sense. I think I''m going to go with creating one zfs filesystem per VDI, then cloning then as needed. I was just wondering if there was a better large scale way to scale a situation like this. Often talking with other people with similar knowledge makes one think of ideas you normally wouldn''t. Thanks Miles. -- This message posted from opensolaris.org
David Abrahams
2008-Oct-17 00:03 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
on Wed Oct 15 2008, Miles Nordin <carton-AT-Ivy.NET> wrote:>>>>>> "s" == Steve <spc1691 at rit.edu> writes: > > s> the use of zfs > s> clones/snapshots encompasses the entire zfs filesystem > > I use one ZFS filesystem per VDI file. It might be better to use > vmdk''s and zvol''s, but right now that''s not what I do. > > I also often copy ExPee VDI''s onto physical hardware using xxcopy, the > MS iSCSI initiator, and FreeNAS, so I use the VDI''s in ZFS act as > bootable-machine backups for this ghetto-ghosting scheme. There is > this retarded dance you have to do with WPA.DBL---write me if you ever > need to do this. > > s> virtualbox clone > > the ZFS snapshot/clone commands are a lot faster and less > mistake-prone than the virtualbox features, and they support a full > clone tree rather than just a linear progression of snapshots.I was going to do the same thing... but I''ve been wondering how to deal with backups to my Amazon s3 storage area. Does zfs send avoid duplicating common data in clones and snapshots? -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Miles Nordin
2008-Oct-17 18:48 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
>>>>> "da" == David Abrahams <dave at boostpro.com> writes:da> how to deal with backups to my Amazon s3 storage area. Does da> zfs send avoid duplicating common data in clones and da> snapshots? how can you afford to use something so expensive as S3 for backups? Anyway ''zfs send'' does avoid duplication but you must never store a ''zfs send'' stream. They''re not robust like ''tar'' and ''cpio'' streams. A bit flip will ruin the entire stream, both before and after the bit flip, while tar/cpio will just search for the next file header and lose very little. Also, correctly restoring them depends on a whole mess of kernel code which is not well-checked for inter-version compatibility. And there is no standalone stream-testing tool. For zpools it seems well-tested that later ZFS code can import earlier zpools, and also x86/SPARC zpools/kernels work together, but neither has been consistently true of the ''zfs send'' format. You can only use ''zfs send'' inside a pipe, where you can try again or give up if it doesn''t work. I asked for access to the si wiki so I could write a clearer ''zfs send'' warning than the rather mild one that''s up there now, but I got no response from siaccount at solarisinternals.com. It sounds silly, but you''d actually be much better off making a compressed zpool on top of an ''mkfile'' vdev, fill it with data, export it, and send that to s3. I don''t know of any proper stream storage format which captures the snapshot/clone tree and also has the relevant characteristics of tarballs: robust to endiness, kernel versions, and bit flips, and validateable without restoring. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081017/0cadcbfb/attachment.bin>
David Abrahams
2008-Oct-17 19:03 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
on Fri Oct 17 2008, Miles Nordin <carton-AT-Ivy.NET> wrote:>>>>>> "da" == David Abrahams <dave at boostpro.com> writes: > > da> how to deal with backups to my Amazon s3 storage area. Does > da> zfs send avoid duplicating common data in clones and > da> snapshots? > > how can you afford to use something so expensive as S3 for backups?Is there a cheaper alternative that will securely and persistently store a copy of my data offsite?> Anyway ''zfs send'' does avoid duplication but you must never store a > ''zfs send'' stream. They''re not robust like ''tar'' and ''cpio'' streams. > A bit flip will ruin the entire stream, both before and after the bit > flip, while tar/cpio will just search for the next file header and > lose very little. Also, correctly restoring them depends on a whole > mess of kernel code which is not well-checked for inter-version > compatibility.I''m on zfs-fuse, so it''s probably not in the kernel... but that probably doesn''t matter much.> And there is no standalone stream-testing tool.OK, then...> For zpools it seems well-tested that later ZFS code can import earlier > zpools, and also x86/SPARC zpools/kernels work together, but neither > has been consistently true of the ''zfs send'' format. You can only use > ''zfs send'' inside a pipe, where you can try again or give up if it > doesn''t work.I understand.> I asked for access to the si wiki so I could write a clearer ''zfs > send'' warning than the rather mild one that''s up there now, but I got > no response from siaccount at solarisinternals.com. > > It sounds silly, but you''d actually be much better off making a > compressed zpool on top of an ''mkfile'' vdev, fill it with data, export > it, and send that to s3.Hmm... not much good for incremental backups, though. Also, it will replicate all the redundant data in snapshots and clones unless I''m misunderstanding you.> I don''t know of any proper stream storage > format which captures the snapshot/clone tree and also has the > relevant characteristics of tarballs: robust to endiness, kernel > versions, and bit flips, and validateable without restoring.tar actually adds redundancy to the degree that it could detect and correct bit flips? This is news to me, and I can''t find anything of the sort on its manpage. Where can I find out more? If that''s true, I suppose one could tar a zfs send stream. -- Dave Abrahams BoostPro Computing http://www.boostpro.com
Miles Nordin
2008-Oct-17 22:15 UTC
[zfs-discuss] Best practice recommendations for ZFS + Virtualbox
>>>>> "da" == David Abrahams <dave at boostpro.com> writes:da> Is there a cheaper alternative that will securely and da> persistently store a copy of my data offsite? rented dedicated servers with disks in them? I have not shopped for this, but for backups it just needs to not lose your data at the same time as the primary copy, not the same quality level as needing to really for certain save the data no matter what. da> tar actually adds redundancy to the degree that it could da> detect and correct bit flips? no, of course not. but the following formats: * tar/cpio * zpool on disk * other filesystems (ufs, xfs, ext3, hfs+, iso9660, ...) All have the characteristic that a flipped bit will usually not affect much---almost always just one file, or sometimes a little more if it''s metadata. Maybe a really small number of bits are extremely special because of filesystem bugs, but experience and hard lessons keeps these minimal. OTOH, If one bit of a zfs send stream is flipped, you are guaranteed to lose the ENTIRE stream. to review, the problems with storing zfs send output are: * the one we just discussed * endiness/zfs-version unportability * lack of validation tool like ''tar t'' or ''zpool scrub'' tar files and (mostly) zpool-inside-mkfile do not have these problems. da> Hmm... not much good for incremental backups, though. right, that''s a big problem. esp with slow zfs send and S3''s copyin/copyout fees. I''ve no answer. da> Also, it will replicate all the redundant data in snapshots da> and clones no, zfs send | zfs recv will recreate the same clone tree on the mkfile-zpool. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 304 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081017/a8da6254/attachment.bin>