I''m using OpenSolaris with ZFS as a backup server. I copy all my data from various sources onto the OpenSolaris server daily, and run a snapshot at the end of each backup. Using gzip-1 compression, mount -F smbfs, and the --in-place and --no-whole-file switches for rsync, I get efficient space usage, only storing the blocks that changed each day. This way, I have a backup server containing all backups for all days going back effectively indefinitely. Works great. Of course, I also want to have something that can be rotated and/or taken offsite. What I''ve done is use an internal drive in the backup server to actually receive and store all the backups and snapshots themselves. Then at the end of the actual backup I run a snapshot, and then do a zfs send -R of my backup pool and all its snapshots to an external drive. Not being able to trust what''s on the drive (its contents could possibly have changed since last time I used it, and I want every snapshot on every external drive), I wipe the external drive clean and then have it receive the full contents of the non-incremental zfs send -R backuppool I mentioned above. This works. However, it''s painfully slow. I get the impression that zfs is de-compressing and then re-compressing the data instead of transferring it in its compressed state, and then when the incrementals start copying over (the snapshots themselves), it gets drastically slower. The whole process works, but I''m thinking that when I start getting too many snapshots, it won''t finish overnight and will run into the next day. I don''t want to just copy over the contents of my most recent snapshot on my backup server to the external drive then run a snapshot on the external drive, because I''d like each external drive to contain ALL the snapshots from the internal drive. Is there any way to speed up a compressed zfs send -R? Or is there some other way to approach this? Maybe some way to do a bit-level clone of the internal drive to the external drive (the internal backup drive is not the same as the OS drive, so it could be unmounted), or SNDR replication or something? Thanks! -- This message posted from opensolaris.org
On Thu, 22 Jan 2009, BJ Quinn wrote:> Is there any way to speed up a compressed zfs send -R? Or is there > some other way to approach this? Maybe some way to do a bit-level > clone of the internal drive to the external drive (the internal > backup drive is not the same as the OS drive, so it could be > unmounted), or SNDR replication or something?Maybe you can make your external drive a ''mirror'' so that it gets resilvered and can be removed. Of course you may want to have several of these drives. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
That sounds like a great idea if I can get it to work-- I get how to add a drive to a zfs mirror, but for the life of me I can''t find out how to safely remove a drive from a mirror. Also, if I do remove the drive from the mirror, then pop it back up in some unsuspecting (and unrelated) Solaris box, will it just see a drive with a pool on it and let me mount it up? What about when I pop in the drive to be resilvered, but right before I add it back to the mirror, will Solaris get upset that I have two drives both with the same pool name? -- This message posted from opensolaris.org
BJ Quinn wrote:> That sounds like a great idea if I can get it to work-- > >What does?> I get how to add a drive to a zfs mirror, but for the life of me I can''t find out how to safely remove a drive from a mirror. > >Have you tried "man zpool"? See the entry for detach.> Also, if I do remove the drive from the mirror, then pop it back up in some unsuspecting (and unrelated) Solaris box, will it just see a drive with a pool on it and let me mount it up?You should be able to import it, but I haven''t tried.> What about when I pop in the drive to be resilvered, but right before I add it back to the mirror, will Solaris get upset that I have two drives both with the same pool name? >No, you have to do a manual import. -- Ian.
>> What about when I pop in the drive to be resilvered, but right before I add it back to the mirror, will Solaris get upset that I have two drives both with the same pool name? >No, you have to do a manual import.What you mean is that if Solaris/ZFS detects a drive with an identical pool name to a currently mounted pool, that it will safely not disrupt the mounted pool and simply not mount the same-named pool on the newly inserted drive? Can I mount a pool "as" another pool name? Message was edited by: bjquinn -- This message posted from opensolaris.org
On 28 Jan 2009, at 19:40, BJ Quinn wrote:>>> What about when I pop in the drive to be resilvered, but right >>> before I add it back to the mirror, will Solaris get upset that I >>> have two drives both with the same pool name? >> No, you have to do a manual import. > > What you mean is that if Solaris/ZFS detects a drive with an > identical pool name to a currently mounted pool, that it will safely > not disrupt the mounted pool and simply not mount the same-named > pool on the newly inserted drive? > > Can I mount a pool "as" another pool name?Yes: "zpool import oldname newname" Cheers, Chris
bjquinn - on article - http://www.opensolaris.org/jive/thread.jspa?threadID=89567 i would like to contact you. i am new to zfs and exactly need what you mentioned your requirements were and that you figured out a solution for it. would you like to share the solution step by step with me. please contact me at nathulal [at] babulal dto com -- This message posted from opensolaris.org