I have a series of daily snapshots against a set of data that go for several months, but then the server crashed. In a hurry, we set up a new server and just copied over the live data and didn''t bother with the snapshots (since zfs send/recv was too slow and would have taken hours and hours to restore). We''ve now run on the replacement server for a year or so and it''s time to upgrade to a new, faster server. As part of building up our newest server, I wanted to combine the older snapshots with the daily snapshots generated on the server that is currently running. I was wondering what the proper way to do this might be. I was considering the following process for building up the new server : 1. Copy over all the snapshots from a backup of the server that crashed (11/01/2008 - 7/14/2009) using zfs send/recv 2. Copy over the oldest snapshot from the current server (7/15/2009) using rsync so that the data from that snapshot is the live filesystem data on the new server. 3. Take a snapshot on the new server and call it the same thing as the snapshot that I copied the data from (i.e. datapool at NIGHTLY20090715) 4. Do an incremental zfs send/recv from 7/15/2009 to today from the current server. I don''t know if this would work, or if it would leave me in a consistent state even if I did make it work. Any suggestions? -- This message posted from opensolaris.org
On Tue, Jun 8, 2010 at 10:51 AM, BJ Quinn <bjquinn at seidal.com> wrote:> 3. ?Take a snapshot on the new server and call it the same thing as the snapshot that I copied the data from (i.e. datapool at NIGHTLY20090715)It won''t work, because the two snapshots are different. It doesn''t matter if they have same name, the snapshots are of two separate filesystems. -B -- Brandon High : bhigh at freaks.com
Is there any way to merge them back together? I really need the history data going back as far as possible, and I''d like to be able to access it from the same place . I mean, worst case scenario, I could rsync the contents of each snapshot to the new filesystem and take a snapshot for each one, but surely there''s a better way than that? -- This message posted from opensolaris.org
You might bring over all of your old data and snaps, then clone that into a new volume. Bring your recent stuff into the clone. Since the clone only updates blocks that are different than the underlying snap, you may see a significant storage savings. Two clones could even be made - one for your live data, another to access the historical data. -Scott -- This message posted from opensolaris.org
On Tue, Jun 8, 2010 at 12:52 PM, BJ Quinn <bjquinn at seidal.com> wrote:> Is there any way to merge them back together? ?I really need the history data going back as far as possible, and I''d like to be able to access it from the same place . ?I mean, worst case scenario, I could rsync the contents of each snapshot to the new filesystem and take a snapshot for each one, but surely there''s a better way than that?You won''t be able to keep them in the same dataset. Because you started your second server from a separate point, there''s no way to merge them. If you''d copied only the most recent snapshot from your old server onto the new one, you could merge them because they would have had a snapshot in common. You''ll have to use two datasets to hold the data from your old server and the data from your current server if you want to keep everything exactly as it is on the sources -B -- Brandon High : bhigh at freaks.com
Not exactly sure how to do what you''re recommending -- are you suggesting I go ahead with using rsync to bring in each snapshot, but to bring it into to a clone of the old set of snapshots? Is there another way to bring my recent stuff in to the clone? If so, then as for the storage savings, I learned a long time ago that rsync --inplace --no-whole-file has the same effect - it makes sure to only touch blocks that changed, so in theory I ought to be able to rsync over my snapshots intelligently without wasting any more space than they took up to begin with. Not sure if that''s what you meant. Thanks! -- This message posted from opensolaris.org
Ugh, yeah, I''ve learned by now that you always want at least that one snapshot in common to keep the continuity in the dataset. Wouldn''t I be able to recreate effectively the same thing by rsync''ing over each snapshot one by one? It may take a while, and I''d have to use the --inplace and --no-whole-file switches to ensure that I didn''t overwrite anything except changed blocks when bringing over each snapshot (avoiding marking all blocks as changed and wasting all sorts of space), but shouldn''t that work at least? I''d hate to have to resort to the two data sets thing. -- This message posted from opensolaris.org
On Tue, Jun 8, 2010 at 4:29 PM, BJ Quinn <bjquinn at seidal.com> wrote:> Ugh, yeah, I''ve learned by now that you always want at least that one snapshot in common to keep the continuity in the dataset. ?Wouldn''t I be able to recreate effectively the same thing by rsync''ing over each snapshot one by one? ?It may take a while, and I''d have to use the --inplace and --no-whole-file switches to ensure that I didn''t overwrite anything except changed blocks when bringing over each snapshot (avoiding marking all blocks as changed and wasting all sorts of space), but shouldn''t that work at least? ?I''d hate to have to resort to the two data sets thing.You could do that, but the snapshot''s creation property is going to be wrong for any snaps that you create that way. Some file properties (such as atime, mtime, and ctime) won''t be preserved by the rsync method. If it''s important to keep an exact copy of the snapshots, you''ll have to receive from each of your old hosts into different datasets, one for each source host. If disk space is a concern, you could enable dedup, which should do well between the last snap of the old server and the first snap of the current server. -B -- Brandon High : bhigh at freaks.com
In my case, snapshot creation time and atime don''t matter. I think rsync can preserve mtime and ctime, though. I''ll have to double check that. I''d love to enable dedup. Trying to stay on "stable" releases of OpenSolaris for whatever that''s worth, and I can''t seem to find a link to download 2010.06. :) At any rate, thanks for the help! I tried rsync''ing a few snapshots, and it doesn''t look like it will take as long as I thought. At first I feared it might run for weeks! -- This message posted from opensolaris.org