I just received a special offer from Sun (marketing...) promising that I will learn "How to use ZFS snapshots for backup and restore purposes." The relevant doc is at... https://www.sun.com/offers/docs/zfs_snapshots.pdf It says: === Begin quote ==Archiving and Restoring Snapshots Another use of snapshots is to create archives for long-term storage elsewhere. In the following sequence of commands, we send the snapshot into a file and then compress it. It can then be retrieved from the file when required. This is also shown: # zfs send pool/filesystem at thursday > /var/tmp/thursday.snap # gzip -9 -v /var/tmp/thursday.snap # zfs create pool/thursday # gzip -d -c /var/tmp/thursday.snap.gz | zfs receive -F pool/thursday === End quote == There have been several threads on this list saying not to do that. It seems that this guide aimed at new users is inviting people to do things that will lead them to unsympathetic ears if things go poorly. -- Mike Gerdts http://mgerdts.blogspot.com/
Cindy.Swearingen at Sun.COM
2009-Sep-02 21:06 UTC
[zfs-discuss] Archiving and Restoring Snapshots
Hi Mike, I reviewed this doc and the only issue I have with it now is that uses /var/tmp an an example of storing snapshots in "long-term storage" elsewhere. For short-term storage, storing a snapshot as a file is an acceptable solution as long as you verify that the snapshots as files are valid like you would for any important data. I test root pool snapshot recovery for every Solaris 10 release and have also for Nevada releases. The point that needs to be clear is that you verify the file or snapshot. I have updated our root pool recovery steps to be performed either using snapshots as remotely stored files or remotely stored snapshots, here: http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#Complete_Solaris_ZFS_Root_Pool_Recovery I will follow-up on this particular marketing document. Thanks for pointing it out... Cindy On 09/02/09 12:37, Mike Gerdts wrote:> I just received a special offer from Sun (marketing...) promising that > I will learn "How to use ZFS snapshots for backup and restore > purposes." The relevant doc is at... > > https://www.sun.com/offers/docs/zfs_snapshots.pdf > > It says: > > === Begin quote ==> Archiving and Restoring Snapshots > > Another use of snapshots is to create archives for long-term storage elsewhere. > > In the following sequence of commands, we send the snapshot into a > file and then compress it. It can then be > retrieved from the file when required. This is also shown: > > # zfs send pool/filesystem at thursday > /var/tmp/thursday.snap > # gzip -9 -v /var/tmp/thursday.snap > # zfs create pool/thursday > # gzip -d -c /var/tmp/thursday.snap.gz | zfs receive -F pool/thursday > === End quote ==> > There have been several threads on this list saying not to do that. > It seems that this guide aimed at new users is inviting people to do > things that will lead them to unsympathetic ears if things go poorly. >
Thanks Cindy! Mike, et.al., I think the confusion is surrounding replacing an enterprise backup scheme with send-to-file. There is nothing wrong with send-to-file, it functions as designed. But it isn''t designed to be a full-featured backup solution. IMHO, the title, "Archiving and Restoring Snapshots" clearly shows their intent. -- richard On Sep 2, 2009, at 2:06 PM, Cindy.Swearingen at Sun.COM wrote:> Hi Mike, > > I reviewed this doc and the only issue I have with it now is that uses > /var/tmp an an example of storing snapshots in "long-term storage" > elsewhere. > > For short-term storage, storing a snapshot as a file is an acceptable > solution as long as you verify that the snapshots as files are valid > like you would for any important data. I test root pool snapshot > recovery for every Solaris 10 release and have also for Nevada > releases. > > The point that needs to be clear is that you verify the file or > snapshot. I have updated our root pool recovery steps to be performed > either using snapshots as remotely stored files or remotely stored > snapshots, here: > > http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#Complete_Solaris_ZFS_Root_Pool_Recovery > > I will follow-up on this particular marketing document. > > Thanks for pointing it out... > > Cindy > > On 09/02/09 12:37, Mike Gerdts wrote: >> I just received a special offer from Sun (marketing...) promising >> that >> I will learn "How to use ZFS snapshots for backup and restore >> purposes." The relevant doc is at... >> https://www.sun.com/offers/docs/zfs_snapshots.pdf >> It says: >> === Begin quote ==>> Archiving and Restoring Snapshots >> Another use of snapshots is to create archives for long-term >> storage elsewhere. >> In the following sequence of commands, we send the snapshot into a >> file and then compress it. It can then be >> retrieved from the file when required. This is also shown: >> # zfs send pool/filesystem at thursday > /var/tmp/thursday.snap >> # gzip -9 -v /var/tmp/thursday.snap >> # zfs create pool/thursday >> # gzip -d -c /var/tmp/thursday.snap.gz | zfs receive -F pool/thursday >> === End quote ==>> There have been several threads on this list saying not to do that. >> It seems that this guide aimed at new users is inviting people to do >> things that will lead them to unsympathetic ears if things go poorly. > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
On Wed, Sep 2, 2009 at 4:06 PM, <Cindy.Swearingen at sun.com> wrote:> Hi Mike, > > I reviewed this doc and the only issue I have with it now is that uses > /var/tmp an an example of storing snapshots in "long-term storage" > elsewhere.One other point comes from zfs(1M): The format of the stream is evolving. No backwards com- patibility is guaranteed. You may not be able to receive your streams on future versions of ZFS. Others have argued that it is not as flexible of a long-term storage mechanism because a restore is all or nothing. In comparison to some other backup formats the implications are: - You cannot selectively restore files. - There is no integrity check or error correction capability in the stream. As such, if you have a 1 TB stream and somewhere around 1 MB a bit is flipped, all is lost. The most conservative (and somewhat common) advice with the output of zfs send is that it should only be trusted if it is immediately consumed by zfs receive. Of course, different people have different needs. For most of my needs, the level of reliability that I would get out of storing the output of zfs send is sufficient.> For short-term storage, storing a snapshot as a file is an acceptable > solution as long as you verify that the snapshots as files are valid > like you would for any important data. I test root pool snapshot > recovery for every Solaris 10 release and have also for Nevada releases. > > The point that needs to be clear is that you verify the file or > snapshot. I have updated our root pool recovery steps to be performed > either using snapshots as remotely stored files or remotely stored > snapshots, here:Storing the output itself on zfs + testing a "zfs receive" of the stored file would go a long ways toward ensuring that the stored file is usable for restores. Storing a strong hash (e.g. SHA*) of the the file would help for the inevitable transfer of that file to some other file system or between file systems. But this is all beyond newbie level of documentation, I think.> http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#Complete_Solaris_ZFS_Root_Pool_Recovery > > I will follow-up on this particular marketing document. > > Thanks for pointing it out... > > Cindy > > On 09/02/09 12:37, Mike Gerdts wrote: >> >> I just received a special offer from Sun (marketing...) promising that >> I will learn "How to use ZFS snapshots for backup and restore >> purposes." ?The relevant doc is at... >> >> https://www.sun.com/offers/docs/zfs_snapshots.pdf >> >> It says: >> >> === Begin quote ==>> Archiving and Restoring Snapshots >> >> Another use of snapshots is to create archives for long-term storage >> elsewhere. >> >> In the following sequence of commands, we send the snapshot into a >> file and then compress it. It can then be >> retrieved from the file when required. This is also shown: >> >> # zfs send pool/filesystem at thursday > /var/tmp/thursday.snap >> # gzip -9 -v /var/tmp/thursday.snap >> # zfs create pool/thursday >> # gzip -d -c /var/tmp/thursday.snap.gz | zfs receive -F pool/thursday >> === End quote ==>> >> There have been several threads on this list saying not to do that. >> It seems that this guide aimed at new users is inviting people to do >> things that will lead them to unsympathetic ears if things go poorly. >> >-- Mike Gerdts http://mgerdts.blogspot.com/
On Wed, Sep 2, 2009 at 4:46 PM, Richard Elling<richard.elling at gmail.com> wrote:> Thanks Cindy! > > Mike, et.al., > I think the confusion is surrounding replacing an enterprise backup > scheme with send-to-file. There is nothing wrong with send-to-file, > it functions as designed. ?But it isn''t designed to be a full-featured > backup solution. IMHO, the title, "Archiving and Restoring Snapshots" > clearly shows their intent. > ?-- richardIn my world, "archiving" is something you do when you want to be sure that you can get back at it several years from now. The advice I have seen so far on this list is that the output of "zfs send" is not suitable for archival purposes. It doesn''t matter if it is corporate financial data or family photos. For an enterprise backup situation, I actually find zfs send output much more palatable because backups are run very frequently on data that doesn''t change that often. Tape rotation policies mean that I have multiple full backups and various levels of incrementals at my disposal. In an enterprise where I have change management procedures that allow me to understand what changed from week to week, going back to a prior full backup is not that big of a deal for anything except transactional data. But transactional data tends not to be suited for generic "back up the file system with tar" unless the transactions aren''t very important. And yes - thank you Cindy. I tend to agree with the spirit of the docs, but I''ve also seen several conversations where storing "zfs send" output is highly discouraged. My intent of bringing this up was to head off the eventual situation where someone came to the list saying that they lost their data doing exactly what Sun told them to do. -- Mike Gerdts http://mgerdts.blogspot.com/
I agree, mailing that to all Sun customers is something I think is likely to turn around and bite you. A lot of people are now going to use that to archive their data, and some of them are not going to be happy when months or years down the line they try to restore it and find that the ''zfs receive'' just fails, with no hope of recovering their data. And they''re not going to blame the device that''s corrupted a bit, they''re going to come here blaming Sun and ZFS. Archiving to a format with such a high risk of loosing everything sounds like incredibly bad advice. Especially when the article actually uses the term "create archives for long-term storage". Long term storage doesn''t mean "storage that could break with the next upgrade", or "storage that will fail if even a single bit has been corrupted anywhere in the process". Sun really need to decide what they are doing with zfs send/receive because we''re getting very mixed messages now. -- This message posted from opensolaris.org
I agree and Cindy Swearingen and I are talking to marketing to get this fixed. Thanks to all for bringing this to our attention. lori On 09/03/09 00:55, Ross wrote:> I agree, mailing that to all Sun customers is something I think is likely to turn around and bite you. > > A lot of people are now going to use that to archive their data, and some of them are not going to be happy when months or years down the line they try to restore it and find that the ''zfs receive'' just fails, with no hope of recovering their data. > > And they''re not going to blame the device that''s corrupted a bit, they''re going to come here blaming Sun and ZFS. Archiving to a format with such a high risk of loosing everything sounds like incredibly bad advice. Especially when the article actually uses the term "create archives for long-term storage". > > Long term storage doesn''t mean "storage that could break with the next upgrade", or "storage that will fail if even a single bit has been corrupted anywhere in the process". Sun really need to decide what they are doing with zfs send/receive because we''re getting very mixed messages now. >
On Sep 2, 2009, at 11:55 PM, Ross wrote:> I agree, mailing that to all Sun customers is something I think is > likely to turn around and bite you.Some points to help clarify the situation: 1. There is no other way to archive a dataset than using a snapshot 2. You cannot build a zpool on a tape 3. The stability of the protocol is only a problem if it becomes impossible to run some version of OpenSolaris on hardware that is needed to receive the snapshot. Given the ubiquity of virtualization and the x86 legacy, I don''t think this is a problem for at least the expected lifetime of the storage medium.> A lot of people are now going to use that to archive their data, and > some of them are not going to be happy when months or years down the > line they try to restore it and find that the ''zfs receive'' just > fails, with no hope of recovering their data. > > And they''re not going to blame the device that''s corrupted a bit, > they''re going to come here blaming Sun and ZFS. Archiving to a > format with such a high risk of loosing everything sounds like > incredibly bad advice. Especially when the article actually uses > the term "create archives for long-term storage". > > Long term storage doesn''t mean "storage that could break with the > next upgrade", or "storage that will fail if even a single bit has > been corrupted anywhere in the process". Sun really need to decide > what they are doing with zfs send/receive because we''re getting very > mixed messages now.Data integrity is a problem for all archiving systems, not just ZFS. -- richard
yes to all the comments below. Those are all mitigating factors. But I also agree with Ross and Mike and others that we should be more clear about when send/recv is appropriate and when it''s not the best choice. We''re looking into it. Lori On 09/03/09 10:06, Richard Elling wrote:> On Sep 2, 2009, at 11:55 PM, Ross wrote: > >> I agree, mailing that to all Sun customers is something I think is >> likely to turn around and bite you. > > Some points to help clarify the situation: > > 1. There is no other way to archive a dataset than using a snapshot > > 2. You cannot build a zpool on a tape > > 3. The stability of the protocol is only a problem if it becomes > impossible > to run some version of OpenSolaris on hardware that is needed to > receive the snapshot. Given the ubiquity of virtualization and > the x86 > legacy, I don''t think this is a problem for at least the > expected lifetime > of the storage medium. > >> A lot of people are now going to use that to archive their data, and >> some of them are not going to be happy when months or years down the >> line they try to restore it and find that the ''zfs receive'' just >> fails, with no hope of recovering their data. >> >> And they''re not going to blame the device that''s corrupted a bit, >> they''re going to come here blaming Sun and ZFS. Archiving to a >> format with such a high risk of loosing everything sounds like >> incredibly bad advice. Especially when the article actually uses the >> term "create archives for long-term storage". >> >> Long term storage doesn''t mean "storage that could break with the >> next upgrade", or "storage that will fail if even a single bit has >> been corrupted anywhere in the process". Sun really need to decide >> what they are doing with zfs send/receive because we''re getting very >> mixed messages now. > > Data integrity is a problem for all archiving systems, not just ZFS. > -- richard > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Please unsubscribe me COLLIER -----Original Message----- From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Lori Alt Sent: Thursday, September 03, 2009 9:14 AM To: Richard Elling Cc: zfs-discuss at opensolaris.org; Ross Subject: Re: [zfs-discuss] Archiving and Restoring Snapshots yes to all the comments below. Those are all mitigating factors. But I also agree with Ross and Mike and others that we should be more clear about when send/recv is appropriate and when it''s not the best choice. We''re looking into it. Lori On 09/03/09 10:06, Richard Elling wrote:> On Sep 2, 2009, at 11:55 PM, Ross wrote: > >> I agree, mailing that to all Sun customers is something I think is >> likely to turn around and bite you. > > Some points to help clarify the situation: > > 1. There is no other way to archive a dataset than using a snapshot > > 2. You cannot build a zpool on a tape > > 3. The stability of the protocol is only a problem if it becomes > impossible > to run some version of OpenSolaris on hardware that is needed to > receive the snapshot. Given the ubiquity of virtualization and > the x86 > legacy, I don''t think this is a problem for at least the > expected lifetime > of the storage medium. > >> A lot of people are now going to use that to archive their data, and >> some of them are not going to be happy when months or years down the >> line they try to restore it and find that the ''zfs receive'' just >> fails, with no hope of recovering their data. >> >> And they''re not going to blame the device that''s corrupted a bit, >> they''re going to come here blaming Sun and ZFS. Archiving to a >> format with such a high risk of loosing everything sounds like >> incredibly bad advice. Especially when the article actually uses the >> term "create archives for long-term storage". >> >> Long term storage doesn''t mean "storage that could break with the >> next upgrade", or "storage that will fail if even a single bit has >> been corrupted anywhere in the process". Sun really need to decide >> what they are doing with zfs send/receive because we''re getting very >> mixed messages now. > > Data integrity is a problem for all archiving systems, not just ZFS. > -- richard > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss_______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Please unsubscribe me COLLIER -----Original Message----- From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Richard Elling Sent: Thursday, September 03, 2009 9:06 AM To: Ross Cc: zfs-discuss at opensolaris.org Subject: Re: [zfs-discuss] Archiving and Restoring Snapshots On Sep 2, 2009, at 11:55 PM, Ross wrote:> I agree, mailing that to all Sun customers is something I think is > likely to turn around and bite you.Some points to help clarify the situation: 1. There is no other way to archive a dataset than using a snapshot 2. You cannot build a zpool on a tape 3. The stability of the protocol is only a problem if it becomes impossible to run some version of OpenSolaris on hardware that is needed to receive the snapshot. Given the ubiquity of virtualization and the x86 legacy, I don''t think this is a problem for at least the expected lifetime of the storage medium.> A lot of people are now going to use that to archive their data, and > some of them are not going to be happy when months or years down the > line they try to restore it and find that the ''zfs receive'' just > fails, with no hope of recovering their data. > > And they''re not going to blame the device that''s corrupted a bit, > they''re going to come here blaming Sun and ZFS. Archiving to a > format with such a high risk of loosing everything sounds like > incredibly bad advice. Especially when the article actually uses > the term "create archives for long-term storage". > > Long term storage doesn''t mean "storage that could break with the > next upgrade", or "storage that will fail if even a single bit has > been corrupted anywhere in the process". Sun really need to decide > what they are doing with zfs send/receive because we''re getting very > mixed messages now.Data integrity is a problem for all archiving systems, not just ZFS. -- richard _______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Please unsubscribe me COLLIER -----Original Message----- From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-bounces at opensolaris.org] On Behalf Of Lori Alt Sent: Thursday, September 03, 2009 8:59 AM To: Ross Cc: zfs-discuss at opensolaris.org Subject: Re: [zfs-discuss] Archiving and Restoring Snapshots I agree and Cindy Swearingen and I are talking to marketing to get this fixed. Thanks to all for bringing this to our attention. lori On 09/03/09 00:55, Ross wrote:> I agree, mailing that to all Sun customers is something I think is likelyto turn around and bite you.> > A lot of people are now going to use that to archive their data, and someof them are not going to be happy when months or years down the line they try to restore it and find that the ''zfs receive'' just fails, with no hope of recovering their data.> > And they''re not going to blame the device that''s corrupted a bit, they''regoing to come here blaming Sun and ZFS. Archiving to a format with such a high risk of loosing everything sounds like incredibly bad advice. Especially when the article actually uses the term "create archives for long-term storage".> > Long term storage doesn''t mean "storage that could break with the nextupgrade", or "storage that will fail if even a single bit has been corrupted anywhere in the process". Sun really need to decide what they are doing with zfs send/receive because we''re getting very mixed messages now.>_______________________________________________ zfs-discuss mailing list zfs-discuss at opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
> Some points to help clarify the situation: > > 1. There is no other way to archive a dataset than > n using a snapshotOther than tar, star, and numerous other archive utilities. The fact that ZFS doesn''t have any alternative built in doesn''t mean that dumping a send to a file suddenly becomes a good idea. Especially when there is no step in those instructions to verify that the archived file is actually a valid zfs send stream.> 2. You cannot build a zpool on a tapeSo you need a different solution for tape, possibly streaming the files to tape, with the zpool configuration also documented and backed up somehow if necessary. -- This message posted from opensolaris.org
Sometimes trying to be clear gets me in trouble :-) On Sep 3, 2009, at 10:24 AM, Ross wrote:>> Some points to help clarify the situation: >> >> 1. There is no other way to archive a dataset than >> n using a snapshot > > Other than tar, star, and numerous other archive utilities.You are talking about files. Snapshots are for datasets. There is more information in a dataset than the collection of files. [perhaps Darren can chime in regarding encrypted datasets :-)]> The fact that ZFS doesn''t have any alternative built in doesn''t > mean that dumping a send to a file suddenly becomes a good idea.It is the only way to archive a dataset. By comparison, for UFS you could dd the file system. But since the pool can contain many different vdevs, and dd only works on one vdev, zfs send is the method to capture the dataset.> Especially when there is no step in those instructions to verify > that the archived file is actually a valid zfs send stream.There is little precedent for this, but I agree that verification should be done, if possible. Process can prevent future pain. Going back to the UFS case, if you dd a UFS file system, the only way to verify is to (effectively) dd again, which means that the file system must be quiesced -- an unpopular requirement because fssnap is ...uhhmm... limited. Similarly, ufsdump captures the logical view of the file system, but has no inherent verification capability. I''ll wager that few people verify UFS dd/ufsdump output (I do, but that is because I''ve got scars from using Exabyte drives :-(>> 2. You cannot build a zpool on a tape > > So you need a different solution for tape, possibly streaming the > files to tape, with the zpool configuration also documented and > backed up somehow if necessary.The information unique to a pool, besides its (mutable) physical layout, is quite small. Other than zpool status and zpool get all, is more needed? Is there a use case lurking here? ZFS represents a new way of managing data. zfs send is just one of the new tools in the kit, but it won''t replace an enterprise backup solution. zfs send is complementary to an enterprise backup solution. -- richard
Hi Richard, Actually, reading your reply has made me realise I was overlooking something when I talked about tar, star, etc... How do you backup a ZFS volume? That''s something traditional tools can''t do. Are snapshots the only way to create a backup or archive of those? Personally I''m quite happy with snapshots - we have a ZFS system at work that''s replicating all of it''s data to an offsite ZFS store using snapshots. Using ZFS as a backup store is something I''m quite happy with, it''s just storing just a snapshot file that makes me nervous. -- This message posted from opensolaris.org
On Fri, Sep 4, 2009 at 12:17 AM, Ross <myxiplx at googlemail.com> wrote:> Hi Richard, > > Actually, reading your reply has made me realise I was overlooking > something when I talked about tar, star, etc... How do you backup a ZFS > volume? That''s something traditional tools can''t do. Are snapshots the > only way to create a backup or archive of those? > > Personally I''m quite happy with snapshots - we have a ZFS system at work > that''s replicating all of it''s data to an offsite ZFS store using snapshots. > Using ZFS as a backup store is something I''m quite happy with, it''s just > storing just a snapshot file that makes me nervous. >The correct answer is ndmp. Whether Sun will ever add it to opensolaris is another subject entirely though. --Tim -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090904/cae8e970/attachment.html>
On Sep 3, 2009, at 10:32 PM, Tim Cook wrote:> On Fri, Sep 4, 2009 at 12:17 AM, Ross <myxiplx at googlemail.com> wrote: > Hi Richard, > > Actually, reading your reply has made me realise I was overlooking > something when I talked about tar, star, etc... How do you backup a > ZFS volume? That''s something traditional tools can''t do. Are > snapshots the only way to create a backup or archive of those?Below the application, dd would do it. But if you want incrementals, then either use the application''s backup scheme or zfs send.> Personally I''m quite happy with snapshots - we have a ZFS system at > work that''s replicating all of it''s data to an offsite ZFS store > using snapshots. Using ZFS as a backup store is something I''m quite > happy with, it''s just storing just a snapshot file that makes me > nervous. > > The correct answer is ndmp. Whether Sun will ever add it to > opensolaris is another subject entirely though.Available since b78 with source Integrated in b102. http://www.opensolaris.org/os/project/ndmp/ But NDMP is just part of an overall data management architecture... -- richard