Other than DRBD and Hot standby are there any other alternatives? We want to have a redundant copy of our data and was wondering if rsync is the only way to accomplish this.
On Mon, 2009-07-20 at 23:41 -0400, Mag Gam wrote:> Other than DRBD and Hot standby are there any other alternatives? We > want to have a redundant copy of our data and was wondering if rsync > is the only way to accomplish this.Until the replication feature is available, rsync (or a suitable replacement) is about the only way to have an independent, redundant copy of your data. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090720/96b0b77e/attachment.bin
On Jul 20, 2009 23:45 -0400, Brian J. Murrell wrote:> On Mon, 2009-07-20 at 23:41 -0400, Mag Gam wrote: > > Other than DRBD and Hot standby are there any other alternatives? We > > want to have a redundant copy of our data and was wondering if rsync > > is the only way to accomplish this. > > Until the replication feature is available, rsync (or a suitable > replacement) is about the only way to have an independent, redundant > copy of your data.In the 2.0 release there will be an efficient mechanism (Changelogs) along with an rsync replacement to incrementally sync a Lustre filesystem to another filesystem. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
----- "Andreas Dilger" <adilger at sun.com> wrote:> > Until the replication feature is available, rsync (or a suitable > > replacement) is about the only way to have an independent, redundant > > copy of your data. > > In the 2.0 release there will be an efficient mechanism (Changelogs) > along with an rsync replacement to incrementally sync a Lustre > filesystem to another filesystem.We are currently using e2scan and distributed rsyncs across the compute farm to do the same thing with Lustre v1.6/1.8 to mirror our main filesystem to a backup every night. Daire
On Tue, Jul 21, 2009 at 4:02 AM, Daire Byrne<Daire.Byrne at framestore.com> wrote:> > ----- "Andreas Dilger" <adilger at sun.com> wrote: >> > Until the replication feature is available, rsync (or a suitable >> > replacement) is about the only way to have an independent, redundant >> > copy of your data. >> >> In the 2.0 release there will be an efficient mechanism (Changelogs) >> along with an rsync replacement to incrementally sync a Lustre >> filesystem to another filesystem. > > We are currently using e2scan and distributed rsyncs across the compute farm to do the same thing with Lustre v1.6/1.8 to mirror our main filesystem to a backup every night.Is this a script you''d be willing to share?
Michael, ----- "Michael Di Domenico" <mdidomenico4 at gmail.com> wrote:> > We are currently using e2scan and distributed rsyncs across the > compute farm to do the same thing with Lustre v1.6/1.8 to mirror our > main filesystem to a backup every night. > > Is this a script you''d be willing to share?Sure, but it is fairly specific to our setup and (to be honest) is pretty hacktastic! The basic idea is that an e2scan is run on the production filesystem every night and new files (last 72 hours) are copied over (incremental). Then once a week we include "deletes" which then compares the e2scan file list from both the production and backup filesystems to work out what needs to be deleted from the backup to synchronise the two filesystems. We never really ever have to do a "full" backup. We also use LVM snapshots on the backup to give us a couple of weeks of retention. It works for us but your mileage may vary. Daire -------------- next part -------------- A non-text attachment was scrubbed... Name: flbackup Type: application/octet-stream Size: 7985 bytes Desc: not available Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090721/5147c980/attachment-0001.obj
Can''t wait until 2.0 and SNS. I think thats the only die-hard feature Lustre is really missing. On Tue, Jul 21, 2009 at 10:19 AM, <Daire.Byrne at framestore.com> wrote:> Michael, > > ----- "Michael Di Domenico" <mdidomenico4 at gmail.com> wrote: >> > We are currently using e2scan and distributed rsyncs across the >> compute farm to do the same thing with Lustre v1.6/1.8 to mirror our >> main filesystem to a backup every night. >> >> Is this a script you''d be willing to share? > > Sure, but it is fairly specific to our setup and (to be honest) is pretty hacktastic! The basic idea is that an e2scan is run on the production filesystem every night and new files (last 72 hours) are copied over (incremental). Then once a week we include "deletes" which then compares the e2scan file list from both the production and backup filesystems to work out what needs to be deleted from the backup to synchronise the two filesystems. We never really ever have to do a "full" backup. We also use LVM snapshots on the backup to give us a couple of weeks of retention. > > It works for us but your mileage may vary. > > Daire > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >