Hello List, I still have not yet backed up my MDT onto a larger LV. (My users like availability times in the monthly range; fortunately Lustre is good enough to provide that to them.) To execute the getfattr and tar used to take me 3 to 8 hours. My last attempt had tar -cf running for over two days which was not acceptable to my users. I would really like to get this work done and to provide my users with a realistic estimate of the time Lustre disks will be unavailable to them. The MDT is: /dev/sdf 58G Total Size 3.4G Used 51G Avail 7% /srv/lustre/mds/crew8-MDT0000. The current inode usage is (from df -ih): /dev/sdf 17M 14M 2.5M 86% /srv/lustre/mds/crew8-MDT0000 What would be a reasonable time estimate to back this up? Is getfattr and tar still preferred over rsync -aS or the getfattr and dump method? Thank you, megan
On Wed, 2009-04-29 at 10:17 -0400, Ms. Megan Larko wrote:> Hello List,Hi,> > I still have not yet backed up my MDT onto a larger LV. (My users > like availability times in the monthly range; fortunately Lustre is > good enough to provide that to them.)That''s good to hear.> To execute the getfattr and > tar used to take me 3 to 8 hours. My last attempt had tar -cf > running for over two days which was not acceptable to my users.Has your dataset grown relative to those two different times? Or has the process gotten less efficient somehow?> What would be a reasonable time estimate to back this up?Of course, this is completely subjective on the hardware involved.> Is > getfattr and tar still preferred over rsync -aSWell, I think the getfattr/tar method are time-tested. I can''t imagine why an rsync (or several of them, more on that below) that includes extended attributes (-X IIRC) in the sync wouldn''t work either. I don''t think we''ve done any testing of the rsync method however.> or the getfattr and > dump method?I''m not so sure you want to use dump. dump effectively takes an image of the filesystem and restore that image exactly. tar and rsync are more like taking the content out of the filesystem and recreating it. As for time to back up/restore. When I need to make a backup or move a filesystem (generally speaking here, not specific to Lustre) I do it in several passes using rsync. I do one full pass while the filesystem is still live and being used -- no downtime. I then do another on the live filesystem -- no downtime -- just to get an idea of the time/data delta and then I shut the filesystem down and do a final rsync. The down time is then only a function of the size of dataset that has changed since my last "live" sync. If I don''t know when my downtime is scheduled for, I continue doing periodic live runs just to keep the source and target as closely synced as possible until I do schedule the downtime. How well this maps to your MDT is for you to decide. :-) b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090429/5809b187/attachment-0001.bin
On Apr 29, 2009 10:36 -0400, Brian J. Murrell wrote:> On Wed, 2009-04-29 at 10:17 -0400, Ms. Megan Larko wrote: > > To execute the getfattr and > > tar used to take me 3 to 8 hours. My last attempt had tar -cf > > running for over two days which was not acceptable to my users. > > Has your dataset grown relative to those two different times? Or has > the process gotten less efficient somehow?The problem with 1.6.7.1 is that the MDS now stores the size of the file from the last time it is closed. This was to be used for some backup tools to generate a reasonable estimate of the amount of data to be backed up, but has caused other issues.> > Is getfattr and tar still preferred over rsync -aS > > Well, I think the getfattr/tar method are time-tested. I can''t imagine > why an rsync (or several of them, more on that below) that includes > extended attributes (-X IIRC) in the sync wouldn''t work either. I don''t > think we''ve done any testing of the rsync method however.The problem is that tar is trying to back up the "file data" from the MDS inodes, but they are all sparse files with no data blocks.> > or the getfattr and dump method?I did a quick test with "dump" and it doesn''t seem to try and read all of the "sparse" file data, so you could give this a try. The minimum version of dump that claims extended attribute support is 0.4b40, though I haven''t tested that. In the absence of dump xattr support you can still use the getfattr + dump + restore + setfattr method described in the manual. Always verify that "getfattr -d -m ".*" on the restored MDS files returns a "trusted.lov" attribute.> I''m not so sure you want to use dump. dump effectively takes an image > of the filesystem and restore that image exactly. tar and rsync are > more like taking the content out of the filesystem and recreating it.That isn''t quite true - the dump process does use device-level access to create the backup, but the restore process is more like "tar" in that it restores files by their pathname. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
On Wed, 2009-04-29 at 14:38 -0600, Andreas Dilger wrote:> > That isn''t quite true - the dump process does use device-level access > to create the backup, but the restore process is more like "tar" in > that it restores files by their pathname.I have to confess to never really using dump just due to not liking the way it operated and the last time I looked at a dump utility probably predates ext{2,3,4} or Linux even, but my understanding was that dump took the whole filesystem with it. That is, you didn''t even need to mkfs to restore. You could restore to a block device and that the restore would lay down the filesystem as it was when it was dumped. Maybe my memory is just showing it''s age. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090429/90e867cd/attachment.bin