Hi all; I need to offer a solution for copying 82 TB data from a Lustre 1.4.6 system to a new Lustre 1.6.x system. Copy process will work daily and first copy will be full and incremantal copies will follow. It will be quite nice if we can offer functionality that backup software have, like being able to restore a file from a previous date not from the last copy. Best regards <http://www.sun.com/> http://www.sun.com/emrkt/sigs/6g_top.gif Mertol Ozyoney Storage Practice - Sales Manager Sun Microsystems, TR Istanbul TR Phone +902123352200 Mobile +905339310752 Fax +902123352222 Email <mailto:Ayca.Yalcin at Sun.COM> mertol.ozyoney at Sun.COM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071107/f42406d4/attachment-0002.html -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.gif Type: image/gif Size: 1257 bytes Desc: not available Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20071107/f42406d4/attachment-0002.gif
On Nov 07, 2007 18:19 +0200, Mertol Ozyoney wrote:> I need to offer a solution for copying 82 TB data from a Lustre 1.4.6 system > to a new Lustre 1.6.x system. Copy process will work daily and first copy > will be full and incremantal copies will follow.Using e2scan will allow you to efficiently generate lists of files that have changed since the last rsync. The e2scan tool is included in e2fsprogs at ftp://ftp.lustre.org/pub/lustre/other/e2fsprogs/testing/ but it is still undergoing some development.> It will be quite nice if we can offer functionality that backup software > have, like being able to restore a file from a previous date not from the > last copy.This will not be easily done until Lustre is moved over to using ZFS with snapshots. Cheers, Andreas -- Andreas Dilger Sr. Software Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Mertol,> Using e2scan will allow you to efficiently generate lists of files > that have changed since the last rsync. The e2scan tool is included in > e2fsprogs at ftp://ftp.lustre.org/pub/lustre/other/e2fsprogs/testing/ but it is > still undergoing some development.This is why we had to go with the "rsync --dry-run --itemise-changes" method. It requires a lot of scripting, LVM snapshots and works well with a gridengine installation. Doing multiple rsyncs of the network filesystem may be easier for you but the counting and comparing of files is going to be very slow and I doubt you will be able trawl an 80TB filesystem within 24 hours. Daire