Brian J. Murrell
2009-Oct-24 03:38 UTC
[Lustre-discuss] rsync --sparse actually work on RHEL5?
I have been experimenting with MDT backup/restore (aka: transplant).
Given that I want to transplant to a new device on the same MDS, I can
use rsync so that the EAs are transplanted in tandem with the files
rather than as an after-thought with getfattr.
Tar requires the more clunky getfattr afterwards because this is RHEL5
and RHEL5''s tar doesn''t appear to have EA support in it.
But, I seem to be finding that rsync''s --sparse option is not quite as
good as tar''s. Witness:
# pwd
/tmp/mdt1_snapshot/ROOT
# rsync --sparse brian/lustre/OFED-1.3.tgz /tmp/mdt1_new/ROOT/brian/lustre/
# ls -ls /tmp/mdt1_{snapshot,new}/ROOT/brian/lustre/OFED-1.3.tgz
12 -rw-r--r-- 1 root root 59918537 Oct 23 23:34
/tmp/mdt1_new/ROOT/brian/lustre/OFED-1.3.tgz
0 -rw-r--r-- 1 1001 1001 59918537 Mar 27 2008
/tmp/mdt1_snapshot/ROOT/brian/lustre/OFED-1.3.tgz
As you can see, rsync allocated actual disk blocks for the copy of the
file where it''s clearly sparse on the source. Now tar doing the same
operation:
# rm /tmp/mdt1_new/ROOT/brian/lustre/OFED-1.3.tgz
# tar -cSvf - brian/lustre/OFED-1.3.tgz | tar -xSvf - -C /tmp/mdt1_new/ROOT/
brian/lustre/OFED-1.3.tgz
brian/lustre/OFED-1.3.tgz
# ls -ls /tmp/mdt1_{snapshot,new}/ROOT/brian/lustre/OFED-1.3.tgz
0 -rw-r--r-- 1 1001 1001 59918537 Mar 27 2008
/tmp/mdt1_new/ROOT/brian/lustre/OFED-1.3.tgz
0 -rw-r--r-- 1 1001 1001 59918537 Mar 27 2008
/tmp/mdt1_snapshot/ROOT/brian/lustre/OFED-1.3.tgz
Looks much better. But I''d rather use rsync. Is this rsync --sparse
issue on RHEL5 known to anyone else?
Is this just some kind of PEBKAC I''m experiencing?
Thanx,
b.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091023/7685d807/attachment.bin
Andreas Dilger
2009-Oct-25 22:38 UTC
[Lustre-discuss] rsync --sparse actually work on RHEL5?
On 2009-10-23, at 21:38, Brian J. Murrell wrote:> I have been experimenting with MDT backup/restore (aka: transplant). > > Given that I want to transplant to a new device on the same MDS, I can > use rsync so that the EAs are transplanted in tandem with the files > rather than as an after-thought with getfattr. > > Tar requires the more clunky getfattr afterwards because this is RHEL5 > and RHEL5''s tar doesn''t appear to have EA support in it.We actually have a patched tar from FC11(?) tar-1.19 that has --xattr support in it. Available at the download site. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Peter Grandi
2009-Oct-25 22:46 UTC
[Lustre-discuss] rsync --sparse actually work on RHEL5?
> Given that I want to transplant to a new device on the same > MDS, I can use rsync so that the EAs are transplanted in > tandem with the files rather than as an after-thought with > getfattr.> Tar requires the more clunky getfattr afterwards because this > is RHEL5 and RHEL5''s tar doesn''t appear to have EA support in > it.Schily TAR (''star'') has extended attr (and ACL support) too if ''exustar'' format is used, and seems available for RHEL5 (not by coincidence I guess).> But, I seem to be finding that rsync''s --sparse option is not > quite as good as tar''s. Witness:> pwd > /tmp/mdt1_snapshot/ROOT > rsync --sparse brian/lustre/OFED-1.3.tgz /tmp/mdt1_new/ROOT/brian/lustre/ > ls -ls /tmp/mdt1_{snapshot,new}/ROOT/brian/lustre/OFED-1.3.tgz > 12 -rw-r--r-- 1 root root 59918537 Oct 23 23:34 /tmp/mdt1_new/ROOT/brian/lustre/OFED-1.3.tgz > 0 -rw-r--r-- 1 1001 1001 59918537 Mar 27 2008 /tmp/mdt1_snapshot/ROOT/brian/lustre/OFED-1.3.tgz> As you can see, rsync allocated actual disk blocks for the > copy of the file where it''s clearly sparse on the source.That ''12'' looks like 3 blocks, that is 2 metadata (indirect?) blocks and 1 data block, therefore probably the last block. I suspect that RSYNC refuses to write wholly empty files.> Now tar doing the same operation:> rm /tmp/mdt1_new/ROOT/brian/lustre/OFED-1.3.tgz > tar -cSvf - brian/lustre/OFED-1.3.tgz | tar -xSvf - -C /tmp/mdt1_new/ROOT/ > brian/lustre/OFED-1.3.tgz > brian/lustre/OFED-1.3.tgz > ls -ls /tmp/mdt1_{snapshot,new}/ROOT/brian/lustre/OFED-1.3.tgz > 0 -rw-r--r-- 1 1001 1001 59918537 Mar 27 2008 /tmp/mdt1_new/ROOT/brian/lustre/OFED-1.3.tgz > 0 -rw-r--r-- 1 1001 1001 59918537 Mar 27 2008 /tmp/mdt1_snapshot/ROOT/brian/lustre/OFED-1.3.tgz> Looks much better. But I''d rather use rsync. Is this rsync > --sparse issue on RHEL5 known to anyone else?My suspicion was that for some reason RSYNC writes at least one byte to the file, at the end, and ''strace'' confirms (on a JFS filesystem with 4KiB minimum extents): $ rm -f /var/tmp/empty /var/tmp/empty2 $ dd bs=1 count=0 if=/dev/zero seek=1048576 of=/var/tmp/empty 0+0 records in 0+0 records out 0 bytes (0 B) copied, 1.2341e-05 seconds, 0.0 kB/s $ strace -f -e trace=file,write,_llseek rsync --no-whole-file -S /var/tmp/empty /var/tmp/empty2 .... [pid 15619] _llseek(1, 1024, [1048576], SEEK_CUR) = 0 [pid 15619] _llseek(1, 1, [1048577], SEEK_CUR) = 0 [pid 15619] _llseek(1, -1, [1048576], SEEK_CUR) = 0 [pid 15619] write(1, "\0", 1) = 1 [pid 15619] lstat64(".empty2.bYDneE", {st_mode=S_IFREG|0600, st_size=1048577, ...}) = 0 [pid 15619] rename(".empty2.bYDneE", "empty2") = 0 .... $ ls -s /var/tmp/empty{,2} 0 /var/tmp/empty 4 /var/tmp/empty2 Note that in the above RSYNC explicitly seeks back to the next-to-last byte ad writes it; the most likely reason is portability, that is probably RSYNC does not assume that the target system has got ''ftruncate'' (while the ''dd'' above uses it). In Perl the same effect can be obtained with perl -e ''truncate(STDOUT,1048576);'' >| /var/tmp/empty There are a couple of other issues with RSYNC and sparse files: http://lists.samba.org/archive/rsync/2003-August/007000.html http://arighi.blogspot.com/2008/03/rsync-and-sparse-files.html http://lists.samba.org/archive/rsync/2008-February/020112.html I regard these as largely unavoidable given its extreme requirements for cross platform portability and the way it operates. But it would be nice to add to it checking whether ''ftruncate'' is available. Recent versions of RSYNC indeed seem to imply it into the new "--preallocate" option. BTW, this is unrelated, these are the options I recommend for backups: rsync -axHAX [-u] --delete [--sparse|--inplace] [--partial] [-z] [-i]
Brian J. Murrell
2009-Oct-26 00:24 UTC
[Lustre-discuss] rsync --sparse actually work on RHEL5?
On Fri, 2009-10-23 at 23:38 -0400, Brian J. Murrell wrote:> > But, I seem to be finding that rsync''s --sparse option is not quite as > good as tar''s.FWIW, this does not reproduce on Ubuntu Jaunty, with rsync 3.0.5. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091025/418dde51/attachment.bin