rsync - Jan 2010 - The patch write-devices.diff (rw-devices.diff) fix

Hello,
I faced the problem of synchronization block devices and found the patch
write-devices.diff by Darryl Dixon, that went trough this mailing list
http://lists.samba.org/archive/rsync/2009-October/024037.html

Unfortunately the patch was one flaw for me. It doesn't improve
performance of synchronization. When I tried to test it using something
like:

./rsync --rsync-path\="$PWD/rsync" -v --bwlimit=100 --progress
--copy-devices --write-devices /dev/mapper/vg-pokus1 root at
localhost:/dev/mapper/vg-pokus2

where pokus1 & pokus2 are two identically initialized block devices
(logical volumes) 32MiB in size. All the data was passed trough
local connection without the benefit. (I set --bwlimit low, so I can see
evidently the rsync performance).

By adding more -v I found, that some stat() call in the generator.c
reports destination device size is zero. I added some code handling
block device size. Option --ignore-times is implied for --rw-devices.
A new patch rw-devices.diff is attached. It is combination of
copy-devices.diff, write-devices.diff and some new code from me.


bobek:/data/soft/rsync/rsync# ./rsync --rsync-path\="$PWD/rsync" -v
--bwlimit=100 --progress --rw-devices root at localhost:/dev/mapper/vg-pokus2
/dev/mapper/vg-pokus1
vg-pokus2
     33,554,432 100%   48.78MB/s    0:00:00 (xfr#1, to-chk=0/1)

sent 40,601 bytes  received 23,264 bytes  18,247.14 bytes/sec
total size is 33,554,432  speedup is 525.40
bobek:/data/soft/rsync/rsync# dd if=/dev/urandom bs=1M seek=16 count=1
of=/dev/mapper/vg-pokus2
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.198132 s, 5.3 MB/s
bobek:/data/soft/rsync/rsync# ./rsync --rsync-path\="$PWD/rsync" -v
--bwlimit=100 --progress --rw-devices root at localhost:/dev/mapper/vg-pokus2
/dev/mapper/vg-pokus1
vg-pokus2
     33,554,432 100%    2.95MB/s    0:00:10 (xfr#1, to-chk=0/1)

sent 40,601 bytes  received 1,076,931 bytes  82,780.15 bytes/sec
total size is 33,554,432  speedup is 30.03

Now I have questions:
    1. Is it worth to have two separate patches copy-devices.diff,
       write-devices.diff? (It is hard to apply them one after other,
       because they overlays.)
    2. Should be there two options `--copy-devices', `--write-devices'?
       I thing one option e.g. `--rw-devices' should be sufficient.
    3. I think the above option should collide with many other options
       like `--archive', `--recursive',...

Any suggestions?
Regards
-- 
Zito
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rw-devices.diff
Type: text/x-diff
Size: 12465 bytes
Desc: not available
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20100111/073ac98b/attachment.diff>

Hi Zito,
> Hello,
> I faced the problem of synchronization block devices and found the patch
> write-devices.diff by Darryl Dixon, that went trough this mailing list
> http://lists.samba.org/archive/rsync/2009-October/024037.html
>
> Unfortunately the patch was one flaw for me. It doesn't improve
> performance of synchronization. When I tried to test it using something
> like:
Yes. I figured it would be something to do with the block device size
calculation, as my initial patch simply punted on that (0 as you saw) - I
didn't have time/motivation to dig through and research the necessary
magic to get good data out. Well done for finding and implementing the
required code :)
>
> ./rsync --rsync-path\="$PWD/rsync" -v --bwlimit=100 --progress
> --copy-devices --write-devices /dev/mapper/vg-pokus1
> root at localhost:/dev/mapper/vg-pokus2
>
> where pokus1 & pokus2 are two identically initialized block devices
> (logical volumes) 32MiB in size. All the data was passed trough
> local connection without the benefit. (I set --bwlimit low, so I can see
> evidently the rsync performance).
>
> By adding more -v I found, that some stat() call in the generator.c
> reports destination device size is zero. I added some code handling
> block device size. Option --ignore-times is implied for --rw-devices.
> A new patch rw-devices.diff is attached. It is combination of
> copy-devices.diff, write-devices.diff and some new code from me.
Excellent stuff. This is what I love about open source software - one
person can get the ball rolling and then the effort and expertise of
others combines to improve matters for everyone.
> Now I have questions:
>     1. Is it worth to have two separate patches copy-devices.diff,
>        write-devices.diff? (It is hard to apply them one after other,
>        because they overlays.)
>     2. Should be there two options `--copy-devices',
`--write-devices'?
>        I thing one option e.g. `--rw-devices' should be sufficient.
>     3. I think the above option should collide with many other options
>        like `--archive', `--recursive',...
I agree maintaining the separate patches are hard. Personally, I'd prefer
to see just one merged patch as you suggest.

regards,
Darryl Dixon
Winterhouse Consulting Ltd
http://www.winterhouseconsulting.com