samba-bugs at samba.org
2018-May-08 12:14 UTC
[Bug 13423] New: Checksum option does not work as expected when append-verify is used
https://bugzilla.samba.org/show_bug.cgi?id=13423 Bug ID: 13423 Summary: Checksum option does not work as expected when append-verify is used Product: rsync Version: 3.1.3 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wayned at samba.org Reporter: dariuszb at me.com QA Contact: rsync-qa at samba.org Checksum option does not work as expected when append-verify is used. I can see that checksums are created for all files but they are not synced. rsync sample files to remote site: # rsync -azhcv --append-verify --progress --stats . rsyncclient at 192.168.142.2::DBTest/DBtest/ sending incremental file list ./ Firefox 60.0.dmg 55.37M 100% 8.40MB/s 0:00:06 (xfr#1, to-chk=3/6) Skype-8.19.0.1.dmg 92.68M 100% 8.35MB/s 0:00:10 (xfr#2, to-chk=2/6) SuperDuper!-2.dmg 6.09M 100% 5.16MB/s 0:00:01 (xfr#3, to-chk=1/6) SuperDuper!.dmg 6.09M 100% 7.20MB/s 0:00:00 (xfr#4, to-chk=0/6) Number of files: 6 (reg: 5, dir: 1) Number of created files: 4 (reg: 4) Number of deleted files: 0 Number of regular files transferred: 4 Total file size: 160.23M bytes Total transferred file size: 160.23M bytes Literal data: 160.23M bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.388 seconds File list transfer time: 0.000 seconds Total bytes sent: 159.83M Total bytes received: 95 sent 159.83M bytes received 95 bytes 8.20M bytes/sec total size is 160.23M speedup is 1.00 Now i use hexcurse and change one of the source files - just flip some bytes. As -c option is present I would expect that rsync detects that one file has changed and updates it accordingly. I run rsync again - no files are synced - I can confirm it by manually checking files' hashes. # rsync -azhcv --append-verify --progress --stats . rsyncclient at 192.168.142.2::DBTest/DBtest/ sending incremental file list Number of files: 6 (reg: 5, dir: 1) Number of created files: 0 Number of deleted files: 0 Number of regular files transferred: 0 Total file size: 160.23M bytes Total transferred file size: 0 bytes Literal data: 0 bytes Matched data: 0 bytes File list size: 0 File list generation time: 0.402 seconds File list transfer time: 0.000 seconds Total bytes sent: 272 Total bytes received: 12 sent 272 bytes received 12 bytes 81.14 bytes/sec total size is 160.23M speedup is 564,202.91 I remove --append-verify option and run rsync again # rsync -azhcv --progress --stats . rsyncclient at 192.168.142.2::DBTest/DBtest/ sending incremental file list Firefox 60.0.dmg 55.37M 100% 101.92MB/s 0:00:00 (xfr#1, to-chk=3/6) Number of files: 6 (reg: 5, dir: 1) Number of created files: 0 Number of deleted files: 0 Number of regular files transferred: 1 Total file size: 160.23M bytes Total transferred file size: 55.37M bytes Literal data: 7.44K bytes Matched data: 55.36M bytes File list size: 0 File list generation time: 0.390 seconds File list transfer time: 0.000 seconds Total bytes sent: 7.44K Total bytes received: 52.14K sent 7.44K bytes received 52.14K bytes 7.94K bytes/sec total size is 160.23M speedup is 2,689.70 This time files were synced as expected -- You are receiving this mail because: You are the QA Contact for the bug.
samba-bugs at samba.org
2018-May-08 12:29 UTC
[Bug 13423] Checksum option does not work as expected when append-verify is used
https://bugzilla.samba.org/show_bug.cgi?id=13423 --- Comment #1 from Kevin Korb <rsync at sanitarium.net> --- If the file has not grown then there is nothing for --append[-verify] to do. Also, without --itemize-changes you have no reporting from --checksum. You probably shouldn't have either option. -- You are receiving this mail because: You are the QA Contact for the bug.
samba-bugs at samba.org
2018-May-08 12:32 UTC
[Bug 13423] Checksum option does not work as expected when append-verify is used
https://bugzilla.samba.org/show_bug.cgi?id=13423 --- Comment #2 from dariuszb at me.com --- Yes it is clear that --append-verify will not update the same size files. But --checksum should check hashes of all files and trigger update if different. What is happening here looks like append-verify overwrites checksum functionality. -- You are receiving this mail because: You are the QA Contact for the bug.
samba-bugs at samba.org
2018-May-08 12:52 UTC
[Bug 13423] Checksum option does not work as expected when append-verify is used
https://bugzilla.samba.org/show_bug.cgi?id=13423 --- Comment #3 from Kevin Korb <rsync at sanitarium.net> --->From man rsync --append: > If a file needs to be transferred and its size on the receiver is the same or longer than the size on the sender, the file is skipped.--append-verify does the verify AFTER an append happens so if --append won't copy the file neither will --append-verify. Since --append essentially means only care if the source file is bigger the file is ignored. The combination is why I said you probably don't want either option and especially not both options. With both, all files on both ends are checksummed first (even files that are only on one side!) Then files that are bigger on the source are appended to. Then the files are verified. Then if the verification fails rsync throws a warning and redoes the whole file --inplace style. Normally I recommend against --checksum but I have no Mac experience and I don't really know how your disk image files work. In many cases it is actually faster to use --ignore-times (delta-xfer everything regardless of timestamp) and maybe --inplace instead of --checksum. Usually --checksum is used when you can't afford to write out a new file because of limited/slow writes or because you are retaining the old version and don't want a new version unless it is actually new. -- You are receiving this mail because: You are the QA Contact for the bug.
samba-bugs at samba.org
2018-May-08 13:01 UTC
[Bug 13423] Checksum option does not work as expected when append-verify is used
https://bugzilla.samba.org/show_bug.cgi?id=13423 --- Comment #4 from dariuszb at me.com --- Thank you for clarification. -- You are receiving this mail because: You are the QA Contact for the bug.