rsync - Apr 2020 - [Bug 14341] New: --compare-source to filter file list at sender

https://bugzilla.samba.org/show_bug.cgi?id=14341

            Bug ID: 14341
           Summary: --compare-source to filter file list at sender
           Product: rsync
           Version: 3.1.3
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
          Assignee: wayne at opencoder.net
          Reporter: samba at haravikk.com
        QA Contact: rsync-qa at samba.org

This proposal is for a new option similar to --compare-dest, for specifying a
local directory structure on the sending side that can be used to more quickly
narrow down the list of files that have changed, especially useful when sending
from a file system containing multiple backups/snapshots that can be compared
locally, and useful when sending over a slower remote connection (by minimising
the file list).

The option would take the form:

    --compare-source {path}

Where {path} points to a parallel file structure that rsync should compare
against locally before sending files.


For example, I use rsync to synchronise my Mac's local Time Machine backups
over to a remote NAS. If my latest backup is 2020-04-08-152937 and I know that
I have previously synchronised the earlier backup 2020-04-08-142243 then I
might do the following:

    rsync -a --compare-source "/Volumes/Backup/Backups.backupdb/My
Mac/2020-04-08-142243" "/Volumes/Backup/Backups.backupdb/My
Mac/2020-04-08-152937" "user at MyNas.local:/volume1/Backups/My
Mac/Latest"

With the above command rsync would scan through files and directories in the
2020-04-08-152937, and compare them against the same path in 2020-04-08-142243
(if it exists), any file or directory found not to have changed between the two
by time and size, or by inode number (if on the same device) will be skipped
from the transfer without deleting, equivalent to excluding and protecting the
file/directory (to prevent deletion with --delete-excluded).


Currently to achieve this requires the use of a complex (and slow) find command
to compare inodes, passing in its results as a filter list of exclude and
protect statements; this works, but it isn't terribly efficient and
effectively
blocks rsync from transferring anything until the find command has finished
(i.e- the filter list is complete). If rsync were to perform the comparison
against a local file structure itself then it would a lot faster.


Of course this option should only be used when you are sure that the previous
directory you are comparing against has already been synchronised previously,
but when using rsync to create additional backups this can be done safely.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.