rsync - Jul 2008 - DO NOT REPLY [Bug 5644] New: Option to recheck basis dirs for existing dest files

https://bugzilla.samba.org/show_bug.cgi?id=5644

           Summary: Option to recheck basis dirs for existing dest files
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P3
         Component: core
        AssignedTo: wayned@samba.org
        ReportedBy: matt@mattmccutchen.net
         QAContact: rsync-qa@samba.org


Currently, rsync does not check basis dirs for existing destination files.  As
the man page description of --link-dest puts it, existing destination files are
"definitive".  I would like an option, tenatively named
--recheck-basis-dirs,
to make them non-definitive.  Specifically, with this option:

- An existing destination file that matches a --compare-dest file would be
deleted regardless of --delete.  (Deletion without --delete might seem
surprising, but the goal is to leave the destination in the correct state with
respect to the basis dir for the paths that are being processed on the current
run.  Think of it like --force.)

- An existing destination file that matches a --link-dest file would be
replaced with a hard link to the --link-dest file.

(The analogue for --copy-dest would be to replace a destination file matched in
a --copy-dest dir with a copy of the --copy-dest file, but I don't see any
point in doing that.)

I'm thinking that both cases should be itemized as if the destination file
never existed, but I'm not sure of this.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5644





------- Comment #1 from matt@mattmccutchen.net  2008-07-29 13:28 CST -------
Inspired by this thread:
http://lists.samba.org/archive/rsync/2008-July/021400.html


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5644


dwdrsync@drdykstra.us changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dwdrsync@drdykstra.us




------- Comment #2 from dwdrsync@drdykstra.us  2008-12-23 15:15 CST -------
I think this is a good idea, although I don't really like the option name. 
I
think it should have "dest" in the name to relate it to the dest
options.  The
term "basis file" is used in the man page but not in any option names,
and the
term "basis dir" doesn't appear in the man page at all.  Perhaps
"--override-dest-files".


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5644





------- Comment #3 from matt@mattmccutchen.net  2008-12-23 15:44 CST -------
Note: The term "basis dir" is widely used in the source and on the
list; its
omission from the man page should be considered a fault of the man page.

The word "override" makes sense with --compare-dest but is
inappropriately
scary when one is using --link-dest to reduce transfer time and/or disk usage
without affecting the contents ultimately found in the destination.  I think
this is a sign that my approach in these three enhancement requests, of
generalizing the existing basis dir semantics along several different axes
until saner semantics are available, will be messier than I realized.  Instead,
perhaps we should make a list of motivating use cases, figure out what the new
design should be, and finally figure out how to retain backward compatibility.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5644





------- Comment #4 from samba at bryanthansen.net  2009-09-29 14:42 CST -------
Created an attachment (id=4759)
 --> (https://bugzilla.samba.org/attachment.cgi?id=4759&action=view)
replace DEST files with basis files when basis files match the src exactly


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5644


samba at bryanthansen.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |samba at bryanthansen.net




------- Comment #5 from samba at bryanthansen.net  2009-09-29 14:42 CST -------
Here is a proposed patch for rsync 3.0.6 that implements this functionality.

There are no extra options, as proposed in some of the previous posts.  If a
"basis file" passes the unchanged_file() *and* the unchanged_attrs()
tests, the
DEST file will be removed and replaced with a hard link to the basis file.

Note: unchanged_file() is the internal rsync test for checking file sizes,
times, and/or checksums when comparing files.  unchanged_attrs() checks
ownership, permissions, and ACL's.

Now the tricky part: in the case that a basis file passes the unchanged_file()
test, but not the unchanged_attrs() test, it's tempting to replace DEST with
a
hardlink *if* the DEST fails the unchanged_file() test.  However, the danger is
that attributes & ACL's from the basis file would eventually be thrown
away in
later rsync operations, since these basis file attributes are now coupled with
the DEST.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5644

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #6 from Wayne Davison <wayned at samba.org> 2013-01-18
19:59:54 UTC ---
I have implemented this as the default behavior for 3.1.0.  I think that this
doesn't need to be an option since most uses of the --ALT-dest options are
copies into a new directory hierarchy.  Those that are not will be better
served by the new behaviors.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=5644

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |brian at interlinx.bc.ca

--- Comment #7 from Wayne Davison <wayned at samba.org> 2013-01-18
20:01:04 UTC ---
*** Bug 8712 has been marked as a duplicate of this bug. ***

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.