rsync - Apr 2012 - [Bug 8847] New: detect-renamed.diff update to ensure existence of directory for partial-dir

https://bugzilla.samba.org/show_bug.cgi?id=8847

           Summary: detect-renamed.diff update to ensure existence of
                    directory for partial-dir
           Product: rsync
           Version: 3.0.9
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: rsnc at opal.com
         QAContact: rsync-qa at samba.org


Created attachment 7435
  --> https://bugzilla.samba.org/attachment.cgi?id=7435
replacement for patches/detect-renamed.diff

When using --detect-renamed, rsync creates "partial-dir"s for use as a
place to
hold files being deleted as part of a rename.  These held-files are then used
as the basis files for the files being renamed.  If the file is indeed being
renamed, this use provides significant bandwidth savings in that the file does
not need to be re-uploaded.

By default, the partial-dir is a directory called ".~tmp~" that will
be created
in the directory of the renamed file.

As described by an earlier developer of detect-renamed.diff <a
href="http://lists.samba.org/archive/rsync/2007-December/019469.html">here</a>,
there exist circumstances when the partial-dir cannot be created because the
directory for it does not yet exist.  This is the case when a directory is
renamed or a file is moved to a new directory.  In such cases, which are, in
fact, quite common, detect-rename fails because the new directory does not
exist when rsync tries to create the partial dir.  So, the file(s) in question
are re-uploaded instead of being detected as renamed existing files.

The attached update to detect-renamed.diff addresses this problem by ensuring
that the needed directories for partial-dir are created.  This is done by use
of the equivalent of "mkdir -p" when creating the partial-dir.  A new
function,
do_mkdir_path(), is provided in syscall.c that recursively creates any needed
directories.

With this change, detect-renamed properly detects files moved to new
directories as well as directory renames.

The attached detect-renamed.diff replaces the current file.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8847

--- Comment #1 from Russell Black <black.russell at gmail.com> 2013-02-01
01:35:37 UTC ---
+1 for this patch.  It looks like a simple change that will result in a huge
performance improvement in certain common situations.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8847

--- Comment #2 from Russell Black <black.russell at gmail.com> 2013-02-02
22:59:17 UTC ---
If it helps to evaluate the patch, here is the difference in the code between
this one and the regular detect-renamed patch.  Seems pretty straightforward.

$ diff rsync-3.0.9\ 2 rsync-3.0.9\ 3
diff rsync-3.0.9 2/proto.h rsync-3.0.9 3/proto.h
307a308
> int do_mkdir_path(char *fname, mode_t mode);
diff rsync-3.0.9 2/syscall.c rsync-3.0.9 3/syscall.c
219a220,235
> int do_mkdir_path(char *fname, mode_t mode)
> {
> 	char fnametmp[MAXPATHLEN], *fnametmpptr;
> 	STRUCT_STAT st;
> 
> 	if (fname) {
> 		strcpy(fnametmp, fname);
> 		if ((fnametmpptr = strrchr(fnametmp, '/')) != NULL) {
> 			*fnametmpptr = '\0';
> 			if (do_stat(fnametmp, &st) < 0)
> 				do_mkdir_path(fnametmp, mode);
> 		}
> 	}
> 	return do_mkdir(fname, mode);
> }
> 
diff rsync-3.0.9 2/util.c rsync-3.0.9 3/util.c
1171c1171
<         if (statret < 0 && do_mkdir(dir, 0700) < 0) {
---
> 		if (statret < 0 && do_mkdir_path(dir, 0700) < 0) {
-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8847

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #7435|0                           |1
           is patch|                            |
   Attachment #7435|application/octet-stream    |text/plain
          mime type|                            |

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8847

--- Comment #3 from bjquinn at seidal.com 2014-08-03 02:54:59 UTC ---
I'm going to do some testing, but it appears that this fix is probably in
3.1.1.

In the process of applying the modifications suggested here (basically, add the
do_mkdir_path function and call it in place of do_mkdir in
util.c/handle_partial_dir), I noticed that it appears that
util.c/handle_partial_dir now calls make_path, which seems to be a wrapper
around do_mkdir that intends to create the entire path, like do_mkdir_path did.

I'll do some testing to follow up, but I'm guessing that this bug can
likely be
closed.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8847

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #4 from Wayne Davison <wayned at samba.org> ---
This got fixed in the patches for 3.1.x back in June of 2013.  It now uses the
make_path() function.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.