rsync - Dec 2007 - Odd behavior with --detect-renamed

Hello,

I'm totally new to this list, so I hope I don't break all the rules. :)
I've looked through the archives (and google), and I really can't find
the
answer to my question.

I'm trying out the 'detect-renamed'-patch, and I've encountered
some odd
behavior. I've applied the patch to both rsync-2.6.9 and rsync-3.0.0pre7,
and it's the same behavior.

Basicly, what I've noticed is that if I move a file into a newly created
directory (which is what happens if I rename a directory, for example), the
file isn't detected as renamed. However, if I manually create the new
directory on the target-side, the file is correctly detected as a rename,
and isn't transfered over the network.

Here I've moved 'file' into a newly created directory
'dir2', which isn't
available on the target-side.
> $ find src
> src
> src/dir1
> src/dir1/dir2
> src/dir1/dir2/file
>
> $ find dst
> dst
> dst/dir1
> dst/dir1/file
>
> $ rsync -avv --detect-renamed -e ssh src/dir1 localhost:dst
> [...]
> total: matches=0  hash_hits=0  false_alarms=0 data=4544512
The file isn't detected as a rename.

If I create 'dir2' manually on the target-side, it is detected
correctly:
> $ find src
> src
> src/dir1
> src/dir1/dir2
> src/dir1/dir2/file
>
> $ find dst
> dst
> dst/dir1
> dst/dir1/file
> dst/dir1/dir2
>
> $ rsync -avv --detect-renamed -e ssh src/dir1 localhost:dst
> [...]
> found renamed: dir1/file => dir1/dir2/file
> [...]
> total: matches=2136  hash_hits=2136  false_alarms=0 data=0
Is this a bug (I think so), or should it be like this?
-------------- next part --------------
HTML attachment scrubbed and removed

On Fri, 2007-12-28 at 17:31 +0100, Erik Pettersson
wrote:
> I'm trying out the 'detect-renamed'-patch, and I've
encountered some
> odd behavior.
> Basicly, what I've noticed is that if I move a file into a newly
> created directory (which is what happens if I rename a directory, for
> example), the file isn't detected as renamed. However, if I manually
> create the new directory on the target-side, the file is correctly
> detected as a rename, and isn't transfered over the network. 
> 
> Here I've moved 'file' into a newly created directory
'dir2', which
> isn't available on the target-side.
> 
> > $ find src
> > src
> > src/dir1
> > src/dir1/dir2
> > src/dir1/dir2/file 
> > 
> > $ find dst
> > dst
> > dst/dir1
> > dst/dir1/file
> > 
> > $ rsync -avv --detect-renamed -e ssh src/dir1 localhost:dst
> > [...]
> > total: matches=0  hash_hits=0  false_alarms=0 data=4544512 
> 
> The file isn't detected as a rename.
I see what the problem is.  When the receiving rsync considers
dst/dir1/file for a rename, dst/dir/dir2 does not yet exist, so rsync
cannot create the partial dir for dst/dir/dir2 in order to stage the
rename.

One workaround would be to do a preliminary run with --include='*/'
--exclude='*' to create all the necessary directories in the destination
before doing the main run.

One possible fix would be to call create_directory_path to ensure that
the destination directory that is the target of the rename exists before
handling the partial dir.  Unfortunately, create_directory_path will
give any created directories default permissions instead of flist ones,
which could be bad.  To avoid this problem, rsync could look up each
intermediate directory in the flist and, if it is found, give it 700
permissions for later fixing (like the incremental recursion code does).
This is awkward, which I believe is a sign that the "reverse" design
of
pre-scanning the destination files and looking up each source file in
turn may be better.  This design was proposed by Charles Perreault to me
off-list and quoted in:

http://lists.samba.org/archive/rsync/2007-November/019067.html

Matt