rsync - Apr 2004 - rsync problems from flist.c change

On behalf of the development team of R (www.R-project.org), I would like
to report the following problem with recent versions of rsync.

First, some background info.  The R Development Core Team uses CVS for
maintainins its source code archive.  Additionally, there is a mechanism
for "mirroring" some important (so-called recommended) extension
packages from a CRAN (Comprehensive R Archive Network) mirror into the
source tree, so that these packages which are not strictly part of the R
sources are built as well.  Mirroring happens via a call to rsync (see
below), which plays various tricks so that CVS, rsync and make happily
coexist.

In my case, I have a local CRAN mirror somewhere in /usr/local, so that
sources for recommended packages for the current development version of
R are in 

  /usr/local/src/apps/stat/R/src/contrib/2.0.0/Recommended/

Mirroring happens via

  $RSYNC -rcIzC --verbose --delete \
     --exclude=Makefile.in --exclude=Makefile --exclude=.cvsignore \
     --include=*.tar.gz --exclude=*.tgz \
     /usr/local/src/apps/stat/R/src/contrib/2.0.0/Recommended/ .

in the src/library/Recommended subdirectory of an R source tree.

Current versions of rsync seem to have a problem with deleting files
from src/library/Recommended in certain situations.  I can partially
reproduce this by (starting from a state where the two dirs are in sync)
copying an additional file into 'Recommended' and trying different
versions of rsync to see whether the file gets removed.  What I find is

	2.5.7		deletes
	2.6.0		deletes
	2.6.0+Debian	does not
	2.6.1rev-2	does not

Inspection of Debian's rsync_2.6.0-3.diff.gz seems to suggest that the
only relevant change is in flist.c:

--- rsync-2.6.0.orig/flist.c
+++ rsync-2.6.0/flist.c
@@ -871,7 +871,7 @@
        if (cvs_exclude) {
                if (strlen(fname) + strlen(".cvsignore") <=
MAXPATHLEN - 1) {
                        strcpy(p, ".cvsignore");
-                      
add_exclude_file(&exclude_list,fname,MISSING_OK,ADD_EXCL
UDE);
+                      
add_exclude_file(&local_exclude_list,fname,MISSING_OK,AD
D_EXCLUDE); // PS XXX
                } else {
                        io_error |= IOERR_GENERAL;
                        rprintf(FINFO,

And in fact, reverting this change seems to "fix" the problem.  (The
code in 2.6.1 is slightly different, but to the same effect.)

If rsync is called without the '-I' option, the problem does not seem to
appear.

Sorry for the rather lengthy description---we've been trying to find a
more reproducible example.

Hopefully, the above description rings a bell somewhere ...

Thanks
-k

On Thu, Apr 22, 2004 at 08:21:24AM +0200, Kurt Hornik
wrote:
> I can partially reproduce this by (starting from a state where the two
> dirs are in sync) copying an additional file into 'Recommended' and
> trying different versions of rsync to see whether the file gets
> removed.
Does the file you added match one of the excluded files (including any
default cvs-exclude patterns and patterns read from $HOME/.cvsignore or
taken from the $CVSIGNORE environment variable, but ignoring the
individual .cvsignore files, which you are excluding)?  You can add an
extra level of -v to see all the files that are being excluded from both
created lists (the sender's list and the receiver's deletion-comparison
list).

If you don't see the additional file you created being excluded, you can
add yet another -v level to see all the make_file() calls.  You should
see the extra file on the receiving side either mentioned in a
make_file() call or in an "excluding" message.  If the file goes into
the receiving side's list, is not present in the sending-side's list,
and the file is not deleted, then there's some kind of a bug.

Let me know what you find out.

..wayne..