rsync - Jun 2004 - Multiple --compare-dest args again

Hi all.

A while ago (April 15th or so) I posted a patch that allows rsync to
take multiple --compare-dest or --link-dest arguments, allowing
fetching of files not present in multiple trees. I never got any
feedback on it, though, so I'm picking it up again. :) Is there any
interest in such a patch at all?

Below is the usage example i outlined back then;

--start--
[...] Its primary usage is to do incremental
backups on top of eachother. (My current backup system stores each
incremental as a set of files that differ from the latest full.) 

Example:

  First full backup:
  rsync -a somedir full-20040415/

  First incremental:
  rsync -a --compare-dest=../full-20040415 \
        somedir incr-20040416/

  Second incremental, on top of first:
  rsync -a --compare-dest=../incr-20040416 --compare-dest=../full-20040415
        somedir incr-20040417/

(The args must be given in the correct order; Latest incremental first,
last resort last.)

It scratches my itch, at least. Perhaps it can be of use to others as
well. :)
--end--

I'll skip attaching the patch for now, as I haven't updated it to fit
CVS, but I'll update it should there be any interest.

Yours,
Vidar

On Tue, Jun 22, 2004 at 11:41:33PM +0200, Vidar Madsen
wrote:
> A while ago (April 15th or so) I posted a patch that allows rsync to
> take multiple --compare-dest or --link-dest arguments, allowing
> fetching of files not present in multiple trees. I never got any
> feedback on it, though, so I'm picking it up again.
Sorry for neglecting to respond earlier.  Here are some comments:

One problem I had with the patch is actually an exacerbation of an
existing deficiency in the current code:  both the generator and the
receiver each do their own scan for a basis-file when the generator
should really tell the receiver anytime that it uses a non-default
value for the basis-file (both for efficiency's sake and also to avoid
a race condition where the generator makes a different decision from
the receiver due to changing files).  This is not a huge objection,
though, especially if the various compare-dest dirs are not being
changed.

Another thing that I note is that you should be able to accomplish the
same thing more efficiently using --backup-dir or --link-dest.  For
instance, using --backup-dir:

    rsync -a somedir full-backup
    rsync -a --backup-dir=/path/incr-20040415 somedir full-backup
    rsync -a --backup-dir=/path/incr-20040416 somedir full-backup

In this idiom the full-backup dir is always fully up-to-date, so that
is different from your current method.  Note that each backup dir
refers to the *previous* day since the changed files from the last
backup get put there.

In the --link-dest method (which requires hard-links to work on the
destination filesystem), each day's backup looks like a full backup,
but it only takes the space of an incremental because of all the
hard-linking (though it does eat more inodes):

    rsync -a somedir back-20040415
    rsync -a --link-dest=/path/back-20040415 somedir back-20040416
    rsync -a --link-dest=/path/back-20040416 somedir back-20040417
> I'll skip attaching the patch for now, as I haven't updated it to
fit
> CVS, but I'll update it should there be any interest.
I've created an updated version of the patch for CVS, but I haven't
yet checked it into the patches dir (I'm still deciding whether I
think it is needed or not).

..wayne..