rsync - Feb 2006 - Copying many sources to different places inside a destination

Rsync people,

I am preparing to overhaul my Web site's build system, and I am looking
for a convenient way to collect files and directories from various
places on my computer and put them at various locations inside a
destination directory to be posted to my Web site.  So far, my script
has been running rsync once for each source, but this is too slow.  Is
there a way to get rsync to collect many sources in one pass to
destination locations more flexible than those that --relative will
allow me?

For example, suppose I want to copy sources something like this:
	~/main-web-content/     -> ~/www/
	~/logicdes/             -> ~/www/logicdes/
	~/bigint/svn-repo/      -> ~/www/bigint-svn/
        ~/vectors/vectors.tex   -> ~/www/math/vectors.tex
        ~/mirror/mirror-latest/ -> ~/www/mirror/bin/
and so forth.  All the sources go different places inside ~/www and
together make up everything I want in ~/www, so I would like the ability
to delete extraneous files.  But the parts of the destination paths
following ~/www are not always suffixes of the source paths, so
--relative with dot-dirs is not good enough.

If, as I suspect, support for such rearrangement of sources would
require a new syntax, let me propose the following.  When --relative is
repeated, the sending rsync takes source arguments (or --files-from
lines) in pairs: the first gives the real location of the source and the
second gives the prefix to put on the source filenames in the file list.
Then these two calls would be equivalent:
	rsync -r foo/src bar/othersrc dest/
	rsync -r -RR foo/src src bar/othersrc othersrc dest/
as would these:
	rsync -r -R foo/src bar/./othersrc dest/
	rsync -r -RR foo/src foo/src bar/othersrc othersrc dest/
-- 
Matt McCutchen
hashproduct@verizon.net
http://hashproduct.metaesthetics.net/

On Mon, Feb 27, 2006 at 06:33:44PM -0500, Matt McCutchen
wrote:
> But the parts of the destination paths following ~/www are not always
> suffixes of the source paths, so --relative with dot-dirs is not good
> enough.
While rsync doesn't currently support this, but you might be able to
accomplish what you want by using some creative symlinks on the
destination side and --keep-dirlinks.  For instance:

mkdir /some/rsync-dest
cd /some/rsync-dest
ln -s ~/www main-web-content
ln -s ~/www/logicdes logicdes
ln -s ~/www/bigint-svn svn-repo
ln -s ~/www/math vectors
ln -s ~/www/mirror/bin mirror-latest

cd ~
rsync -avRK main-web-content logicdes bigint/./svn-repo \
           vectors/vectors.tex mirror/./mirror-latest /some/rsync-dest

With this arrangement you would need to be careful to either not use
--delete or to use some excludes to avoid having a problem with the
main-web-content interacting with the other content that is going into
the same dir.  E.g., you might need to specify options such as:

    --exclude=main-web-content/{logicdes,bigint-svn,math,mirror}

A similar symlink solution could be done on the source side using the
new --copy-dirlinks option, but only if the nesting of directories
allowed it to happen (which does not seem to be the case in your
example).

..wayne..