rsync - Feb 2009 - rsync --files-from

Hi,

I am working on a replication tool for lustre filesystems.

We have introduced a mechanism for getting the changes that have been
made to a lustre filesystem - similar to inotify. We call this
mechanism, changelogs.

For performing replication, we open the source file using its 'file
identifier' or FID (think of it an inode that is unique cluster wide)
instead of its path. The file is then replicated to the target.

We would like to use rsync to do this data copy. I was investigating
ways to pass a stream of filenames to rsync. It would seem that we
cannot use '--files-from' as we copy from $SRC/.lustre/$FID to
$DST/path/to/file.

Is there a way of passing a list of source _and_ destination paths??


It would have been even better if there were a rsync library that our
replication tool could link to, instead of passing a list of files.

The librsync project on sourceforge (librsync.sourceforge.net) seems to
be unmaintained. Does anyone on this list have any experience with using
rsync as a library?

Kind regards,
Manoj

-- 
Manoj Joseph
Lustre Group,
Sun Microsystems, Inc.

On Thu, Feb 26, 2009 at 07:55:22PM -0700, Manoj Joseph
wrote:
> Is there a way of passing a list of source _and_ destination paths??
Only for a single-file transfer.  So, to do this with rsync, you'd need
to transfer each file separately, possibly using the lighter daemon
protocol (which could be routed over an established ssh tunnel).
> The librsync project on sourceforge (librsync.sourceforge.net) seems to
> be unmaintained. Does anyone on this list have any experience with using
> rsync as a library?
To see an example of using librsync to build an rsync-like tool, see my
rZync code:

    http://opencoder.net/new-protocol.html

This tool supports being controlled by a series of transfer commands,
much like an sftp that can use the rsync delta-transfer protocol.  The
code was never fully polished (e.g. the uid/gid code needs work), but
you might find it useful in either making something simpler, or in
testing/debugging/enhancing what is there.

..wayne..