atomic transaction set option to rsync

Dag Wieers
Mon, 03 Jan 2005 16:37:12 -0000

Hi,

Apparently a change of behaviour from rsync 2.5 to rsync 2.6 affected the
way I worked. I provide RPM repositories that I mirror using rsync. It is
important to have the repository meta-data in sync with the data otherwise
people have errors using Yum or Apt.

In the old days (with older rsyncs) I was able to influence the order in
which my transaction set was processed by changing the order of
directories I wanted rsync to mirror, so that the metadata was uploaded
after most of the data and deletes were done at the end of the
transaction.

With newer rsyncs, rsync seems to sort the transaction set, so I cannot
longer use this trick to have the metadata uploaded just after the data in
the same transaction set.

I was wondering if it was possible and acceptable to have an rsync option
to update the whole transaction in a atomic (or near-atomic way). This
will also prevent the current problems when a mirror is rsyncing another
mirror that is rsyncing itself. Since I have little bandwidth to update
than most other mirrors, I'm often caught in this secnario.

There are other ways to work around it, either by uploading in different
steps (which is impractical in my scenario) or by using a staging area
(which is impossible and impractical for large mirror sites).

An option to atomically sync a transaction set would be a god gift for
situations like these and probably (if not too much overhead) a behaviour
most repository mirrors would want by default.

--   dag wieers,  dag@wieers.com,  http://dag.wieers.com/   --
[all I want is a warm bed and a kind word and unlimited power]