rsync - May 2014 - directory permissions not set until all files copied

I noticed today, that rsync does not appear to set the permissions on a
directory until all of the contents are in place.  this only really becomes
noticeable and/or problematic when rsync'ing a large directory tree as root
to a new, empty, location.

For example, if you have the following tree:

/foo
  /bar
    /file1
    /file2
  /baz
    /file3
    /file4

When rsyncing this tree to another local location, the permissions on /bar
will be 700, the owner root, and the group root until file[12] has been
fully copied and the permissions set.  The permissions on foo will mirror
those mentioned for /bar untill file[1-4] have been fully copied and the
permissions/ownership have been set for bar and baz.

I can think of good reasons to want it to behave in this manner, but there
are also times where you would want a directory accessible as soon as
possible, even if all the contents are not yet there.  I can work around
this mostly by rsyncing only the directories first, but it seems like this
should be an option.  Is this something that has been considered?

I did search through the bug tracked and didn't find anything that seemed
related, but then again, I don't think this qualifies as a but, but rather
an enhancement.

If you are wondering about a specific use case where the current behavior
is problematic, here is my real life example.

I've been running btfrs for several years now, with the understanding that
it is still not "production" ready.  As such the data I have not it is
either expendable or backed up elsewhere, but I stil don't want to lose any
of it.  To limit my risk, rather than one large btrfs filesystem I have
several.  The ensure that if one volume is corrupted in some way, the
corruption will not affect the other volumes automatically.  At the same
time, I want to treat these as one unified filesystem, so I effectively
concatenate them via aufs.  Incidentally, this also allows me to more
easily address strange behavior because juggle smaller chunks.  If I had 1
10TB fs, and something was going wonky, I would have to have an additional
10TB of capacity to be able to offload it, but as 5 2TB filesystems, I can
juggle 2TB at a time.

It just so happens that I recently replaced my disk controller and I
noticed an unexpected side effect which was not immediately impactful to my
btrfs volumes, but prevented disks formatted with other filesystems from
being mountable at all.  As a precaution, I've been recreating all of my
btrfs volumes one at a time, and cycling my data to the newly created btrfs
filesystems.  I can do this all online without leaving my data unaccessible
due to the magic of aufs.  But... this permissions issue results in the
data not being accessible except by root until the whole tree has finished
copying.  (unless I manually do a chmod/chown -R.

In any case, it's not a major issue, but I figured it was worth mentioning
as this specific behavior is not documented as far as I could find and thus
it's not clear if this is expected behavior or not.

Thanks for your time!

BTW, I realize saying btrfs filesystem is the equivalent to saying ATM
machine, but saying "I created a btrfs" or "I created 2
btrfses" sounds
strange and perhaps misleading.  :)

-- 
Michael Johnson - MJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20140507/568b7203/attachment.html>

On Wed, May 7, 2014 at 1:13 AM, Michael Johnson - MJ <mj at revmj.com>
wrote:
> I noticed today, that rsync does not appear to set the permissions on a
> directory until all of the contents are in place.
>
This is true if you are using incremental recursion since rsync is trying
to finish a dir's contents in time to set the dir's mtime before moving
on
to the reset of the tree.  If you specify --no-inc-resursive you'll avoid
these early-create directories.  I am also checking in a change to the
--omit-dir-times code (-O) (or --no-times) to make it also avoid these
early-create directories, which allows someone to continue to use
incremental recursion w/o seeing these early-create directories.

..wayne..
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20140525/ce3efab0/attachment.html>