rsync - Jul 2011 - [Bug 8336] New: parent dir permission

https://bugzilla.samba.org/show_bug.cgi?id=8336

           Summary: parent dir permission
           Product: rsync
           Version: 3.0.8
          Platform: All
        OS/Version: Solaris
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: andras.porjesz at ericsson.com
         QAContact: rsync-qa at samba.org


Hi all,
I use the following syntax:
rsync -av --relative --link-dest=/a/refdir /a/sourcedir/./dirtree/to/transfer
<ip>::module/a/sourcedir
in rsyncd.conf I set the following:
uid=root
gid=root
use chroot=true
--------------
the problem is the directory named sourcedir on the receiver side. rsyncd sets
its ownership to root:root instead of either leaving it as is or taking the
values from the sender (this is also valid for the directory a).

Am I missed something, or is this a bug, or ??
Is there any way to set those rights?
Thanks

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Matt McCutchen <matt at mattmccutchen.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
            Summary|parent dir permission       |--implied-dirs should
                   |                            |include the file-list root
                   |                            |by default

--- Comment #1 from Matt McCutchen <matt at mattmccutchen.net> 2011-07-28
19:09:12 UTC ---
The "a" directory is outside the scope of the file list.  Rsync is not
responsible for its attributes.

"sourcedir" on the destination is at the file-list root.  Rsync does
not
include the file-list root as an implied dir unless you write an extra
"." in
the source arg: /a/sourcedir/./.g/dirtree/to/transfer .  Leaving out the
file-list root seems like a gratuitous special case to me.  Wayne, is there a
reason for this (other than backward compatibility now that rsync has been
doing it)?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #2 from Wayne Davison <wayned at samba.org> 2011-08-01
05:34:53 UTC ---
If you move the dot prior to sourcedir in the source arg
(/a/./sourcedir/dirtree/to/transfer) and omit sourcedir in the dest
(foo::module/a/), then sourcedir will get its attributes set by the transfer.

Matt mentions another potential arg change that would work, though you should
remove the superfluous "g" in the ".g" arg (that was just a
typo).

As for why the root dir doesn't get set by the transfer, it's because
that
directory is not being sent, just specific files within it.  Rsync always
treats piecemeal transfers as not including the . dir unless explicitly
requested.  e.g., sending (no -R) sub1/foo sub2/bar to dest/ does not affect
"dest", it just puts 2 files into that dir.  In the same way, sending
the same
args but with -R puts two dirs into the dest, but doesn't affect
"dest".

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Matt McCutchen <matt at mattmccutchen.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WONTFIX                     |

--- Comment #3 from Matt McCutchen <matt at mattmccutchen.net> 2011-08-01
12:05:06 UTC ---
(In reply to comment #2)
> If you move the dot prior to sourcedir in the source arg
> (/a/./sourcedir/dirtree/to/transfer) and omit sourcedir in the dest
> (foo::module/a/), then sourcedir will get its attributes set by the
transfer.
And a specially prepared symlink would be needed for the --link-dest to work;
that's why I prefer my method.
> As for why the root dir doesn't get set by the transfer, it's
because that
> directory is not being sent, just specific files within it.
That rationale does not fully justify the behavior with -R.  If I do "rsync
-R
sub/foo sub/bar dest/", I could make the exact same argument that "sub
is not
being sent, just specific files within it", yet "sub" is included
as an implied
dir.  If you think of file-list space as a tree, then a source arg that targets
a particular node of the tree will pick up all the ancestors as implied dirs --
except the root.  In that sense, the behavior seems pretty arbitrary.  IMO,
rsync should include all of the ancestors or none of them.

Without -R, since there can only be one component of path info on the source
and "rsync /path/to/item dest/" to put the "item" in the
"dest" without
extraneous effects is very traditional usage, the current behavior makes sense.
 But leaving --implied-dirs off by default when -R is off would be enough to
maintain this behavior with my proposed change.

I'm reopening because I have refuted the justification you put forth. 
You're
welcome to WONTFIX again as your prerogative, which leaves me with the usual
option (fork if I care enough).

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #4 from Wayne Davison <wayned at samba.org> 2011-08-02
01:15:31 UTC ---
(In reply to comment #3)
> If I do "rsync -R sub/foo sub/bar dest/", I could make the exact
same
> argument that "sub is not being sent, just specific files within
it"
But that's exactly what -R does -- it sends the dirs mentioned as path
elements
(it just doesn't scan the dirs).  There's no '.' dir mentioned
in the path, so
it doesn't get sent.  This behavior is very old, so I would be very hesitant
to
change it even if I thought it made more sense (which I don't agree that it
does).  The general rule is that the dot dir only gets sent if you're
sending
the full contents of a directory, or if you've included '.' as a
part of the
path with --relative.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

andras.porjesz at ericsson.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WONTFIX                     |

--- Comment #5 from andras.porjesz at ericsson.com 2011-08-08 09:05:42 UTC ---
Sorry, I cannot agree.
rsync must take care of the dirs "a" and "sourcedir" -
especially when they do
not exist. Rsync must create them and I cannot find any option to configure
rsync on "how to create that root dir". I have at about 500
"sourcedir"s to
rsync and they belong to different users. The created /a/sourcedir structure on
the receiver side must allow to the users to modify its content or even remove
it completely.
I understand, rsync is unable to set those permissions because currently does
not send these information (just only the name of these dirs), so it can be
either a bug or a feature request.
The idea to use /a/sourcedir/././something was promising but it wont work on
"a".

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #6 from andras.porjesz at ericsson.com 2011-08-08 10:29:54 UTC ---
and also, using the /a/sourcedir/././something syntax and a non-existent
/a/sourcedir will not work as planned.
rsync will create /a/sourcedir, but instead of setting the original owner/group
of the directory . it is set by the user/group who started the rsync process.

on sender side
drwxr-xr-x  42 71262    8092  sourcedir
id
uid=32504 gid=16019

on receiver side the new sourcedir
drwxrwxr-x   3 32504    16019 

the rsync command was:
/usr/local/bin/rsync -azhh --relative --stats --numeric-ids
--link-dest=/a/sourcedir   /view/sourcedir/././some/dir/to/transfer
ip::module/a/sourcedir

I think there is a bug here....

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #7 from Matt McCutchen <matt at mattmccutchen.net> 2011-08-08
12:31:52 UTC ---
(In reply to comment #4)
> (In reply to comment #3)
> > If I do "rsync -R sub/foo sub/bar dest/", I could make the
exact same
> > argument that "sub is not being sent, just specific files within
it"
> 
> But that's exactly what -R does -- it sends the dirs mentioned as path
elements
> (it just doesn't scan the dirs).  There's no '.' dir
mentioned in the path, so
> it doesn't get sent.
You've just restated the current behavior.  My question is, why is it
useful?
When would you ever want to preserve attributes of all ancestors except the
root dir?  I cannot imagine any use for avoiding attribute preservation on that
particular directory.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #8 from Wayne Davison <wayned at samba.org> 2011-08-08
14:49:51 UTC ---
(In reply to comment #7)
> I cannot imagine any use for avoiding attribute preservation on that
> particular directory.
That's weird, because that has been the desired behavior in all the cases I
have ever used the option.  If I'm copying a particular set of files, I
don't
want effects outside of the files I specify.  I might not even have permissions
to tweak the root dir in a situation where a particular set of subdirs was
created for me (e.g. a copy of my home subdir).  The current method lets me
choose to include it or not.  Making it mandatory takes away the user's
choice.
 The parent dir may not even be the same dir, e.g. /path1/./some/path
/path2/./another/one -- in such a case, which dir should be the root of the
transfer?  With the current method, the user can choose one, should they so
desire.

As for being unable to specify the permissions that get set on the created
directory, that is true for all instances where the destination directory gets
created when it is outside the transfer (e.g. rsync -ai file1 file2 dest), so
is not something that directly pertains to --relative.  In fact, the --relative
option gives us one way affect it, should we want to do so: rsync -aiR ./file1
./file2 dest.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #9 from andras.porjesz at ericsson.com 2011-08-08 15:09:41 UTC ---
(In reply to comment #8)
> (In reply to comment #7)
> > I cannot imagine any use for avoiding attribute preservation on that
> > particular directory.
> That's weird, because that has been the desired behavior in all the
cases I
> have ever used the option.  If I'm copying a particular set of files, I
don't
> want effects outside of the files I specify.  I might not even have
permissions
> to tweak the root dir in a situation where a particular set of subdirs was
> created for me (e.g. a copy of my home subdir).  The current method lets me
> choose to include it or not.  Making it mandatory takes away the user's
choice.
>  The parent dir may not even be the same dir, e.g. /path1/./some/path
> /path2/./another/one -- in such a case, which dir should be the root of the
> transfer?  With the current method, the user can choose one, should they so
> desire.
> As for being unable to specify the permissions that get set on the created
> directory, that is true for all instances where the destination directory
gets
> created when it is outside the transfer (e.g. rsync -ai file1 file2 dest),
so
> is not something that directly pertains to --relative.  In fact, the
--relative
> option gives us one way affect it, should we want to do so: rsync -aiR
./file1
> ./file2 dest.
so if you want to copy a particular set of files (owned by several users) and
you want to have the same permissions on the receiver side you must set the
permission of the root to the same as it was (and when newly created).
Otherwise you may also make the copy unavailable to the original user.
It is not related anyhow to the --relative flag.
Noone requested it to be mandatory, it would be much better to implement a new
flag. By default it will do nothing (like now), but by setting it rsync will
take care of a few more permissions.
To implement this feature outside of rsync will degrade the performance of the
syncing process significantly.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #10 from Wayne Davison <wayned at samba.org> 2011-08-08
15:20:39 UTC ---
./file2 dest.(In reply to comment #5)
> The created /a/sourcedir structure on the receiver side
> [...] it wont work on "a"
There is no created "a" on the receiver side in your examples, since
"a" is
outside of the transfer and not the top-most directory in the destination path.
 That directory has to pre-exist, and you can create it however you wish.  Or,
you could choose to put "a" in the transfer by omitting the /./
elements from
the source path, in which case the link-dest dir would also need those path
elements or a symlink on a subpath of the omitted dirs that points to the root
of the link-dest hierarchy (which would be best put into a separate directory
to avoid any conflict with arriving directories).  e.g.

Either this:

mkdir -p /a/refdir2/a
mv /a/refdir /a/refdir2/a/sourcedir
mv /a/refdir2 /a/refdir
LDEST=/a/refdir

Or this:

mkdir -p /linkdest/a
ln -s /a/refdir /linkdest/a/sourcedir
LDEST=/linkdest

And run:

rsync -avR --link-dest=$LDEST /a/sourcedir/dirtree/to/transfer
<ip>::module/

As for a new flag, that would be equivalent to just adding ./ to the source
path, which is what we have now.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #11 from Matt McCutchen <matt at mattmccutchen.net> 2011-08-08
16:45:51 UTC ---
(In reply to comment #8)
> That's weird, because that has been the desired behavior in all the
cases I
> have ever used the option.  If I'm copying a particular set of files, I
don't
> want effects outside of the files I specify.
Absolutely... but how is /path/to/./foo/bar/ any less a
"specification" of .
than of ./foo ?
> I might not even have permissions
> to tweak the root dir in a situation where a particular set of subdirs was
> created for me (e.g. a copy of my home subdir).
Could you give me a more detailed example?
> The current method lets me
> choose to include it or not.  Making it mandatory takes away the user's
choice.
Valid point.  This is probably a case like --chmod=ugo=rwX where the default
should stand but users should realize it often isn't what they want.
> The parent dir may not even be the same dir, e.g. /path1/./some/path
> /path2/./another/one -- in such a case, which dir should be the root of the
> transfer?
/path1/, according to the usual rule that earlier source args take priority. 
This issue is in no way specific to the file-list root (you can equally well
have /usr1/./share/rsync and /usr2/./share/frobnitz collide at
"share"), so it
does not justify treating the file-list root specially.
> With the current method, the user can choose one, should they so
> desire.
That only helps with the file-list root.  The complete solution would be to
support --exclude of implied dirs, like IIRC rsync 2.6.9 did.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #12 from Wayne Davison <wayned at samba.org> 2011-08-08
17:00:12 UTC ---
(In reply to comment #11)
> how is /path/to/./foo/bar/ any less a "specification" of . than
of ./foo ?
Because the first /./ is spec'ed to be a separator for the out-of-tree dirs
from the in-tree dirs.  If it had additional meaning of adding '.' to
the path,
it would make that mandatory, which is bad.  Remember, choice is good.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

andras.porjesz at ericsson.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WONTFIX                     |

--- Comment #13 from andras.porjesz at ericsson.com 2011-08-09 12:01:12 UTC ---
I think you missed the main point.
Let we return to the description, to the original command. Additionally:
on the sender side the active user is: a_user:a_group
the ownership of sourcedir is: d_user:d_group
the permission: drwxr-xr-x

Please tell me what is the expected result, how will sourcedir on the receiver
side created (including permissions and ownership)?

I do not think it is weird to create that directory (if does not exist).
To handle it somehow if it was already created and the permissions are not set
properly (?) you can have the possibility to warn the user or do whatever you
want, but this is not the original case.
> If I'm copying a particular set of files, I don't want effects
outside of the files I specify.
This is your case. My case is:
I want to sync hundreds of workspaces used by several users and I need all the
same permissions on both sides. Is this weird?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #14 from Wayne Davison <wayned at samba.org> 2011-08-09
15:55:31 UTC ---
(In reply to comment #13)
> I want to sync hundreds of workspaces used by several users
> and I need all the same permissions on both sides.
Yup.  And we've given you 2 different ways to do this.  If you just need the
one-level of created dir set right, add the extra dot dir to the path.  If you
need all the path elements in the destination path set, make them all implied
directories in the transfer by including them in the transfer path (and tweak
your link-dest setup).

I noticed that in one comment you said that the /././ bit didn't work for
you,
but I tested it back to 2.6.9, and it works in all rsync versions that I tried
(and with both "use chroot" and w/o in the daemon config).  Make sure
that the
daemon has that module configured to run as root if you want it to be changing
the directory ownership.

If you add 2 -i options to your command, rsync will output all the items in the
transfer, and the extra dot will show up as an initial "./" dir in the
list of
transferred files.  That dot dir will affect the destination dir.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

andras.porjesz at ericsson.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|WONTFIX                     |

--- Comment #15 from andras.porjesz at ericsson.com 2011-08-10 11:18:30 UTC ---

1. there is a question in comment #13, what is the expected behaviour?
Is there any official documentation on it? I assume no, from my side it looks
like rsync must do something and makes something strange, unpredictable and
unacceptable.

2. using a one dot syntax rsync will create that dir with root:root ownership,
and with permissions 755.

3. using a double dot syntax rsync will create that dir with a strange
ownership and permission settings. I told you the ownership depends on the user
not the dir. I'm sure you can reproduce it using 3.0.8.

4. The links you wrote about: I have at about 500 sourcedirs and they are
created on demand. I'm not able to prepare those links on the fly - or as I
told you I have to extend the sync process by checking and optionally create
those links (not to speak about that any of the sourcedirs can be used as
reference).

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID

--- Comment #16 from Wayne Davison <wayned at samba.org> 2011-08-10
23:22:54 UTC ---
(In reply to comment #15)
> 3. using a double dot syntax rsync will create that dir with a strange
> ownership and permission settings. I told you the ownership depends on the
user
> not the dir. I'm sure you can reproduce it using 3.0.8.
Nope, rsync works great.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

andras.porjesz at ericsson.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |REOPENED
         Resolution|INVALID                     |

--- Comment #17 from andras.porjesz at ericsson.com 2011-08-11 08:14:21 UTC ---
just answer me please, how should rsync work?

1. there is a question in comment #13, what is the expected behaviour?
Is there any official documentation on it?

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #18 from andras.porjesz at ericsson.com 2011-08-11 15:09:06 UTC ---
I found something strange.
(You can say it is another bug)
So running rsync with a double dot arg /a/sourcedir/././dirtree/to/transfer
will cause the sourcedir to be copied again and again.
I found now that the permission and ownership of that sourcedir will be copied
from the actual current dir (.) on the client side.
(if I cd into /var/adm/log, the sourcedir will be set to adm:adm)
see the log here:
-------------------------------------------------------------
[sender] change_dir(/var/adm/log)
send_files(0, .)
.d..t.og... ./
[sender] change_dir(/a/sourcedir)
send_files(3, /a/sourcedir/this)
.d          this/
send_files(5, /a/sourcedir/this/is)
.d          this/is/
send_files(7, /a/sourcedir/this/is/the)
.d          this/is/the/
send_files(9, /a/sourcedir/this/is/the/target)
.d          this/is/the/target/
----------------------------------------------------------------
this is obviously not ok, just a hard to find bug, because usually noone starts
rsync outside his own environment.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

--- Comment #19 from Wayne Davison <wayned at samba.org> 2011-08-27
19:18:24 UTC ---
Ouch, the dot dir was indeed coming from the current directory, not the
relative path prefix.  In my testing those 2 directories happened to have the
same ownership and permissions, so I didn't notice that.  That bug is now
fixed
in the master and b3.0.x branches, so the fix will first appear in 3.0.9.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=8336

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |FIXED

--- Comment #20 from Wayne Davison <wayned at samba.org> 2011-09-10
21:19:40 UTC ---
The dot-dir fix coming in 3.0.9 should resolve this.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.