rsync - Oct 2003 - How to change permissions on a directory without meaning to... - include/exclude semantics

Hi,

Think I've found either a documentation bug, in rsync (or possibly a 
code bug), with the include/exclude feature.  This occurs on (at least) 
2.5.5 and 2.5.6

First I created a file list of files that I wanted backed up


[root@poppy root]# cat /tmp/sendmail-rpmverify-out
/etc/
/etc/aliases
/etc/mail/
/etc/mail/helpfile
/etc/mail/local-host-names
/etc/mail/sendmail.mc
/etc/mail/statistics
/etc/mail/trusted-users
/etc/mail/virtusertable
/etc/sendmail.cf
/etc/sysconfig/sendmail

2. Then I fed this file to rsync as the files that it should copy from 
one path to another:

# rsync -av --include-from /tmp/sendmail-rpmverify-out --exclude '*' /
/tmp/

3. Check file permissions on /tmp

[root@poppy root]# ls -ald /tmp/
drwxr-xr-x   36 root     root         4096 Oct  3 05:44 /tmp/

So, the --exclude '*'  doesn't actually exclude '/' from the
list, so
the file permissions on / are copied over to /tmp

No files under /tmp, except those specified in the file touched tho'.

 From the rsync manual page:

"rsync builds an ordered list of include/exclude options as specified on 
the command line. When a filename is encountered, rsync checks the name 
against  each  exclude/include pattern in turn. The first matching 
pattern is acted on. If it is  an  exclude  pattern,  then  that file 
is skipped. If it is an include pattern then that filename is not 
skipped.  If no matching include/exclude pattern is found then  the
filename is not skipped."


I thought maybe I wasn't telling it explicitly enough not to include
'/'
in its list, so I tried this:

rsync -av --include-from /tmp/sendmail-rpmverify-out --exclude '**'
/ /tmp/

and this:

rsync -av --include-from /tmp/sendmail-rpmverify-out --exclude '/**'
/ /tmp/

and even this:

rsync -av --include-from /tmp/sendmail-rpmverify-out --exclude '/**'
--exclude / / /tmp/

but it still does the same thing, and updates the permissions on /tmp -
thus it seems that the source path specified to rsync (in this case '/')
is always implicitly included, even though this isn't documented in the
man page.  If this is a feature, then I think it would be helpful to 
state it in the "include/exclude" part of the manual page..

Unfortunately, before I worked out what had happened, the mod-perl web 
server on this machine had got the load up to 40, and partially stopped 
working, as it was unable to create a temporary socket in /tmp/   :o(


Comments/thoughts?


Cheers,

Tim.

On Mon, Oct 06, 2003 at 05:41:45PM +0100, Tim Small wrote:
[redacted]
> Hi,
> 
> Think I've found either a documentation bug, in rsync (or possibly a 
> code bug), with the include/exclude feature.  This occurs on (at least) 
> 2.5.5 and 2.5.6
> 
> First I created a file list of files that I wanted backed up
> 
[snip]
> 
> 2. Then I fed this file to rsync as the files that it should copy from 
> one path to another:
> 
> # rsync -av --include-from /tmp/sendmail-rpmverify-out --exclude
'*' / /tmp/
> 
> 3. Check file permissions on /tmp
> 
> [root@poppy root]# ls -ald /tmp/
> drwxr-xr-x   36 root     root         4096 Oct  3 05:44 /tmp/
> 
> So, the --exclude '*'  doesn't actually exclude '/'
from the list, so
> the file permissions on / are copied over to /tmp
> 
> No files under /tmp, except those specified in the file touched tho'.
> 
> From the rsync manual page:
> 
> "rsync builds an ordered list of include/exclude options as specified
on
> the command line. When a filename is encountered, rsync checks the name 
> against  each  exclude/include pattern in turn. The first matching 
> pattern is acted on. If it is  an  exclude  pattern,  then  that file 
> is skipped. If it is an include pattern then that filename is not 
> skipped.  If no matching include/exclude pattern is found then  the
> filename is not skipped."
> 
> 
> I thought maybe I wasn't telling it explicitly enough not to include
'/'
> in its list, so I tried this:
> 
[snip]
> but it still does the same thing, and updates the permissions on /tmp -
> thus it seems that the source path specified to rsync (in this case
'/')
> is always implicitly included, even though this isn't documented in the
> man page.  If this is a feature, then I think it would be helpful to 
> state it in the "include/exclude" part of the manual page..
You are fighting conflicting semantics of slash.  In a path
specification a leading slash indicates absolute path.  In
an exclude pattern the leading slash anchors the pattern to
the basedir.  In the patterns a trailing slash matches only
directories.  As a source path a trailing slash means
transfer the contents but not the directory.  Yet with all
of those special meanings as a path a single slash must
still indicate the root directory with the least surprise.

Wayne has been fighting this issue lately in CVS so any
suggestions i make here regarding variations on the path are
going to be very version dependant.

Your attempt to exclude the root directory, if successful,
would have prevented the whole transfer on the parent
directory pathwalking principle.  So that is not the way to
go.

Using a source of "/." should give the same behavior as you
are experiencing and "/./" the behavior you want.  Wayne may
have some comments on this.

My recommendation is to avoid the issue entirely.  If you
are going to sync from root, don't make a directory with
special permissions requirements the destination.  In other
words, use a subdirectory of /tmp, not /tmp.  Changing the
permissions of /tmp could have dire consequences ranging
from breaking applications to adversly affecting security.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Yes, it is not immediately obvious that when rsync is transferring a
path that ends in '/' that it is really transferring the "."
directory.
Thus, this command:

    rsync -av /some/path/ /another/spot/

... actually transfers "/some/path/." to "/another/spot/.",
which
updates its permissions/ownership/etc.  The code even makes sure that it
is impossible to exclude this "." dir from the transfer, or else
recursion would stop.  So, I actually consider this a bad design in the
rsync protocol, but not one that I would want to attempt to change at
the moment (if ever).  You are right that this should be documented
better.

..wayne..