rsync - Mar 2014 - [Bug 10495] New: "skipping directory foo" (does not transfer directories by default)

https://bugzilla.samba.org/show_bug.cgi?id=10495

           Summary: "skipping directory foo" (does not transfer
                    directories by default)
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: major
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: chealer at gmail.com
         QAContact: rsync-qa at samba.org


rsync's manual does not specify its usage. However, examples are provided in
the USAGE section. Unfortunately, there is a huge error in the very first
example:

rsync -t *.c foo:src/ 


 This would transfer all files matching the pattern *.c from the current
directory to the directory src on the machine foo. If any of the files already
exist on the remote system then the rsync remote-update protocol is used to
update the file by sending only the differences. See the tech report for
details. 

In fact, only the files matching the pattern and with the right type are
copied. I do not know which types are copied by default, but I know that
directories are not (unless this would be a Debian-specific behavior). If
--dirs is not specified, trying the above will throw a "skipping directory
x"
for all matching directories.

By the way, it is far from obvious which "tech report" this
description refers
to.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |INVALID

--- Comment #1 from Wayne Davison <wayned at samba.org> 2014-03-14
00:03:46 UTC ---
The example is copying *.c files, not directories.  The manpage clearly states
"all files matching the pattern *.c".

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #2 from Kevin Korb <rsync at sanitarium.net> 2014-03-14
00:12:53 UTC ---
BTW, there is also a recursive example in the include/exclude pattern
rules:
>  The combination  of  "+ */", "+ *.c", and "-
*" would include all directories and C source files but nothing else (see
also the --prune-empty-dirs option)
In other words:
rsync -rt --prune-empty-dirs --include="*/" --include="*.c"
--exclude="*"
/path/to/source/ /path/to/target/
will copy all *.c files and the directories leading to them in an entire tree.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Filipus Klutiero <chealer at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |

--- Comment #3 from Filipus Klutiero <chealer at gmail.com> 2014-03-14
03:05:10 UTC ---
Files and directories are not exclusive. Directories are files, as the base
specifications explain.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #4 from Kevin Korb <rsync at sanitarium.net> 2014-03-14
03:16:04 UTC ---
Directories are only followed if you use -r aka --recursive.  Note that in my
example I did so.  However, if you added -r to your example you would get any
directory in the current directory named *.c recursively.  IOW it would
recursively copy a directory named foo.c but would not copy foo/bar.c

Simply put, *.c as the source is expanded by the shell globbing.  Rsync is
actually given a list of every file/directory that matches *.c.  Then without
--recursive it ignores the ones that are directories.  Without --specials it
ignores those too.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #5 from Filipus Klutiero <chealer at gmail.com> 2014-03-14
03:36:27 UTC ---
I understand that --recursive will consider directories. The problem is that
the example doesn't contain --recursive.

I managed to get my backup working, I was just pointing out the problem in the
documentation (as your answers show, the problem is not an unexpected rsync
behavior I was getting, it's just the documentation).

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #6 from Kevin Korb <rsync at sanitarium.net> 2014-03-14
03:41:18 UTC ---
Because the user probably doesn't want *.c to match files and directories. 
If
they want *.c recursively they probably need the include/exclude example that
is listed elsewhere.  This is exactly how cp behaves btw.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #7 from Filipus Klutiero <chealer at gmail.com> 2014-03-14
04:39:59 UTC ---
Sorry if I was unclear, but I didn't mean to say that rsync's behavior
should
be changed. I do believe that users would expect rsync to transfer directories
by default, but this doesn't necessarily mean that changing the default is
desirable at this point. Indeed, cp has the same issue as rsync, so changing
one without the other would reduce consistency.

What I meant to say was that the documentation is misleading. If directories
are not copied by default, the manpage should probably have a fat warning, but
at least, it should not mislead readers.

The example description should probably say that only regular files will be
transferred, or that all files other than directories will be transferred. Or
something more complicated (I do not know rsync's default behavior). For
sure,
it would be more accurate to say "all files other than directories".

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID

--- Comment #8 from Wayne Davison <wayned at samba.org> 2014-03-14
06:38:48 UTC ---
That example you cited shows how to copy some c files to a remote src dir. 
There are no dirs involved, thus -r is not needed.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Filipus Klutiero <chealer at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |

--- Comment #9 from Filipus Klutiero <chealer at gmail.com> 2014-03-14
15:23:38 UTC ---
Ths example is clearly intended to copy C files, but that's not the
description
given:
 This would transfer all files matching the pattern *.c from the current
directory to the directory src on the machine foo.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID

--- Comment #10 from Wayne Davison <wayned at samba.org> 2014-03-14
23:20:01 UTC ---
... which is exactly what it does.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Filipus Klutiero <chealer at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |

--- Comment #11 from Filipus Klutiero <chealer at gmail.com> 2014-03-16
17:15:28 UTC ---
No, [it] would transfer *some* files matching the pattern *.c from the current
directory to the directory src on the machine foo.

At least directories will not be transferred.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #12 from Paul Slootman <paul at debian.org> 2014-03-19
06:57:03 UTC ---
(In reply to comment #11)
> No, [it] would transfer *some* files matching the pattern *.c from the
current
> directory to the directory src on the machine foo.
> 
> At least directories will not be transferred.
I think you're confusing "current directory" with "current
directory and all
subdirectories".

The example is clear.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

--- Comment #13 from Filipus Klutiero <chealer at gmail.com> 2014-03-22
05:18:35 UTC ---
That's not what I meant. Here is what this example will do on a simple
directory having 2 files matching *.c:

$ ls -lR
.:
total 4
drwxr-xr-x 2 test test 4096 mar 22 01:04 bar.c
-rw-r--r-- 1 test test    0 mar 22 01:04 foo.c

./bar.c:
total 0
test at vinci:~/testsrc$ rsync -t *.c ~/testdst/ 
skipping directory bar.c
test at vinci:~/testsrc$ ls -lR ~/testdst/
/home/test/testdst/:
total 0
-rw-r--r-- 1 test test 0 mar 22 01:04 foo.c
test at vinci:~/testsrc$

All files in testsrc/ are directory entries of testsrc/. Yet, after the
transfer, even though all of these files match *.c, only some have been
transferred. bar.c was not transferred, since it is a directory and rsync does
not transfer directories unless some options which weren't given are given.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID

--- Comment #14 from Wayne Davison <wayned at samba.org> 2014-03-22
06:56:32 UTC ---
People don't name their directories with .c suffixes.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Filipus Klutiero <chealer at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |

--- Comment #15 from Filipus Klutiero <chealer at gmail.com> 2014-03-22
20:54:31 UTC ---
People name directories in many ways. Almost all UNIX systems will have
directories matching *.d. The very first system I connected to has hundreds of
matches for locate *.c/. Is that enough to convince you to leave this ticket
alone until the bug has been fixed? If you'd rather spend more time in a way
which doesn't help rsync, you'll have to do that without me.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |WONTFIX

--- Comment #16 from Wayne Davison <wayned at samba.org> 2014-03-25
05:35:36 UTC ---
Examples are not the correct place to get into minutia details.  The example as
given is correct for what it is trying to convey.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10495

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          QAContact|rsync-qa at samba.org          |

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.