rsync - Oct 2015 - [Bug 11572] New: rsync --debug doesn't work, and gives erroneous results when taken out

https://bugzilla.samba.org/show_bug.cgi?id=11572

            Bug ID: 11572
           Summary: rsync --debug doesn't work, and gives erroneous
                    results when taken out
           Product: rsync
           Version: 3.1.2
          Hardware: x64
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: core
          Assignee: wayned at samba.org
          Reporter: lutz.john at yahoo.com
        QA Contact: rsync-qa at samba.org

--debug doesn't work, while not using debug half works.

'half works' is such a way that there will
be empty directories of improper file directory attributes.


rsync -rav 2015 delete_me/
sending incremental file list
created directory delete_me
2015/
2015/reset.sh
2015/activism/

....

this is the result destination directory of the rsync:

ls -altr delete_me/2015
total 124
-rwxr--r--.  1 john john  508 Mar 22  2015 reset.sh
drwxrwxr-x. 30 john john 4096 Oct 18 05:59 .
drwx------.  2 john john 4096 Oct 24 12:45 dev
drwx------.  2 john john 4096 Oct 24 12:45 computer
drwx------.  2 john john 4096 Oct 24 12:45 chess
drwx------.  2 john john 4096 Oct 24 12:45 c64
drwx------.  2 john john 4096 Oct 24 12:45 business
drwx------.  2 john john 4096 Oct 24 12:45 artwork
drwx------.  2 john john 4096 Oct 24 12:45 animals
drwxrwxr-x.  3 john john 4096 Oct 24 12:45 ..
drwx------.  2 john john 4096 Oct 24 12:45 vr
drwx------.  2 john john 4096 Oct 24 12:45 tech
drwx------.  2 john john 4096 Oct 24 12:45 spirit
drwx------.  2 john john 4096 Oct 24 12:45 science
drwx------.  2 john john 4096 Oct 24 12:45 receipt
drwx------.  2 john john 4096 Oct 24 12:45 reallife
drwx------.  2 john john 4096 Oct 24 12:45 new_linux
drwx------.  2 john john 4096 Oct 24 12:45 nature
drwx------.  2 john john 4096 Oct 24 12:45 musician
drwx------.  2 john john 4096 Oct 24 12:45 music
drwx------.  2 john john 4096 Oct 24 12:45 memory
drwx------.  2 john john 4096 Oct 24 12:45 mathematics
drwx------.  2 john john 4096 Oct 24 12:45 jobs
drwx------.  2 john john 4096 Oct 24 12:45 homeschoo
drwx------.  2 john john 4096 Oct 24 12:45 health
drwx------.  2 john john 4096 Oct 24 12:45 game
drwx------.  2 john john 4096 Oct 24 12:45 fun
drwx------.  2 john john 4096 Oct 24 12:45 food
drwx------.  2 john john 4096 Oct 24 12:45 export
drwxr-xr-x. 21 john john 4096 Oct 24 12:45 ai
drwxr-xr-x. 13 john john 4096 Oct 24 12:45 activism



the original source directory is here:
ls -altr 2015
total 208
-rwxr--r--.   1 john john   508 Mar 22  2015 reset.sh
drwxr-xr-x.   2 john john  4096 Jun  9 11:46 vr
drwxr-xr-x.   3 john john  4096 Jul 30 08:16 tech
drwxr-xr-x.   2 john john  4096 Aug 12 10:17 nature
drwxrwxr-x.   2 john john  4096 Aug 12 17:13 export
drwxr-xr-x.   7 john john  4096 Aug 12 18:42 mathematics
drwxr-xr-x.  40 john john  4096 Sep 13 01:00 c64
drwxr-xr-x.   2 john john  4096 Sep 18 17:29 music
drwxr-xr-x.  11 john john  4096 Sep 19 00:17 homeschoo
drwxr-xr-x.   6 john john  4096 Sep 19 05:06 chess
drwxr-xr-x.   3 john john  4096 Sep 21 09:30 jobs
drwxr-xr-x.   4 john john  4096 Sep 24 04:27 business
drwxr-xr-x.  21 john john  4096 Sep 26 09:53 ai
drwxr-xr-x.  19 john john  4096 Sep 30 12:49 musician
drwxr-xr-x.  34 john john  4096 Oct  8 13:40 game
drwxr-xr-x.  56 john john  4096 Oct  9 12:13 dev
drwxr-xr-x.   2 john john  4096 Oct 10 11:52 artwork
drwxr-xr-x.  14 john john  4096 Oct 14 20:59 spirit
drwxr-xr-x.   2 john john  4096 Oct 17 12:57 animals
drwxrwxr-x.  30 john john  4096 Oct 18 05:59 .
drwxr-xr-x.   4 john john  4096 Oct 22 11:05 fun
drwxr-xr-x.  49 john john 12288 Oct 22 11:07 memory
drwxr-xr-x.  45 john john 12288 Oct 22 11:20 computer
drwxr-xr-x.  45 john john 12288 Oct 22 11:31 receipt
drwxr-xr-x.   5 john john  4096 Oct 22 11:31 new_linux
drwxr-xr-x. 108 john john 36864 Oct 22 11:32 health
drwxr-xr-x.  15 john john  4096 Oct 23 20:16 reallife
drwxr-xr-x.  28 john john 16384 Oct 23 20:52 science
drwxr-xr-x.  13 john john 20480 Oct 23 20:59 activism
drwxr-xr-x.  20 john john  4096 Oct 24 12:41 food
drwxr-xr-x.   7 john john  4096 Oct 24 12:45 ..

-------------------------------------------------
//
//  debug on
//
//
///////////////////////////////////////////////////
[john at johnsplace Downloads]$ rsync -rav --debug 2015 delete_me/
FILE_STRUCT_LEN=24, EXTRA_LEN=4
cmd=<NULL> machine=<NULL> user=<NULL> path=.
cmd[0]=. cmd[1]=.
msg checking charset: UTF-8
[sender] safe_read(5)=4
(Client) Protocol versions: remote=31, negotiated=31
[Receiver] safe_read(0)=4
(Server) Protocol versions: remote=31, negotiated=31
[sender] safe_read(5)=1
[sender] safe_read(5)=4
sending incremental file list
rsync: change_dir "/home/john/Downloads//delete_me" failed: No such
file or
directory (2)
[sender] change_dir(/home/john/Downloads)
[sender] flist start=1, used=0, low=0, high=-1
send_file_list done
[sender] flist_eof=1
file list sent
send_files starting
server_recv(2) starting pid=10804
received 0 names
[Receiver] flist start=1, used=0, low=0, high=-1
recv_file_list done
get_local_name count=0 .
generator starting pid=10804
delta-transmission disabled for local transfer or --whole-file
recv_files(0) starting
[receiver] flist_eof=1
[generator] flist_eof=1
generate_files phase=1
send_files phase=1
recv_files phase=1
generate_files phase=2
send_files phase=2
send files finished
total: matches=0  hash_hits=0  false_alarms=0 data=0
recv_files phase=2
recv_files finished
generate_files phase=3
[receiver] send_msg(10, 8)
generate_files finished
client_run waiting on 10804

sent 20 bytes  received 436 bytes  912.00 bytes/sec
total size is 0  speedup is 0.00
[sender] _exit_cleanup(code=23, file=main.c, line=1165): entered
rsync error: some files/attrs were not transferred (see previous errors) (code
23) at main.c(1165) [sender=3.1.1]
[sender] _exit_cleanup(code=23, file=main.c, line=1165): about to call exit(23)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #1 from john lutz <lutz.john at yahoo.com> ---
also rsync -rav on external usb3 ext4 hard drive systems like ubuntu or debian
(and very much UNLIKE Fedora where it fails completely) sometimes lock to the
point that the mouse cursor locks. but after a while it frees up and locks
again only to completely the task after some time.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #2 from john lutz <lutz.john at yahoo.com> ---
Also. The directories with improper file permissions are EMPTY!!!

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #3 from Kevin Korb <rsync at sanitarium.net> ---
Instead of --debug (which should really have an additional parameter) try with
-vvii  Also, since you clearly have ACLs you probably want -A too.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #4 from john lutz <lutz.john at yahoo.com> ---
no! rsync -rav -vvii 2015 ./delme
results in no preservation of attributes. -a includes -A BTW!

[john at johnsplace Downloads]$ rsync -ravAvvii --debug 2015 delme

results in all directories and files having the current date/time.


akso I tried the following:

rsync -ravAvvii --debug /usr/lib/* delme
Unknown --debug item: "/usr/lib/abrt-java-connector"
rsync error: syntax or usage error (code 1) at options.c(450) [client=3.1.1]
[root at johnsplace 2015]#

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #5 from john lutz <lutz.john at yahoo.com> ---
the thing is. i been using rsync for 4 years and 'rsync -rav src dest'
always
worked for me under ubuntu and debian yet under Fedora it seems to be a
treasure trove of bugs and general disarray.

now don't get me wrong rsync will freeze and unfreeze and freeze event he
mouse
cursor ,etc al. a long running transfer under ubuntu and debian but it will
retain the ACLS and the transfer the proper files. while under Fedora it does 
not at all.

i am using a external usb3 hard drive with ext4. same exact hardware setup with
debian and ubuntu i've already mentioned.

john

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #6 from Kevin Korb <rsync at sanitarium.net>
---
> no! rsync -rav -vvii 2015 ./delme
> results in no preservation of attributes. -a includes -A BTW!
That makes no sense at all.  -vvii only modifies output.  -a includes -r but it
specifically does NOT include -A.  Here is the line from the man page:
    -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
> [john at johnsplace Downloads]$ rsync -ravAvvii --debug 2015 delme
> results in all directories and files having the current date/time.
Because you are using --debug wrong (see the next block)
> akso I tried the following:
> rsync -ravAvvii --debug /usr/lib/* delme
> Unknown --debug item: "/usr/lib/abrt-java-connector"
> rsync error: syntax or usage error (code 1) at options.c(450)
[client=3.1.1]
Again, this is not how you use --debug.  The --debug option takes a second
parameter and it is interpreting the first filename as that parameter.  Frankly
I am not sure why "--debug 2015" wasn't also causing this syntax
error.

When you run:
rsync -rav 2015 delete_me/
The 2015 parameter is what kind of debugging you want not a source parameter. 
All rsync is going to do is list the delete_me directory.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #7 from john lutz <lutz.john at yahoo.com> ---
okay, i tried

rsync -ravA --debug 9 /usr/lib/* delme

it was copying , still wrong ACL information and nothing discernable
that raised an error.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #8 from Kevin Korb <rsync at sanitarium.net> ---
Forget --debug.  The --debug option is for providing LESS (as in more specific)
debugging information than additional instances of -v.  The modifier option
isn't a number it is which debugging info you want it to show.  For instance
--debug ACL.


Try this:
rsync -avviiA /usr/lib/ delme/
(you should almost never have a * in the source parameter)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #10 from john lutz <lutz.john at yahoo.com> ---
..regarding previous... all empty directories. but i did control-c it.

again, when i used rsync -rav rav and waited for completion still got
empty directories.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #11 from Kevin Korb <rsync at sanitarium.net> ---
Permissions are set at the end.  If you ^C rsync then permissions won't be
set.
 Could you try my last suggested command with a new (empty) target and let it
finish?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

Wayne Davison <wayned at samba.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |INVALID

--- Comment #12 from Wayne Davison <wayned at samba.org> ---
This is the sort of discussion that would be a better fit on the mailing list:

https://rsync.samba.org/lists.html

As for the "--debug 2015" option, that is currently being taken as a
short-hand
for "--debug ALL2015" (level 2015 for all categories), which I will
change to
produce an error.

The wrong perms on the directories are indeed caused by you interrupting the
transfer.  You can avoid that early-creation idiom by either using -O
(--omit-dir-times) but ONLY in 3.1.1 or later, OR by specifying --no-recursive
in any 3.x version.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #13 from john lutz <lutz.john at yahoo.com> ---
this doesn't fix the fact that 'rsync -rav src dest' still
doesn't work
as intended on Fedora only.

so using -a and -A apparently fixes the problem. Even though -a should
specifically include -A .This would suggest that
either -r or -a doesn't have the proper code to transfer all file
information
correctly in rsync under Fedora only.

that debug stuff i'm not totally concerned with and had nothing to do
with fixing the problem. this smells like a bad block copy in rsync code
under Fedora only.

John

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #14 from john lutz <lutz.john at yahoo.com> ---
Also, if I stop a 'rsync -aiivvA src dest' with control-c it will
still copy *all* the correct ACL and file content data.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #15 from Kevin Korb <rsync at sanitarium.net> ---
rsync's -a specifically does not include -A or -X because these are not
supported by all versions of rsync, or on all platforms, or on all filesystems,
and they can be disabled at compile time.  If you specify -A or -X and either
end doesn't support them you get an error.

-H is also not included in -a but I personally would be happy if it were.

Again, from the man page:
    -a, --archive               archive mode; equals -rlptgoD (no -H,-A,-X)

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=11572

--- Comment #16 from john lutz <lutz.john at yahoo.com> ---
I had erroneously though that -A was attribute information (those that
are controlled by chmod) and even more erroneously thought -A was part of
-a.

anyway, i went to my Ubuntu configuration and 'rsync -rav src dest'
worked out of the box perfectly. I guess Fedora does something unfancy
with -A that Ubuntu does not.

I would gather -A is defaulted under Ubuntu and not Fedora. '

tune2fs -l /dev/mapper/fs' have both mounted 'acl' as default.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.