rsync - Mar 2008 - DO NOT REPLY [Bug 5299] New: 2.6.9 client cannot receive files from 3.0.0 server

https://bugzilla.samba.org/show_bug.cgi?id=5299

           Summary: 2.6.9 client cannot receive files from 3.0.0 server
           Product: rsync
           Version: 3.0.0
          Platform: x86
        OS/Version: Windows XP
            Status: NEW
          Severity: major
          Priority: P3
         Component: core
        AssignedTo: wayned@samba.org
        ReportedBy: ruralrob@gmail.com
         QAContact: rsync-qa@samba.org


My client is a Windows machine running rsync 2.6.9 under Cygwin. The server is
a Debian system that, until today, was also running rsync 2.6.9. Every day on
the Windows machine I would issue a command something like the following:

rsync -tzvr --delete rob@xx.xx.xx.xx:sourcepath destpath

to sync some files from the Linux machine to the Windows machine. This worked
fine until today, when I upgraded rsync on the server to 3.0.0. (The newer
version is not available for Cygwin yet). Now when I try to do the above
operation, I get:

Invalid block length 146536 [sender]
rsync error: protocol incompatibility (code 2) at io.c(1324) [sender=3.0.0]
rsync: connection unexpectedly closed (639 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at
/home/lapo/packaging/tmp/rsync-2.6.9/io.c(453) [receiver=2.6.9]
rsync: connection unexpectedly closed (639 bytes received so far) [generator]
rsync error: error in rsync protocol data stream (code 12) at
/home/lapo/packaging/tmp/rsync-2.6.9/io.c(453) [generator=2.6.9]

Shouldn't a 2.6.9 client still be able to copy files from a 3.0.0 server
(with
the server falling back to the older protocol)?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299





------- Comment #1 from wayned@samba.org  2008-03-03 01:03 CST -------
It works fine in all the testing I've done on Linux systems, so you should
do
some more investigation to see if a particular option is causing Cygwin
problems (perhaps -z?) or perhaps a certain file?


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299





------- Comment #2 from ruralrob@gmail.com  2008-03-03 15:37 CST -------
I booted my Sidux LiveCD on the Windows system, and ran rsync 2.6.9 to fetch
the same set of files, and got the same error messages as mentioned above. So I
don't think it has anything to do with Cygwin.

The files I'm copying are many gigabytes each; I tried some smaller files
(in
the 2-5Mb range) and they went across fine. So this seems to only happen with
really big files. I'll keep experimenting and see if I can find a definite
cutoff size. (Something tells me it's gonna be the infamous 2Gb or 4Gb.)

Also, while still booted into Sidux, I upgraded to rsync 3.0.0 and tried my
original file transfer again. This fixed the problem. So hurry up with the
Cygwin port of 3.0.0!  :-D


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299


wayned@samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED




------- Comment #3 from wayned@samba.org  2008-03-16 22:15 CST -------
So far I haven't had any luck trying to reproduce this.  If you have a set
of
files that makes this reproducible and you can point me at them for me to
download and test, please email me.  I will also go over the protocol-29 code
to see if I made any unintended changes.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299





------- Comment #4 from ruralrob@gmail.com  2008-03-17 11:15 CST -------
I'm still working on coming up with a set of files that I can share with you
that will reproduce the problem. So far I have only seen this happen with my
VMware files for my work development environment. The company I work for would
be most unhappy with me letting that out in the wild.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299


cwallace@lodgingcompany.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |cwallace@lodgingcompany.com




------- Comment #5 from cwallace@lodgingcompany.com  2008-05-04 21:29 CST
-------
I'm experiencing the same problem when copying from a 3.0.0 rsync client to
a
2.6.9 server.

---
janus:/var/log/asterisk# rsync --bwlimit=1000 -v --progress --protocol=29 full
neptune::janus/var/log/asterisk/
Invalid block length 132328 [sender]
rsync error: protocol incompatibility (code 2) at io.c(1324) [sender=3.0.0]
---

I've narrowed it down to that one file (/var/log/asterisk/full), which is
only
23MB but was once (and still is on the destination) 17GB.  I suspect if I
delete or truncate the 17GB copy on the destination, the problem will not
occur. 

(...trying it out...brb...)

Bingo!

---
janus:/var/log/asterisk# rsync --bwlimit=1000 -v --progress --protocol=29 full
neptune::janus/var/log/asterisk/
full
    24391536 100% 1001.09kB/s    0:00:23 (xfer#1, to-check=0/1)

sent 24394593 bytes  received 38 bytes  995699.22 bytes/sec
total size is 24391536  speedup is 1.00
---

Well, now that I'm compressing my log files, I won't have any more that
are
17GB... but I thought I'd share my experience with you guys anyway... 
Unfortunately, that file (/var/log/asterisk/full) contains a lot of sensitive
information, so I can't share it.  I'll try to reproduce it later by
creating
large files from /dev/zero on the destination.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299


remy.blank@pobox.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |remy.blank@pobox.com




------- Comment #6 from remy.blank@pobox.com  2008-06-23 02:40 CST -------
I have been bitten by this issue as well, when transferring a single big file:

  -rw-r-----  1 root backup 21474836480 2008-03-24 16:59 morpheus.athome.vol

That is, it's 20GB big. I am copying from a 3.0.2 client to a 2.6.6 server.
The
command line I use is the following:

  rsync -av --ignore-times --inplace -e "ssh -i /root/.ssh/id_backup"
--stats
/home/mirror/morpheus.athome.vol rsync:backup/morpheus.athome.vol

And this gives the following error:

  Invalid block length 146536 [sender]
  rsync error: protocol incompatibility (code 2) at io.c(1367) [sender=3.0.2]

If I fix the block size to 128kB by adding "-B 131072", the transfer
succeeds.
So it seems that older rsync versions don't cope with block sizes over
128kB,
which means a file size limit of 16GB.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=5299


wayned@samba.org changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED
            Version|3.0.0                       |3.0.3




------- Comment #7 from wayned@samba.org  2008-06-24 01:21 CST -------
This is now fixed in the latest code -- rsync 3 will use/allow a block size
that is compatible with an older rsync when using protocol 29 or lower.  (The
3.0.3pre3 release contains this fix.)

Thanks for all the help getting the cause sorted out.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.