rsync - Jan 2005 - buffer overflow in recv_exclude_list using rsync under windows?

Hi,

First post to the list, so please feel free to set me straight if I'm not
following some protocol or other :o)

We need to use rsync to send files to a client, being a Windows user, we
decided to try both the cwrsync implementation and also a straight
cygwin/rsync install. I'm experiencing the following errors:

using cygwin implementation:

$ sh clientupload.sh /cygdrive/c/batch1/ready/
Sending files...
Sending /cygdrive/c/batch1/ready/:
building file list ...
25 files to consider
ERROR: buffer overflow in recv_exclude_list
rsync error: error allocating core memory buffers (code 22) at util.c(121)
rsync: read error: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at
/home/lapo/packaging/tmp/rsync-2.6.3/io.c(515)

using cwrsync:

C:\cwrsync>rsync --rsh="ssh" --recursive --times --partial
--exclude=".*" --
verbose --progress --stats /cygdrive/c/batch1/ready/*
server-up:content/client/
building file list ...
24 files to consider
ERROR: buffer overflow in recv_exclude_list
rsync error: error allocating core memory buffers (code 22) at util.c(121)
rsync: read error: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(515)

Some other attempts at getting some sensible error messages:

C:\cwrsync>rsync --rsh="ssh" --verbose --progress --stats --dry-run
/cygdrive/c/batch1/ready/*  server-up:content/client/
skipping directory /cygdrive/c/batch1/ready/CHR_BAH25.itmsp
skipping directory /cygdrive/c/batch1/ready/CHR_CDANA53.itmsp
ERROR: buffer overflow in recv_exclude_list
rsync error: error allocating core memory buffers (code 22) at util.c(121)
rsync: read error: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(515)

C:\cwrsync>rsync --rsh="ssh" --verbose --progress --stats --dry-run
/cygdrive/c/batch1/ready/*  server-up:content/client/
skipping directory /cygdrive/c/batch1/ready/.
Invalid file index 1095567365 (count=0)
rsync error: protocol incompatibility (code 2) at sender.c(152)

I've spent many happy hours googling and searching, as well as modifying the
options, but am at a complete loss as to where the problem might lie,
whether it's client side, server side or some linux/windows incompatibility.

We desperately need to find a solution, so any advice would be gratefully
received.

Thanks in advance

Murray
--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.6.8 - Release Date: 03/01/2005

I was advised to re-run using the  -vv option and minimal other options, so
here is a report of what came back, if it helps:

C:\cwrsync>rsync --rsh="ssh" --verbose --progress --stats -vv
--recursive --
dry
run /cygdrive/c/batch1/ready/*  server-up:content/client/
opening connection using ssh server-up rsync --server -vvvnr .
content/client

building file list ...
[sender] make_file(CHR_BAH25.itmsp,*,2)
[sender] expand file_list to 131072 bytes, did move
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track01.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track02.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track03.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track04.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track05.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track06.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track07.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track08.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track09.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track10.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track11.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track12.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/CHR_16804552_track13.m4a,*,2)
[sender] make_file(CHR_BAH25.itmsp/cover.jpg,*,2)
[sender] make_file(CHR_BAH25.itmsp/metadata.xml,*,2)
[sender] popping per-dir .cvsignore exclude list
[sender] make_file(CHR_CDANA53.itmsp,*,2)
[sender] make_file(CHR_CDANA53.itmsp/CHR_1537673_track01.m4a,*,2)
[sender] make_file(CHR_CDANA53.itmsp/CHR_1537673_track02.m4a,*,2)
[sender] make_file(CHR_CDANA53.itmsp/CHR_1537673_track03.m4a,*,2)
[sender] make_file(CHR_CDANA53.itmsp/CHR_1537673_track04.m4a,*,2)
[sender] make_file(CHR_CDANA53.itmsp/cover.jpg,*,2)
[sender] make_file(CHR_CDANA53.itmsp/metadata.xml,*,2)
[sender] popping per-dir .cvsignore exclude list
[sender] make_file(listing.txt,*,2)
24 files to consider
send_file_list done
send_files starting
ERROR: buffer overflow in recv_exclude_list
rsync error: error allocating core memory buffers (code 22) at util.c(121)
rsync: read error: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(515)
_exit_cleanup(code=12, file=io.c, line=515): about to call exit(12)


--
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.300 / Virus Database: 265.6.8 - Release Date: 03/01/2005

On Wed, Jan 05, 2005 at 11:16:34AM -0000, Murray Elliot
wrote:
> ERROR: buffer overflow in recv_exclude_list
It looks like some kind of corruption is going on over your ssh channel.
One way to investigate this would be to make a copy of the data that is
being sent and is being received.  I whipped up a C program to do this,
which I have attached to this message.

The program has a kluge in because it does not properly handle the EOF
condition when rsync is all done (I just made it timeout at 30 seconds
of inactivity and call it quits).  If you put the file cacheit.c into
your rsync directory (so it can use rsync.h and config.h), you can type
"make cacheit" and then put the resulting program into /tmp on each
system (or whereever else you like).  You should then be able to run
a command like this one:

rsync -av --rsh='/tmp/cacheit /tmp/rsync-sent ssh' [OPTIONS...]
    --rsync-path='/tmp/cacheit /tmp/rsync-got rsync' /from/ host:/to/

That will make a copy of the sent data in /tmp/rsync-sent on the sending
host, and a copy of the received data in /tmp/rsync-got on the receiving
host.  If the files are not identical, then there is a problem with your
transport/network.  If the files are identical (and rsync fails), feel
free to send me one of them.

..wayne..
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cacheit.c
Type: text/x-csrc
Size: 2417 bytes
Desc: not available
Url :
http://lists.samba.org/archive/rsync/attachments/20050105/42fd2127/cacheit.bin