rsync - Jun 2003 - various "rsync: connection unexpectedly closed" errors on debian unstable rsync 2.5.6-0.1 package

Hi.  I have a debian/unstable machine, which I use to rsync with other 
machines, and on which I run an rsync daemon.  rsync --version gives:

    rsync  version 2.5.6  protocol version 26
    Copyright (C) 1996-2002 by Andrew Tridgell and others
    <http://rsync.samba.org/>
    Capabilities: 64-bit files, socketpairs, hard links, symlinks,
    batchfiles,
                  IPv6, 64-bit system inums, 64-bit internal inums


Very frequently when rsync'ing with some machines, I get an error 
similar to the following:

    rsync: connection unexpectedly closed (1491993 bytes read so far)
    rsync error: error in rsync protocol data stream (code 12) at io.c(165)
    rsync: connection unexpectedly closed (1491973 bytes read so far)
    rsync error: error in rsync protocol data stream (code 12) at io.c(165)

In this particular case, I ran "rsync --recursive --links --hard-links 
--times --verbose --delete --timeout=600 
rsync.jp.gentoo.org::gentoo-portage/  
/home/mirrors/ftp_root/sites/gentoo/".  On the other hand, when I rsync 
against some sites such as rsync.au.gentoo.org, I rarely get such errors.

For my rsync daemon, I run it as "/usr/bin/nice /usr/bin/rsync --daemon 
--compress --safe-links --bwlimit=400 --timeout=600" where the 
appropriate stanza in /etc/rsyncd.conf is

    [gentoo-portage]
            comment = Gentoo Linux Portage tree
            path = /home/mirrors/ftp_root/sites/gentoo
            read only = yes
            use chroot = true
            list = yes
            uid = sync
            gid = sync

 From some places, rsyncs to myself get errors similar to the following:

    write failed on /tmp/mirstat.temp.K2gzyR : Bad address
    rsync error: error in file IO (code 11) at receiver.c(271)
    rsync: connection unexpectedly closed (798 bytes read so far)
    rsync error: error in rsync protocol data stream (code 12) at io.c(150)


I'd be very grateful if anyone can shed light on why this is happening.  
Is my version of rsync somehow incompatible with some versions?  Does 
the debian rsync package have known problems? (on the latter, I alien'd 
the redhat rawhide rsync 2.5.6-4 package, and got the exact same 
problems, so it may not be a debian package-specific issue).
 
Thanks in advance...

On Sat, Jun 21, 2003 at 09:21:11AM +0800, wearitdown@myrealbox.com
wrote:
> Hi.  I have a debian/unstable machine, which I use to rsync with other 
> machines, and on which I run an rsync daemon.  rsync --version gives:
> 
>    rsync  version 2.5.6  protocol version 26
>    Copyright (C) 1996-2002 by Andrew Tridgell and others
>    <http://rsync.samba.org/>
>    Capabilities: 64-bit files, socketpairs, hard links, symlinks,
>    batchfiles,
>                  IPv6, 64-bit system inums, 64-bit internal inums
> 
> 
> Very frequently when rsync'ing with some machines, I get an error 
> similar to the following:
> 
>    rsync: connection unexpectedly closed (1491993 bytes read so far)
>    rsync error: error in rsync protocol data stream (code 12) at io.c(165)
>    rsync: connection unexpectedly closed (1491973 bytes read so far)
>    rsync error: error in rsync protocol data stream (code 12) at io.c(165)
> 
> In this particular case, I ran "rsync --recursive --links --hard-links
> --times --verbose --delete --timeout=600 
> rsync.jp.gentoo.org::gentoo-portage/  
> /home/mirrors/ftp_root/sites/gentoo/".  On the other hand, when I
rsync
> against some sites such as rsync.au.gentoo.org, I rarely get such errors.
> 
> For my rsync daemon, I run it as "/usr/bin/nice /usr/bin/rsync
--daemon
> --compress --safe-links --bwlimit=400 --timeout=600" where the 
> appropriate stanza in /etc/rsyncd.conf is
> 
>    [gentoo-portage]
>            comment = Gentoo Linux Portage tree
>            path = /home/mirrors/ftp_root/sites/gentoo
>            read only = yes
>            use chroot = true
>            list = yes
>            uid = sync
>            gid = sync
> 
> From some places, rsyncs to myself get errors similar to the following:
> 
>    write failed on /tmp/mirstat.temp.K2gzyR : Bad address
>    rsync error: error in file IO (code 11) at receiver.c(271)
>    rsync: connection unexpectedly closed (798 bytes read so far)
>    rsync error: error in rsync protocol data stream (code 12) at io.c(150)
> 
> 
> I'd be very grateful if anyone can shed light on why this is happening.
> Is my version of rsync somehow incompatible with some versions?  Does 
> the debian rsync package have known problems? (on the latter, I alien'd
> the redhat rawhide rsync 2.5.6-4 package, and got the exact same 
> problems, so it may not be a debian package-specific issue).
Please check your rsync log file (created by the daemon) for
error messages.  Also try without compression.

It seems as though something happened in 2.5.6 where
compression and daemon produce this.  My impression is that
it doesn't happen via ssh nor without compression.  So far
we don't have a test case nor have any of the developers
experienced it.  You may want to try cvs where some of the
io routines have been changed.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Hi jw schultz,
>Please check your rsync log file (created by the daemon) for
>error messages.  Also try without compression.
>  
>
When I sync'ed against other servers, my rsync client process always ran 
without --compress.  Nevertheless, I've tried increasing verbosity by 3, 
and this is what I get:

    receiving file list ...
    recv_file_name(.)
    recv_file_name(net-zope)
    recv_file_name(net-zope/btreefolder2)
    [snip]
    recv_generator(x11-wm/xpde/xpde-0.2.0.ebuild,49890)
    x11-wm/xpde/xpde-0.2.0.ebuild is uptodate
    recv_generator(x11-wm/xpde/xpde-0.3.0.ebuild,49891)
    x11-wm/xpde/xpde-0.3.0.ebuild is uptodate
    generate_files phase=1
    recv_files(metadata/timestamp)
    recv mapped metadata/timestamp of size 29
    metadata/timestamp
    rsync: connection unexpectedly closed (1494175 bytes read so far)
    rsync error: error in rsync protocol data stream (code 12) at io.c(165)
    _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12)
    rsync: connection unexpectedly closed (1494155 bytes read so far)
    rsync error: error in rsync protocol data stream (code 12) at io.c(165)
    _exit_cleanup(code=12, file=io.c, line=165): about to call exit(12)

I'll disable compression on my rsync daemon, increase verbosity and see 
how that goes.
>It seems as though something happened in 2.5.6 where
>compression and daemon produce this.  My impression is that
>it doesn't happen via ssh nor without compression.  So far
>we don't have a test case nor have any of the developers
>experienced it.  You may want to try cvs where some of the
>io routines have been changed.
>
I've been running rsync as a client without --compress; is it still 
likely to be caused by compression?

I'll think about trying cvs to see if that helps; right now, I'm trying 
not to introduce too many variables at a time.