Hi Folks, For some time, we've been having some issues with our mirroring with rsync. The symptoms are a broken transfer, with the 'cryptic' error message: rsync: connection unexpectedly closed (1128806 bytes read so far) (the number of bytes changes). I've gone a number of hours searching with Google, and have heard a variety of things. Some messages say that they see this behavior due to using certain combinations of client and server. Others say they only see this when using EXT3 FS (but not with Reiser, XFS or EXT2). I'm wondering if anybody hear has any insight into the problem? And more importantly, how we fix it? We're using the latest release of Rsync (2.6.2) built from source. I can't find the ./configure line right this second - but will post it if I can find it. Any/all help greatly appreciated. Regards, Dan Goodes : Systems Programmer : dang@planetmirror.com Help support PlanetMirror - Australia's largest Internet archive by signing up for PlanetMirror Premium : http://planetmirror.com
Hi Again, On Thu, 27 May 2004 at 09:52, Dan Goodes wrote:> Hi Folks, > > For some time, we've been having some issues with our mirroring with > rsync. The symptoms are a broken transfer, with the 'cryptic' error > message: > > rsync: connection unexpectedly closed (1128806 bytes read so far)Following the feedback I've gotten so far - it appears that the problem is some incompatibilities between different versions of rsync. It also appears that the problem is that the server end is 'going away' before the rsync has finished (perhaps due to a segfault). Here are a couple of examples of rsync command-lines for which this happens regularly. /usr/bin/rsync -rltvH --stats --delete download.fedora.redhat.com::fedora-linux-core-updates/ /pub/fedora/linux/core/updates/ /usr/bin/rsync -rltvH --stats --delete --max-delete=500 mirror.caosity.org::cAos/ /ftp/pub/caosity/ You get the idea. Obviously without access to the server logs on these machines, I can't tell what exactly happened at the precise time that the rsync failed. Any further assistance would be apprecaited. Thanks -Dan
On Thu, May 27, 2004 at 09:52:30AM +1000, Dan Goodes wrote:> rsync. The symptoms are a broken transfer, with the 'cryptic' error > message: > rsync: connection unexpectedly closed (1128806 bytes read so far)This may mean that the remote rsync crashed, but it may also mean that the connection went down beyond rsync's control. To differentiate what's happening you'll need to somehow gather extra data for what happened on the remote system: - Enable crash-dump core production on the remote machine so that a crashed rsync produces a core file. - Attach a debugger to the remote rsync while it's running so that you can catch its exit condition. - Have someone look through the log files to see if the OS killed your process due to something such as too-much memory use. Without such extra information it is impossible to know what is going on. However, one other sleuthing idea just occurred to me: try using a --bwlimit value to see if slowing rsync down might keep the connection open for the whole transfer (some folks have reported that rsync revealed that their network drivers/hardware were deficient in some way, and their sporadic failure was taking down the connection). ..wayne..