Doug Lochart
2006-May-18 14:58 UTC
rsync client to daemon over ssh tunnel connection issues
Yes this is another connection reset issue but all I am asking is some pointers on how I can possibly determine what and why this is happening. Its easy to say its a network issue but the bosses and clients want to know how to fix this said "network issue." I have read the rsync issues list. Some of what is suggested we might try HOWEVER we cannot duplicate the errors that occur and those tactics seem driven towards reproducable errors. It is not consistent, sometimes the rsync transfer works flawlesly sometimes if we wipe the server module clean and restart it will fail. We wipe and restart it will fail somewhere else down the line and then it might work all the way through. I would put the strace on it if we could recreate the error. I did come upon an rsync hang 2 days ago where the client had aborted but the server still had rsync running and I believe the server side of the sshd tunnel running. I installed strace (Centos did not have it by default). The ssh process terminated but the rsync was still going. So I ran strace and it was in a select call. This tells me that something happened between the endpoints of the ssh tunnel for this issue. Hwat follows are different issues. Client: Windows XP running CWRsync 2.6.8. We are creating an SSH Tunnel from the client machine (external network) to our server Server: running rsync 2.6.8 using rsync --daemon. No chroot jails or the like. This Happened Last Night: ------------------------------------- 1) Client log showed this: Read from remote host 209.225.128.138: Connection reset by peer^M^M rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown" [sender]: Connection reset by peer (104)^M rsync: read error: Connection reset by peer (104)^M rsync error: error in rsync protocol data stream (code 12) at io.c(622) [sender]^M Server showed this: rsync: connection unexpectedly closed (148817310 bytes received so far) [receiver] 2006/05/17 22:23:24 [8706] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8] 2006/05/17 22:23:24 [8706] rsync: writefd_unbuffered failed to write 93 bytes [generator]: Broken pipe (32) 2006/05/17 22:23:24 [8706] rsync error: error in rsync protocol data stream (code 12) at io.c(1119) [generator=2.6.8] Is this a spurious network error or could it be something else. The processes involved with this error have since terminated so this is all I have at the moment Anybody have a clue as to what I can do to research this? We can't control (ie install diagnostic software) the client pc much as its not ours. Of course ALL tests (internal client) and all but 1 test (external client) worked before doing large sets of data (up to 10 gig) 2) Client Log Shows This: unexpected tag 3 [sender]^M rsync error: error in rsync protocol data stream (code 12) at io.c(836) [sender]^M Terminating secure channel ...^M^M Server side log showed this: sync: connection unexpectedly closed (235695 bytes received so far) [receiver] 2006/05/17 23:01:49 [8782] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8] 2006/05/17 23:01:49 [8782] rsync: writefd_unbuffered failed to write 93 bytes [generator]: Broken pipe (32) 2006/05/17 23:01:49 [8782] rsync error: error in rsync protocol data stream (code 12) at io.c(1119) [generator=2.6.8] Now this error (unexpected tag 3) I found a post on 05-03 and response from Wayne with a small patch to io.c. My only question is I can apply the patch or get the nightly tar for the server but we are using CwRsync on the client. Does this patch have to be applied on the client side as well? The two errors above were coupled with 4 successful runs from the same machines. Each machine had 1 failure and 4 out of 6 rsync transfers worked. Thanks again, Douglas Lochart Cape Computing Corp -- What profits a man if he gains the whole world yet loses his soul? -------------- next part -------------- HTML attachment scrubbed and removed
Tevfik Karagülle
2006-May-18 15:39 UTC
rsync client to daemon over ssh tunnel connection issues
Hi, There is a bug in rsync 2.6.8 causing this. Links below can give you some abckground info: http://www.mail-archive.com/rsync@lists.samba.org/msg16205.html http://www.itefix.no/phpws/index.php?module=phpwsbb <http://www.itefix.no/phpws/index.php?module=phpwsbb&PHPWSBB_MAN_OP=view&PHP WS_MAN_ITEMS[]=404> &PHPWSBB_MAN_OP=view&PHPWS_MAN_ITEMS[]=404 I have recently published an patch package available from http://prdownloads.sourceforge.net/sereds/unexpected-tag-3-patch.zip Give a try !! Best rgrds Tev cwRsync maintainer http://itefix.no/cwrsync _____ From: rsync-bounces+tevfik=itefix.no@lists.samba.org [mailto:rsync-bounces+tevfik=itefix.no@lists.samba.org] On Behalf Of Doug Lochart Sent: Thursday, May 18, 2006 4:58 PM To: rsync@lists.samba.org Cc: Doug Lochart Subject: rsync client to daemon over ssh tunnel connection issues Yes this is another connection reset issue but all I am asking is some pointers on how I can possibly determine what and why this is happening. Its easy to say its a network issue but the bosses and clients want to know how to fix this said "network issue." I have read the rsync issues list. Some of what is suggested we might try HOWEVER we cannot duplicate the errors that occur and those tactics seem driven towards reproducable errors. It is not consistent, sometimes the rsync transfer works flawlesly sometimes if we wipe the server module clean and restart it will fail. We wipe and restart it will fail somewhere else down the line and then it might work all the way through. I would put the strace on it if we could recreate the error. I did come upon an rsync hang 2 days ago where the client had aborted but the server still had rsync running and I believe the server side of the sshd tunnel running. I installed strace (Centos did not have it by default). The ssh process terminated but the rsync was still going. So I ran strace and it was in a select call. This tells me that something happened between the endpoints of the ssh tunnel for this issue. Hwat follows are different issues. Client: Windows XP running CWRsync 2.6.8. We are creating an SSH Tunnel from the client machine (external network) to our server Server: running rsync 2.6.8 using rsync --daemon. No chroot jails or the like. This Happened Last Night: ------------------------------------- 1) Client log showed this: Read from remote host 209.225.128.138: Connection reset by peer^M^M rsync: writefd_unbuffered failed to write 4 bytes: phase "unknown" [sender]: Connection reset by peer (104)^M rsync: read error: Connection reset by peer (104)^M rsync error: error in rsync protocol data stream (code 12) at io.c(622) [sender]^M Server showed this: rsync: connection unexpectedly closed (148817310 bytes received so far) [receiver] 2006/05/17 22:23:24 [8706] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8] 2006/05/17 22:23:24 [8706] rsync: writefd_unbuffered failed to write 93 bytes [generator]: Broken pipe (32) 2006/05/17 22:23:24 [8706] rsync error: error in rsync protocol data stream (code 12) at io.c(1119) [generator=2.6.8] Is this a spurious network error or could it be something else. The processes involved with this error have since terminated so this is all I have at the moment Anybody have a clue as to what I can do to research this? We can't control (ie install diagnostic software) the client pc much as its not ours. Of course ALL tests (internal client) and all but 1 test (external client) worked before doing large sets of data (up to 10 gig) 2) Client Log Shows This: unexpected tag 3 [sender]^M rsync error: error in rsync protocol data stream (code 12) at io.c(836) [sender]^M Terminating secure channel ...^M^M Server side log showed this: sync: connection unexpectedly closed (235695 bytes received so far) [receiver] 2006/05/17 23:01:49 [8782] rsync error: error in rsync protocol data stream (code 12) at io.c(463) [receiver=2.6.8] 2006/05/17 23:01:49 [8782] rsync: writefd_unbuffered failed to write 93 bytes [generator]: Broken pipe (32) 2006/05/17 23:01:49 [8782] rsync error: error in rsync protocol data stream (code 12) at io.c(1119) [generator=2.6.8] Now this error (unexpected tag 3) I found a post on 05-03 and response from Wayne with a small patch to io.c. My only question is I can apply the patch or get the nightly tar for the server but we are using CwRsync on the client. Does this patch have to be applied on the client side as well? The two errors above were coupled with 4 successful runs from the same machines. Each machine had 1 failure and 4 out of 6 rsync transfers worked. Thanks again, Douglas Lochart Cape Computing Corp -- What profits a man if he gains the whole world yet loses his soul? -------------- next part -------------- HTML attachment scrubbed and removed
Wayne Davison
2006-May-18 17:40 UTC
rsync client to daemon over ssh tunnel connection issues
On Thu, May 18, 2006 at 02:57:50PM +0000, Doug Lochart wrote:> I would put the strace on it if we could recreate the error.There's no need when you have the daemon's log-file telling you why the remote rsync exited. In the first case, both sides complain about the connection going away, so that closure was out of rsync's hands (and diagnosing rsync can tell you nothing more since something else failed). The "unexpected tag 3" error is something that is already fixed in CVS, but can be manually patched into the 2.6.8 release if you'd prefer to take the more cautious route than using the CVS version (there is a link to the NEWS file of the CVS version right on the main webpage for rsync that will tell you what has changed in the CVS version). I'm about to post a slightly updated patch for 2.6.8 that will make the patch correspond with the final fix that ended up in CVS. ..wayne..