andrew.marlow at uk.bnpparibas.com
2010-May-24 12:04 UTC
rsync 3.0.7 intermittent failures with many large files
I have recently changed the version of rsync I use from the ancient 2.6.6 to 3.0.7. Every since then it seems to me that I am getting more rsync failures than before. Hopefully, other people can share their experiences and point to the cause which, I acknowledge, might be me doing something wrong. The rsync error long indicates things like: rsync: writefd_unbuffered failed to write 4092 bytes to socket [generator]: Connection reset by peer (104)rsync: read error: Connection reset by peer (104) rsync error: error in rsync protocol data stream (code 12) at io.c(1530) [generator=3.0.7] rsync error: error in rsync protocol data stream (code 12) at io.c(760) [receiver=3.0.7] I am using rsync to perform a distributed software release to around 200 machines. Several hundred files are transferred. Some are quite large executables. Each time it is a different set of machines that fail. Out of 200 it is 3 to 6 that fail. I have not had a run where all 200 work for quite some time. With the older version of rsync total success was the norm. We got a few failures such as the above every now and then. I am not sure it is to do with moving to a more recent version of rsync. It might be to do with a flakey network. It's hard to say. But I did see an rsync discussion thread at http://serverfault.com/questions/27137/rsync-or-dfs-r-for-remote-backup-over-slow-links-of-windows-2003-r2-sp2. This seems to be talking about the same kind of problems I have been having. The distribution is over a WAN, tranferring files from London to Geneva. I am using Windows-XP (SP2), with rsync built using the latest version of cygwin. I am also defining the environment variable CYGWIN to be NONTSEC to head off any ACL-related permission problems. The thread I refer to makes me think that it might actually be a problem with rsync. Maybe rsync should watch out for temporary loss of network connectivity or slowness like you get in WANs sometimes. Maybe it should tolerate these sorts of errors subject to a maximum number of retries. Regards, Andrew Marlow NOTE TO MODERATORS: I apologise for the length of the disclaimer that the emailer attaches. There is nothing I can do about it. Please feel free to delete it. --- ___________________________________________________________ This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is prohibited. Please refer to http://www.bnpparibas.co.uk/en/information/legal_information.asp?Code=ECAS-845C5H for additional disclosures. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20100524/e477bc7a/attachment.html>