Keith Christian
2012-Mar-07 20:01 UTC
Long delays in rsync manifested by repeated entries, CentOS, rsync v2.6.8.
Hello, rsync list folks, Recently, rsyncs abort during busier times of the day, although they run error-free during non-busy hours. Problem 1 - Many rsyncs abort these errors: Read from remote host www.xxx.yyy.zzz: Connection reset by peer rsync: writefd_unbuffered failed to write 4 bytes [sender]: Broken pipe (32) rsync: connection unexpectedly closed (3929920 bytes received so far) [sender] rsync error: unexplained error (code 255) at io.c(463) [sender=2.6.8] ---or--- rsync: read errors mapping "/home/someuser/.mail": No data available (61) The only solution I've seen to the above errors was to upgrade to rsync 2.6.8, but that's the version we're running, and, this configuration ran flawlessly for several years, without any changes to the servers. These abort problems started in the recent past. Problem 2 - With the logging level turned up with the "-vvv" flag, I notice some filenames are repeated in the output, sometimes for several minutes. The files are not that large, perhaps 500k max. What would cause this? The source and target machines are connected by a high speed link, most files are copied in a few seconds. The example below was created by tailing the last three lines of a log file in a 5 second delay loop, to watch for "stuck" file transfers. Wed Feb 29 11:28:32 MST 2012 recv_file_name(keith001/.bash_profile) recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 11:28:37 MST 2012 recv_file_name(keith001/.bash_profile) recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 11:28:42 MST 2012 recv_file_name(keith001/.bash_profile) recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 11:28:47 MST 2012 recv_file_name(keith001/.bash_profile) recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 11:28:52 MST 2012 recv_file_name(keith001/.bash_profile) recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 11:28:57 MST 2012 recv_file_name(keith001/.bash_profile) recv_file_name(keith001/.bashrc) received 352427 names Any suggestions appreciated. Thanks. ======Keith
Kevin Korb
2012-Mar-07 20:09 UTC
Long delays in rsync manifested by repeated entries, CentOS, rsync v2.6.8.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 You didn't give an example of your command line so I am going to assume you are running rsync over ssh. The first part of your first problem generally indicates that the ssh connection timed out while rsync was busy on the disk and not using the network stream. On modern rsync this tends to happen on really large directories or when using --checksum. On ancient rsync it was fairly common for it to happen right in the beginning. The rest of your problems are things that have likely been solved in the 6 years since rsync-2.6.8. It is unlikely anyone will be willing to devote significant time to help you isolate those problems unless you can duplicate them on a modern version considering that the fix would only apply to a new version anyway. On 03/07/12 15:01, Keith Christian wrote:> Hello, rsync list folks, > > Recently, rsyncs abort during busier times of the day, although > they run error-free during non-busy hours. > > Problem 1 - Many rsyncs abort these errors: Read from remote host > www.xxx.yyy.zzz: Connection reset by peer rsync: writefd_unbuffered > failed to write 4 bytes [sender]: Broken pipe (32) rsync: > connection unexpectedly closed (3929920 bytes received so far) > [sender] rsync error: unexplained error (code 255) at io.c(463) > [sender=2.6.8] ---or--- rsync: read errors mapping > "/home/someuser/.mail": No data available (61) > > > The only solution I've seen to the above errors was to upgrade to > rsync 2.6.8, but that's the version we're running, and, this > configuration ran flawlessly for several years, without any changes > to the servers. These abort problems started in the recent past. > > > Problem 2 - With the logging level turned up with the "-vvv" flag, > I notice some filenames are repeated in the output, sometimes for > several minutes. The files are not that large, perhaps 500k max. > What would cause this? The source and target machines are > connected by a high speed link, most files are copied in a few > seconds. > > The example below was created by tailing the last three lines of a > log file in a 5 second delay loop, to watch for "stuck" file > transfers. > > Wed Feb 29 11:28:32 MST 2012 > recv_file_name(keith001/.bash_profile) > recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 > 11:28:37 MST 2012 recv_file_name(keith001/.bash_profile) > recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 > 11:28:42 MST 2012 recv_file_name(keith001/.bash_profile) > recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 > 11:28:47 MST 2012 recv_file_name(keith001/.bash_profile) > recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 > 11:28:52 MST 2012 recv_file_name(keith001/.bash_profile) > recv_file_name(keith001/.bashrc) received 352427 names Wed Feb 29 > 11:28:57 MST 2012 recv_file_name(keith001/.bash_profile) > recv_file_name(keith001/.bashrc) received 352427 names > > > > Any suggestions appreciated. > > Thanks. > > > ======Keith- -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.17 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk9XwGAACgkQVKC1jlbQAQeWlACg3+90/rnqHGJ2YQDZaInwYAJo d/8AoNT251rMDGvCBwJC9wr+/BIE6mZe =2T/A -----END PGP SIGNATURE-----