Hello all! I've noticed that rsync performs significantly worse than wget on slow congested wireless links (GPRS in my case). I don't have large statistics, but in my tests rsync often stalls for 3-5 minutes, while wget stalls only for several seconds and then continues download. Is there any rsync protocol features which are sensitive to packet loss and unpredictable delay? I understand that rsync transaction has additional metadata transfer in the beginning (file list etc.), but after that what can affect the speed? Are there additional handshakes or flow control other than tcp ones? My rsync command looks like this: rsync -rtmz --partial --progress --delete-delay --delay-updates --stats --itemize-changes --timeout 120 --contimeout 20 source.example.com::module/file.zip /tmp/ I've tried to use it directly over ppp or via openvpn udp tunnel. Is there anything that can be done to improve situation? Setting sockopts or block size? Fine tcp tuning? If required, I can collect tcpdump data from both sides (but in short there are lots of TCP duplicate ACKs and retransmissions). Thanks
On Thu, 2009-12-10 at 13:08 +0700, Max Arnold wrote:> I've noticed that rsync performs significantly worse than wget on slow congested wireless > links (GPRS in my case). I don't have large statistics, but in my tests rsync often stalls > for 3-5 minutes, while wget stalls only for several seconds and then continues download. > > Is there any rsync protocol features which are sensitive to packet loss and unpredictable delay? > I understand that rsync transaction has additional metadata transfer in the beginning (file > list etc.), but after that what can affect the speed? Are there additional handshakes or > flow control other than tcp ones?> rsync -rtmz --partial --progress --delete-delay --delay-updates --stats --itemize-changes > --timeout 120 --contimeout 20 source.example.com::module/file.zip /tmp/Rsync isn't doing anything fancy that would cause it to be especially affected by packet loss or delay. The protocol takes a few round trips to set up, and then it is largely pipelined, so rsync can tolerate some amount of latency without slowing down the whole process. I can't explain why rsync would stall much longer than wget. The only thought I had is that the network might have a quality-of-service policy that favors port 80. -- Matt
On Wed, Dec 9, 2009 at 10:08 PM, Max Arnold <lwarxx at gmail.com> wrote:> in my tests rsync often stalls > for 3-5 minutes, while wget stalls only for several seconds and then continues download.Have you verified that rsync is actually stalled, and not simply sending incremental file-list information in between files? (e.g. strace rsync and see what it's doing.) You could try using --no-inc-recursive to see if doing all the file-list changes up front makes any difference. ..wayne..