I am doing some benchmarking of rsync. I am using the --bwlimit= option to throttle down rsync to predict its operation over slow communications links. I am using rsync 2.6.2 from the release site without any patches. I downloaded the release rather than pull from the CVS tree. I have 2 servers "wilber" (the remote archive) and "judy" (the local archive) connected with a gig ethernet. I have a file on "judy" that if I use the following command completes in under 1 second: time rsync -ar --rsh=rsh bluesAlbums/Pilgrim/track1.mp3 wilber://test/bluesAlbums/Pilgrim the track1 file on wilber exists and the track1 file on judy has been touched. The file Track1 is 6.3Meg in size. The checksums file if you do --write-batch is 60K bytes. The difference file is 40k bytes (no differences). I check the modification time on wilber after each transfer to make sure the transfer actually happened. If I use the command time rsync -ar -rsh=rsh --bwlimit=4001 bluesAlbums/Pilgrim/track1.mp3 wilber://test/bluesAlbums/Pilgrim real = 0.70 to 0.90 If I use the command time rsync -ar -rsh=rsh --bwlimit=4000 bluesAlbums/Pilgrim/track1.mp3 wilber://test/bluesAlbums/Pilgrim real = 1m34.000 to 1m35.000 There are no other processes running on either server, and I touch the file on judy each time I repeat the test. I actually have 2 telnet sessions open on judy and in one I repeat the touch and in the other I repeat the command string after I change the --bwlimit= option. I do this to make sure I don't fat finger things. I can repeat this time after time. If --bwlimit is > 4000 (ie. 4005, 4025, 4050,5000,7500,10000,100000) real is in the same range as 4001. If --bwlimit is 4000 or under (ie. 3725, 2000, 1000, 100) real is in the same range as 4000. I can understand bipolar behavior at extremes of bandwidth availability but I cant understand the cutoff being that acute that a 1k difference would yield such a dramatic result. At unlimited bandwidth, the real time is twice what it is for rcp so I believe the 0.70 to 0.90 is correct. ??? Is there something going on with --bwlimits around the value of 4000 that could be causing this sharp break.???? Wally
Since --bwlimit depends upon sleep(1 second), I repeated the experiment with a file that was 383 Megabyte so that when I am running unthrottled it takes significantly longer than a second (ie. ~50 seconds) to complete. I get the same bi-modal behavior but with different values for 4000 and 4001 respectively. The fact that the break point stays fixed isnt intuitive (to me at least). wally
On Fri, May 21, 2004 at 02:48:12PM -0400, Wallace Matthews wrote:> I can repeat this time after time. If --bwlimit is > 4000 (ie. 4005, > 4025, 4050,5000,7500,10000,100000) real is in the same range as 4001. > If --bwlimit is 4000 or under (ie. 3725, 2000, 1000, 100) real is in > the same range as 4000.That is because of this calculation: tv.tv_usec = bytes_written * 1000 / bwlimit; Rsync calls this function after a lot of 4-byte writes, and thus the sleep time for "4 * 1000 / 4001" (or anything higher than 4001) is 0. Thus, rsync neglects a bunch of sleep calls (but not all of them). I'm looking into some of the old bwlimit patches to see about improving this. ..wayne..