Has anyone noticed that the --bwlimit doesn't really work? I have MRTG stats happening, and on a 128Kbps circuit, an rsync with the following syntax takes up the whole line. $ rsync -avz -e ssh localfiles.tar.gz user@remote:/path/to/file/arch Any ideas? It's on an OpenBSD box going to a FreeBSD box. Both are 2.4.6. -#0
On 19 Nov 2001, uid0@catastrophe.net wrote:> Has anyone noticed that the --bwlimit doesn't really work? > > I have MRTG stats happening, and on a 128Kbps circuit, an rsync > with the following syntax takes up the whole line. > > $ rsync -avz -e ssh localfiles.tar.gz user@remote:/path/to/file/arch > > Any ideas? It's on an OpenBSD box going to a FreeBSD box. Both are > 2.4.6.--bwlimit only works if you use it... -- Martin
Try this commandline:
rsync -avz -e ssh --bwlimit=64 localfiles.tar.gz
user@remote:/path/to/file/arch
That should average out to use only about 1/2 of your bandwidth. You will
find periods where it uses all of it, and periods where it uses none, but
on average, it will use only as many kbps as you permit it on the cmdline
.I notice that your example didn't actually use the --bwlimit parameter.
rsync doesn't do any kind of adaptive logic to determine how much
bandwidth it should use... it just takes everything it can, either up to
actual bandwidth limits, the bwlimit parameter, or how fast it can put
together and recieve the stream.
Tim Conway
tim.conway@philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
uid0@catastrophe.net
Sent by: rsync-admin@lists.samba.org
11/19/2001 05:27 PM
To: rsync@lists.samba.org
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject: Bandwidth Limits
Classification:
Has anyone noticed that the --bwlimit doesn't really work?
I have MRTG stats happening, and on a 128Kbps circuit, an rsync
with the following syntax takes up the whole line.
$ rsync -avz -e ssh localfiles.tar.gz user@remote:/path/to/file/arch
Any ideas? It's on an OpenBSD box going to a FreeBSD box. Both are
2.4.6.
-#0
That was just too small a test. --bwlimit= causes rsync to sleep for an
appropriate interval after each data block (not tcp packet), in order to
pull the average transfer rate down to the specified limit. Your reported
transfer rate was only a little above 1kbps anyway, as the test was so
small, there wasn't time to even get the send>sleep>send>sleep
cycle
going... overhead stretched the time out so much that the 128k transferred
took an insignificant portion of the total runtime. Try a larger test.
maybe a meg or so. averages are meaningless on single instances...
average height of an adult american male is about 6 feet, right? if you
find a man 4 feet tall, that doesn't mean the average is wrong.
Tim Conway
tim.conway@philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
uid0@catastrophe.net
Sent by: rsync-admin@lists.samba.org
11/19/2001 06:04 PM
To: Tim Conway/LMT/SC/PHILIPS@AMEC
cc: rsync@lists.samba.org
Subject: Re: Bandwidth Limits
Classification:
On Mon, 2001-11-19 at 17:45:26 -0700, tim.conway@philips.com wrote...
; Try this commandline:
; rsync -avz -e ssh --bwlimit=64 localfiles.tar.gz
; user@remote:/path/to/file/arch
I did. This is what I get....I created a 128K file..
bash-2.05$ rsync --stats -avz --bwlimit=8 -e ssh blah:~/out .
user@remote's password:
receiving file list ... done
out
Number of files: 1
Number of files transferred: 1
Total file size: 127976 bytes
Total transferred file size: 127976 bytes
Literal data: 127976 bytes
Matched data: 0 bytes
File list size: 57
Total bytes written: 32
Total bytes read: 15325
wrote 32 bytes read 15325 bytes 1228.56 bytes/sec
total size is 127976 speedup is 8.33
bash-2.05$ ls -l out
-rw------- 1 user luser 127976 Nov 19 19:01 out
It is a bit less than obvious... It can be sure about its own kBps, but
wire protocols may vary... 100Mbps of ethernet is not 100/16 ofr 16Mbps
token ring is not 100/1.044 of T1 is not ...whatever.... rsh,, ssh, and
rsync transport protocols vary in overhead... the one thing of which rsync
can be certain is the number of bytes of actual data it transfers... At
least, that's my take on it.
Tim Conway
tim.conway@philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn,
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970),
".\n" '
"There are some who call me.... Tim?"
uid0@catastrophe.net
Sent by: rsync-admin@lists.samba.org
11/20/2001 10:03 AM
To: rsync@lists.samba.org
cc: (bcc: Tim Conway/LMT/SC/PHILIPS)
Subject: Re: Bandwidth Limits
Classification:
On Tue, 2001-11-20 at 07:49:59 -0700, tim.conway@philips.com wrote...
; That was just too small a test. --bwlimit= causes rsync to sleep for an
; appropriate interval after each data block (not tcp packet), in order to
; pull the average transfer rate down to the specified limit. Your
reported
; transfer rate was only a little above 1kbps anyway, as the test was so
; small, there wasn't time to even get the send>sleep>send>sleep
cycle
; going... overhead stretched the time out so much that the 128k
transferred
; took an insignificant portion of the total runtime.
Ok, here's my question. Is rsync using K/Bytes per second (KBps) or
K/Bits per second (Kbps)?? There's a huge difference here.
Doh!! It is KBytes. I just did an RTFM.
I'm at a loss as to why it would be written as KBytes, but whatever.
Thanks for the help.