bugzilla-daemon at bugzilla.mindrot.org
2018-Nov-08 13:45 UTC
[Bug 2927] New: scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927
Bug ID: 2927
Summary: scp -l can't limit the bandwidth at the first several
seconds
Product: Portable OpenSSH
Version: 7.7p1
Hardware: amd64
OS: Linux
Status: NEW
Severity: major
Priority: P5
Component: scp
Assignee: unassigned-bugs at mindrot.org
Reporter: lihao129 at huawei.com
When I use `-l` to limit the bandwidth to `10240Kbits`, the actual
bandwidth at the first several seconds is `10MB/s`. After several
seconds, the bandwidth will be limitted to `1MB/s` as expetced. I don't
know if this is a bug.
[root at danban-1 ~]# scp -l 10240 CentOS-7-x86_64-Everything-1804.iso
root at xxx.xxx.xxx.xxx:/
Authorized users only. All activities may be monitored and reported.
CentOS-7-x86_64-Everything-1804.iso
0% 10MB 10.0MB/s 14:53 ETA
--
You are receiving this mail because:
You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Nov-10 13:33 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927 --- Comment #1 from Hao Lee <lihao129 at huawei.com> --- In the first 8 seconds, the actual bandwidth is 8 times of expected bandwidth limit. If the bandwidth limit is 10240Kbit/s, the actual bandwidth is 10MB/s. After 8 seconds, the actual bandwidth limit will reduce to the expected limit which is 1.25MB/s. -- You are receiving this mail because: You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Dec-07 04:02 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927
Darren Tucker <dtucker at dtucker.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |djm at mindrot.org,
| |dtucker at dtucker.net
Attachment #3212| |ok?(djm at mindrot.org)
Flags| |
--- Comment #2 from Darren Tucker <dtucker at dtucker.net> ---
Created attachment 3212
--> https://bugzilla.mindrot.org/attachment.cgi?id=3212&action=edit
Fix scp bwlimit calculation.
Looking at the misc.c:bandwidth* functions I think there's two
problems.
> bandwidth_limit(struct bwlimit *bw, size_t read_len)
> { [...]
> if (!timerisset(&bw->bwstart)) {
> monotime_tv(&bw->bwstart);
> return;
> }
> bw->lamt += read_len;
The first (minor) problem: the first buffer written (16k, by default)
is not accounted for at all because it returns before it gets to the
code that accounts for it.
> if (bw->lamt < bw->thresh)
> return;
This is the main problem: bw->thresh is set to the bandwidth limit (in
kbit/s), which means that for any non-trivial bandwidth limit it'll
receive many writes before it even considers inserting a delay, so you
get an initial burst followed by pause until the average drops back to
the intended rate.
This patch moves accounting so the first block is accounted for, and
sets the initial threshold such that it'll start computing the rate on
the second and subsequent blocks.
Without patch:
$ ./scp -l 1000 /tmp/1MB localhost:/dev/null
1MB 0% 0 0.0KB/s --:-- ETA
1MB 100% 1024KB 1.0MB/s 00:01
1MB 100% 1024KB 511.8KB/s 00:02
1MB 100% 1024KB 341.2KB/s 00:03
1MB 100% 1024KB 255.9KB/s 00:04
1MB 100% 1024KB 204.7KB/s 00:05
1MB 100% 1024KB 170.6KB/s 00:06
1MB 100% 1024KB 146.2KB/s 00:07
1MB 100% 1024KB 127.9KB/s 00:08
With patch:
$ ./scp -l 1000 /tmp/1MB localhost:/dev/null
1MB 0% 0 0.0KB/s --:-- ETA
1MB 12% 128KB 127.9KB/s 00:07 ETA
1MB 25% 256KB 127.9KB/s 00:06 ETA
1MB 37% 384KB 127.9KB/s 00:05 ETA
1MB 50% 512KB 127.9KB/s 00:04 ETA
1MB 60% 624KB 126.3KB/s 00:03 ETA
1MB 73% 752KB 126.5KB/s 00:02 ETA
1MB 85% 880KB 126.6KB/s 00:01 ETA
1MB 98% 1008KB 126.8KB/s 00:00 ETA
1MB 100% 1024KB 124.6KB/s 00:08
--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Dec-07 04:02 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927
Darren Tucker <dtucker at dtucker.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
Blocks| |2915
Referenced Bugs:
https://bugzilla.mindrot.org/show_bug.cgi?id=2915
[Bug 2915] Tracking bug for 8.0 release
--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Dec-07 04:16 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927
Damien Miller <djm at mindrot.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Attachment #3212|ok?(djm at mindrot.org) |ok+
Flags| |
--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Dec-07 05:13 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927
Darren Tucker <dtucker at dtucker.net> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |FIXED
--- Comment #3 from Darren Tucker <dtucker at dtucker.net> ---
Thanks for the report, the patch has been applied and will be in the
OpenSSH 8.0 release.
--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Dec-07 07:15 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927 --- Comment #4 from Hao Lee <lihao129 at huawei.com> --- (In reply to Darren Tucker from comment #3)> Thanks for the report, the patch has been applied and will be in the > OpenSSH 8.0 release.Hi, I also submitted my patch to resolve this problem last month. However, I may misunderstand the algorithm for limiting the bandwidth, so I think my patch is wrong after reading you explanation. :) -- You are receiving this mail because: You are watching someone on the CC list of the bug. You are watching the assignee of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2018-Dec-07 07:18 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927 --- Comment #5 from Hao Lee <lihao129 at huawei.com> --- This is my patch :) https://lists.mindrot.org/pipermail/openssh-unix-dev/2018-November/037346.html -- You are receiving this mail because: You are watching the assignee of the bug. You are watching someone on the CC list of the bug.
bugzilla-daemon at bugzilla.mindrot.org
2019-May-03 04:42 UTC
[Bug 2927] scp -l can't limit the bandwidth at the first several seconds
https://bugzilla.mindrot.org/show_bug.cgi?id=2927
Damien Miller <djm at mindrot.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|RESOLVED |CLOSED
--- Comment #6 from Damien Miller <djm at mindrot.org> ---
Move resolved bugs -> CLOSED after 8.0 release
--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.