openssh bugs - Jan 2004 - [Bug 799] scp incorrectly reports "stalled" on slow copies

http://bugzilla.mindrot.org/show_bug.cgi?id=799

           Summary: scp incorrectly reports "stalled" on slow copies
           Product: Portable OpenSSH
           Version: 3.7.1p2
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: minor
          Priority: P4
         Component: scp
        AssignedTo: openssh-bugs at mindrot.org
        ReportedBy: peter.jeremy at alcatel.com.au


The following description relates to OpenSSH 3.5p1 as embedded in
FreeBSD/i386 4.9p1.  The overall behaviour (incorrectly reporting
"stalled") is identical on OpenSSH 3.7.1p2 and a quick check shows
that the scp blocksize and ssh hysteresis behaviour have not changed.
I believe the behaviour is not platform or OS dependent.

By default, scp(1) will provide a progress meter showing the
transfer ETA.  If the link is slow, the transfer meter will
alternate between displaying "- stalled -" and unrealistically
short ETAs even though the actual connection is transferring
data smoothly (as shown by tcpdump).

By default, the progress meter is updated every second.  If
there has been no apparent progress in the transfer after 5
seconds, the progress meter will report "stalled" until some
progress is reported.  There appear to be two issues that
will result in long delays between output progress being seen
by the progress meter.

Firstly, output from the scp process is in filesystem blocksize
blocks - the number of bytes transferred (used by the progres
meter) will only be incremented when a full block of data has
been transferred.  Therefore if the transfer rate is less than
1.6KB/sec (old 8K filesystem) or 3.2KB/sec (newer 16KB filesystem)
then the link will report as "stalled".  (Identified by code
inspection).

Secondly, the ssh process spawned by the scp process to perform
the actual encryption and transfer includes a substantial
internal buffer (>64KB) and appears to implement hysteresis.
ktrace output of a sample transfer shows a peak of over 96KB
buffered - at which point the ssh process stops reading until
the buffer drops to about 32KB.  This implies that there is
approximately 64KB hysteresis and a transfer rate below about
13KB/sec can result in "stalled" reports.

This behaviour is a regression from from an earlier version of
OpenSSH but I have not tracked down when it occurred.


To reproduce:
With FreeBSD, it is possible to use ipfw/dummynet to artificially
reduce the outgoing ssh bandwidth to a second system.  If traffic
shaping is not necessary, it will be necessary to create a low
speed link - eg ppp over an analogue modem or serial link.

I used the following commands:
# ipfw pipe 20 config queue 10 bw 80000
# ipfw add 1005 pipe 20 tcp from any to 192.168.164.18 22
# dd if=/dev/urandom of=data count=512
# scp data 192.168.164.18:/tmp

The appropriate fix is unclear - the buffering in both scp and ssh as
well as the hysteresis in ssh are beneficial to maximize transfer
bandwidth and minimise context switching.  It would not be desirable
to reduce these sizes when ssh is used across a LAN.

In the case of scp, changing from atomicio(write, ...) to write(...)
would remove the requirement to write at least filesystem_blocksize
bytes/sec to the remote system.

In the case of ssh, the hysteresis needs to be adjusted based on the
outgoing bandwidth - this could possibly be done by resetting the
"don't read more" flag after (say) 1 second.



------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.