bugzilla-daemon at mindrot.org
2007-Feb-18 16:53 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 Summary: SFTP keeps reading input until it runs out of buffer space Product: Portable OpenSSH Version: v4.5p1 Platform: All OS/Version: Linux Status: NEW Keywords: patch Severity: normal Priority: P2 Component: sftp AssignedTo: bitbucket at mindrot.org ReportedBy: thuejk at gmail.com I had a problem with the sshfs connection dying all the time. I have tracked the problem down to the check if (newlen > BUFFER_MAX_LEN) fatal("buffer_append_space: alloc %u not supported", newlen); in buffer.c:buffer_append_check() The problem is that when sending a file data, sshfs will just keep sending without waiting for the server to catch up. This does not need to be a problem, as the sftp server should just stop receiving TCP packets when the buffers are full, causing TCP resends and automatically slowing down the sending. However, the openssh sftp server loop (openssh:sftp-server.c) will just keep trying to read from stdin. It will read from stdin to its input buffer at most once per loop, and dispatch at most one sftp packet per loop. But as each read from stdin can read up to 8 sftp packets into the input buffer then the input buffer risks running out of space. When it runs out of space then it just kills itself (openssh:buffer.c:buffer_append_space). I note that the openssh sftp client has a mechanism to limit the number of unanswered requests, which probably means unlike sshfs, it is not affected. I found that a reliable way to trigger this bug was to start a parallel process which hogs the disk, causing the consuming sftp server loop to slow down. -sshfs-mount localhost:/tmp test -ionice -c1 cp large_file /tmp/deleteme -dd if=/dev/zero of=test/deleteme2 The error I get is dd: writing to `test/deleteme2': Input/output error dd: closing output file `test/deleteme2': Transport endpoint is not connected It is somewhat a matter of opinion, but I would say that it is openssh and not sshfs which should be fixed. If I look at http://www.openssh.org/txt/draft-ietf-secsh-filexfer-02.txt (which, however, does not seem to be the protocol actually implemented by openssh sftp, AFAICT), it says that There is no limit on the number of outstanding (non-acknowledged) requests that the client may send to the server. In practice this is limited by the buffering available on the data stream and the queuing performed by the server. If the server's queues are full, it should not read any more data from the stream, and flow control will prevent the client from sending more requests. Note, however, that while there is no restriction on the protocol level, the client's API may provide a limit in order to prevent infinite queuing of outgoing requests at the client. Two versions of a patch to fix this is attached, one for 4.3p2, and one for 4.5p1. This stops reading new data from stdin if the input buffer is full. As a bonus, I also handled the case where the output buffer is overwhelmed. ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Feb-18 16:58 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 ------- Comment #1 from thuejk at gmail.com 2007-02-19 03:58 ------- Created an attachment (id=1238) --> (http://bugzilla.mindrot.org/attachment.cgi?id=1238&action=view) A patch for sftp buffer handling applies to 4.5p1 ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Feb-18 16:59 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 ------- Comment #2 from thuejk at gmail.com 2007-02-19 03:59 ------- Created an attachment (id=1239) --> (http://bugzilla.mindrot.org/attachment.cgi?id=1239&action=view) A patch for sftp buffer handling applies to 4.5p1 ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Feb-18 17:00 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 thuejk at gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #1239|A patch for sftp buffer |A patch for sftp buffer description|handling applies to 4.5p1 |handling applies to 4.3p2 ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Feb-19 00:36 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 ------- Comment #3 from thuejk at gmail.com 2007-02-19 11:36 ------- A note: if you apply my 4.5p1 patch you should probably switch the two conditions if (buffer_compact(buffer)) goto restart; if (roundup(buffer->alloc + len, BUFFER_ALLOCSZ) <BUFFER_MAX_LEN) return (1); in buffer_check_alloc(), or you risk memmove()'ing for every incoming patchet. ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Feb-20 09:58 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 thuejk at gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #1238 is|0 |1 obsolete| | ------- Comment #4 from thuejk at gmail.com 2007-02-20 20:58 ------- Created an attachment (id=1241) --> (http://bugzilla.mindrot.org/attachment.cgi?id=1241&action=view) A patch for sftp buffer handling applies to 4.5p1 There was a problem in the old patch. Fixed. ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Feb-21 12:01 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 ------- Comment #5 from thuejk at gmail.com 2007-02-21 23:01 ------- Actually I don't think the part of the patch dealing with the output buffer is sufficient, since (I think) a single input package can create multiple output packages. I can make a patch if you want it, but I would like somebody to acknowledge this bug first. The part of the patch dealing with the input buffer is still good enough, and very much needed... ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Mar-01 12:17 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 dtucker at zip.com.au changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED OtherBugsDependingO| |1289 nThis| | ------- Comment #6 from dtucker at zip.com.au 2007-03-01 23:17 ------- Yes I think we need to look at this. In answer to the question about which draft it implements, it's filexfer-03, we should update the link. ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.
bugzilla-daemon at mindrot.org
2007-Mar-04 18:24 UTC
[Bug 1286] SFTP keeps reading input until it runs out of buffer space
http://bugzilla.mindrot.org/show_bug.cgi?id=1286 ------- Comment #7 from thuejk at gmail.com 2007-03-05 05:24 ------- Actually, on a closer reading of the code it does only send one response package per input package. So my patch is good. ------- You are receiving this mail because: ------- You are the assignee for the bug, or are watching the assignee.