openssh bugs - Nov 2019 - [Bug 3096] New: sftp requires multiple round trips to do a write with a fsync()

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

            Bug ID: 3096
           Summary: sftp requires multiple round trips to do a write with
                    a fsync()
           Product: Portable OpenSSH
           Version: 8.1p1
          Hardware: Other
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: sftp
          Assignee: unassigned-bugs at mindrot.org
          Reporter: shawn at git.icu

https://libssh2.org/libssh2_sftp_fsync.html

Correct me if I am wrong, but the sftp protocol seems to require
multiple round trips to do a write followed by a fsync. Perhaps O_SYNC
is a solution here. The issue is that sftp can be over very high
latency links.

I edit source code over sftp. Very few editors do their IO on a
separate thread (vscode is an exception) and thus can be used for this.
But even if they can, the latency is horrible, because of multiple
round trips. And I *do* want fsync behavior, and while sync() calls are
notorious for being expensive, in this case the round trips are more
expensive.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

--- Comment #1 from Shawn Landden <shawn at git.icu> ---
This bug is really just for fixing the protocol, but consideration for
how applications can use that is of course important. O_SYNC is
probably the most straight-forward way, perhaps with opening the same
file twice so that it doesn't limit the program's options regarding
sync behavior.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Darren Tucker <dtucker at dtucker.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dtucker at dtucker.net

--- Comment #2 from Darren Tucker <dtucker at dtucker.net> ---
Why would the protocol require a round-trip?

The fsync extension operates on an open file handle
(https://cvsweb.openbsd.org/src/usr.bin/ssh/PROTOCOL section 3.6),
there can be multiple requests in flight from client to server
(https://tools.ietf.org/html/draft-ietf-secsh-filexfer-02 section 3)
and the server must process the operations on a given file handle in
order  (filexfer-02 section 6.1), so a client should be able to send
open,write,write,write,fsync,close, then process all of the replies.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org

--- Comment #3 from Damien Miller <djm at mindrot.org> ---
Darren is correct - the sftp protocol doesn't require an additional
round-trip for a fsync operation. It can be pipelined after (or during)
writes. Our sftp-server should deal with this just fine.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #4 from Damien Miller <djm at mindrot.org> ---
I think the problem here is that libssh2 doesn't offer a way to
pipeline the request, not with sftp-server itself.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|FIXED                       |WONTFIX

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #5 from Damien Miller <djm at mindrot.org> ---
closing resolved bugs as of 8.6p1 release

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Shawn Landden <shawn at git.icu> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|CLOSED                      |REOPENED
         Resolution|WONTFIX                     |---

--- Comment #6 from Shawn Landden <shawn at git.icu> ---
You didn't read your own comment when you closed this bug. You said it
was possible, and then contradicted yourself. I know there might not be
the interest or resources to do this, but that is not a reason to close
he bug.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |WORKSFORME

--- Comment #7 from Damien Miller <djm at mindrot.org> ---
Pretty sure it's not my reading comprehension that is at fault here. 

As Darren pointed out in comment #2, nothing in the sftp protocol or
sftp-server requires a roundtrip in this case.

As I pointed out in comment #4, the problem likely lies in your
(non-OpenSSH) client software.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3096

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.