openssh bugs - Jun 2023 - [Bug 3578] New: RFE: forward error correction

https://bugzilla.mindrot.org/show_bug.cgi?id=3578

            Bug ID: 3578
           Summary: RFE: forward error correction
           Product: Portable OpenSSH
           Version: 9.3p1
          Hardware: Other
                OS: Linux
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: ssh
          Assignee: unassigned-bugs at mindrot.org
          Reporter: openssh at richardneill.org

Rationale:

Sometimes, the underlying link has flaky hardware, and yet plenty of
speed and bandwidth. 

For example, Wi-Fi, with sporadic bursts of radio-interference, or a
failing network switch which drops 25% of packets, or resets every 10
seconds (being out for 5 seconds), even when the network is far from
saturated. In this case, the channel-capacity, and error-rate is
oscillating between high and low.

This results in an unusable SSH connection, because of packet drops,
and worse, the TCP flow-control algorithm misinterprets this as
"network saturation / congestion", so it backs off, when the correct
response to this situation is "try harder".

This is particularly challenging when you are trying to remotely debug
over such a link, and SSH gets "stuck" even when some pings are
getting
through.


Proposal:

I suggest a --flaky option, which would do 3 things:

* forward error-correction: preemptively transmit each packet 3x (both
from the client-end and the server-end) without waiting to find out
whether it was lost.

* tweak the TCP timeout-timer for this connection to < 0.5 second, i.e.
be much more aggressive about when a packet is deemed to be lost, and
is re-requested.

* If interactive, flash the cursor red, to indicate the moments that it
is trying to retransmit.

An alternative might be to do this at the protocol-layer, or to do some
sort of RAID-style error-correction / data-interleaving so that all the
data can be reconstructed, only if 1/3 of the packets are received.


Examples:

* Voyager space-probe does forward error correction on transmitted
data, because it knows some data will be lost to interference.

* CD players can cope with huge blocks of read-failures (experiment on
an unwanted disc - you can draw 8 radial lines with a 3mm thick black
marker pen and it will still playback ok).


Test case:

If this works, it would allow a somewhat sluggish, but still usable
interactive SSH session to work, even if:

* 50% of packets, randomly selected, were dropped / delayed

* 50% of packets, in square-wave-bursts at a 100ms, 1-second, or
10-second scale were dropped.


I hope the idea is helpful - thank you for your time.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3578

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org

--- Comment #1 from Damien Miller <djm at mindrot.org> ---
This would be nice, but the features that you're asking for are really
at the TCP level - protocols that use TCP have very little control over
timeouts and retransmits that happen in TCP. Similarly, SSH sending
FEC'd data over a TCP channel would do little, since the bad behaviour
in the presence of packet loss is a TCP phenomenom.

Other protocols such as QUIC avoid this by switching the transport from
TCP to UDP, but this comes at the price of the protocol having to
manage packet loss detection, in-order delivery, retransmission, etc. I
don't think there is much appetite to take on this burden in SSH.

Given that SSH is likely to remain on TCP for now, the best you can do
is get TCP to behave better in environments with high packet loss. Some
platforms (e.g. Linux) do offer a menu of TCP packet scheduling
algorithms, and some of these are designed to do better in these cases.
https://linuxgazette.net/135/pfeiffer.html has some details.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3578

--- Comment #2 from Darren Tucker <dtucker at dtucker.net> ---
(In reply to Richard Neill from comment #0)
[...]
> * forward error-correction: preemptively transmit each packet 3x
> (both from the client-end and the server-end) without waiting to
> find out whether it was lost.
The SSH protocol is designed to use an 8-bit clean error free
transport.  In OpenSSH this is usually provided by TCP, but you can
implement whatever transport you want via a ProxyCommand.

Preemptively transmitting multiple copies of TCP data packets is not
possible in any TCP socket interface I am aware of.  Sending multiple
copies at the application layer will just result in TCP delivering all
(or in the case of network problems, none) of them.
> * tweak the TCP timeout-timer for this connection to < 0.5 second,
> i.e. be much more aggressive about when a packet is deemed to be
> lost, and is re-requested.
You can use a ProxyCommand as a dialer (ProxyUseFdpass) and make
whatever tweaks you want to the TCP socket before handing it back to
ssh.
> * If interactive, flash the cursor red, to indicate the moments that
> it is trying to retransmit.
I think the thing you are looking for is called "mosh".
> * Voyager space-probe does forward error correction on transmitted
> data, because it knows some data will be lost to interference.
Voyager is also somewhat beyond the 120 second MSL upper bound that TCP
has.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3578

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |WONTFIX

--- Comment #3 from Damien Miller <djm at mindrot.org> ---
I don't think we will make any changes in this direction for the
reasons outlined. As Darren pointed out however, it's possible to run
ssh over a ProxyCommand that uses another protocol, including ones with
FEC.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.