On 02/12/2018 06:34 PM, Stephen John Smoogen wrote:> On Mon, Feb 12, 2018 at 6:25 PM H <agents at meddatainc.com> wrote: > >> Running CentOS 7 on workstation and having a problem with ssh disconnects. >> My ssh_config contains: >> >> Host * >> TCPKeepAlive yes >> ServerAliveInterval 30 >> ServerAliveCountMax 300 >> >> and sshd_config on the server contains: >> >> TCPKeepAlive yes >> ClientAliveInterval 60 >> ClientAliveCountMax 300 >> >> Have I missed any setting needed to prevent these random disconnects? I >> don't think there is anything wrong with the network card, the driver, or >> the cable, since if I am on a VPN connection via another server, the VPN >> and any ssh connection stay up indefinitely. >> >> Thanks. > > > There are usually 2 different reasons for this: > 1. The VPN is UDP and times out/drops keeps alives so that they no longer > function properly. [The UDP connection will make it look like you have a > new SSH connection which of course the system will drop because that would > allow for security problems.] > > 2. A firewall in the chain of things (system you are on, the system you are > going to, or somewhere in between) has session flushing issues. If you have > the firewall set up to only accept NEW port 22 connections and then just > looks to see if the ESTABLISHED, RELATED tables are accepted elsewhere then > if the session somehow ages out or is flushed due to usage, the ssh > connection can get dropped. > > The solution to one is to see if a TCP VPN fixes the problem. The second > one is to either make the iptables kernel tables larger or to have all port > 22 accepted even if it is not ESTABLISHED. > > These aren?t the only ways the problem you see can occur but they are some > of the most common I have run into. > > > >> >> _______________________________________________ >> CentOS mailing list >> CentOS at centos.org >> https://lists.centos.org/mailman/listinfo/centos >>Not sure if I am reading your reply correctly but I should clarify that I have problems when running naked ssh to the server, when I run ssh to the same server but over the VPN connection (that goes via third server) everything is flawless. I should also explain that: - I am on a workstation (located in the US), ssh-ing into server 1 (located in the US). - From server 1 I use scp to transfer large files from server 2 (located in Europe) to server 1 (in the US). The above randomly disconnects. However, when: - I use a VPN connection to server 3 (also located in Europe). - From the same workstation as above, do exactly as above, connections are rock-solid.
On Mon, Feb 12, 2018 at 08:05:18PM -0500, H wrote: ...> Not sure if I am reading your reply correctly but I should clarify > that I have problems when running naked ssh to the server, when I > run ssh to the same server but over the VPN connection (that goes > via third server) everything is flawless. > > I should also explain that: > > - I am on a workstation (located in the US), ssh-ing into server 1 > (located in the US). > > - From server 1 I use scp to transfer large files from server 2 > (located in Europe) to server 1 (in the US). > > The above randomly disconnects.You have two ssh sessions here. AFAICT, it's the first one (workstation->server 1) that randomly disconnects, right?> > However, when: > > - I use a VPN connection to server 3 (also located in Europe). > > - From the same workstation as above, do exactly as above, > connections are rock-solid.It would seem to me that something between your workstation and server 1 is possibly mishandling TCP options (being it sack, timestamps or both). Is there a router or a firewall in between? Try doing a traffic capture between both and see why it hangs. If you're hesitant to post the binary traffic capture, post the text version of it with anonymized IP addresses. Just please be sure to disable ssh protocol so it would include TCP details on it then. Marcelo
On 02/19/2018 10:09 PM, Marcelo Ricardo Leitner wrote:> On Mon, Feb 12, 2018 at 08:05:18PM -0500, H wrote: > ... >> Not sure if I am reading your reply correctly but I should clarify >> that I have problems when running naked ssh to the server, when I >> run ssh to the same server but over the VPN connection (that goes >> via third server) everything is flawless. >> >> I should also explain that: >> >> - I am on a workstation (located in the US), ssh-ing into server 1 >> (located in the US). >> >> - From server 1 I use scp to transfer large files from server 2 >> (located in Europe) to server 1 (in the US). >> >> The above randomly disconnects. > You have two ssh sessions here. AFAICT, it's the first one > (workstation->server 1) that randomly disconnects, right? > >> However, when: >> >> - I use a VPN connection to server 3 (also located in Europe). >> >> - From the same workstation as above, do exactly as above, >> connections are rock-solid. > It would seem to me that something between your workstation and server > 1 is possibly mishandling TCP options (being it sack, timestamps or > both). Is there a router or a firewall in between? > > Try doing a traffic capture between both and see why it hangs. If > you're hesitant to post the binary traffic capture, post the text > version of it with anonymized IP addresses. Just please be sure to > disable ssh protocol so it would include TCP details on it then. > > MarceloLet's reduce the problem to a simpler case: the workstation running CentOS 7 is behind a router, behind the same router is the server running CentOS 6. For various reasons, I access the server via its external IP address, not the local one. Access is via ssh. When using straight ssh, the connection times out after random times, always less than 20 minutes or so, though, with the error message "packet_write_wait: broken pipe", not the common "write failed: broken pipe". However, when I am in a ssh session on a VPN connection (which terminates in an outside server far away from my network), the ssh connection does /not/ time out. Further, accessing the same server as first mentioned, in the identical fashion, from another computer running CentOS 6, I never encounter the "packet_write_wait" error and disconnection. My conclusion is thus that this is not a router issue but could conceivable be a bug in CentOS 7. I am also ruling out a cable issue or a physical network card issue since the VPN connection is stable.