I've setup a lab to test the new tunneling options in the latest
openssh. Things work well... for a while... then the tunnel goes
unidirectional.
openssh-SNAP-20060122.tar.gz
lab1 config:
# uname -a
FreeBSD lab1 6.0-RELEASE FreeBSD 6.0-RELEASE #0: Fri Jan 13 13:01:17
EST 2006 root@:/usr/src/sys/i386/compile/SMP i386
# cat /etc/sysctl.conf | egrep -v '(^#|^$)'
net.inet.ip.fastforwarding=1
# cat ~/.ssh/config
Host 169.254.254.20
Tunnel yes
TunnelDevice 0:any
PermitLocalCommand yes
LocalCommand sh /root/scripts/netstart tun0
# cat /root/scripts/netstart
#!/bin/sh
ifconfig $1 inet 169.254.253.10 169.254.253.20 netmask 255.255.255.0 &&
\
route add host2 169.254.253.20
lab 2 config:
# uname -a
FreeBSD lab2 6.0-RELEASE FreeBSD 6.0-RELEASE #0: Fri Jan 13 12:00:56
EST 2006 root@:/usr/src/sys/i386/compile/SMP i386
# cat /etc/sysctl.conf | egrep -v '(^#|^$)'
net.inet.ip.fastforwarding=1
# cat /etc/ssh/sshd_config | egrep -v '(^#|^$)'
PermitTunnel point-to-point
# cat ~/.ssh/authorized_keys2
tunnel="1",command="/root/scripts/netstart tun1" ssh-dss
AAAA...
# cat /root/scripts/netstart
#!/bin/sh
ifconfig $1 inet 169.254.253.20 169.254.253.10 netmask 255.255.255.0 &&
\
route add host1 169.254.253.10
The test consists of a single TCP stream operating at the maximum
capacity of a FastEthernet connection 12.5MBps using the command:
#setup the tunnel and routes
lab1# ssh -fw 0:any 169.254.254.20 true
host1$ cat /dev/zero | nc -l 2000
host2$ nc -o host1 2000 > /dev/zero
This test usually runs between 4-5 seconds before the tunnel goes
unidirectional: packets tunnelized between lab1 -> lab2 work but
tunneled packets between lab2 -> lab1 do not.
After changing the MTU of the test TCP stream to 1400 it ends up
lasting around 2 minutes before going unidirectional.
lab2# ping 169.254.253.10
PING 169.254.253.10 (169.254.253.10): 56 data bytes
ping: sendto: No buffer space available
Without the ssh tunnel the lab boxes easily route that much bandwidth
indefinitely.
Any ideas on how I can further diagnose the problem would be appreciated.
-Corey Smith
On Tue, Jan 24, 2006 at 12:52:05PM -0500, Corey Smith wrote:> I've setup a lab to test the new tunneling options in the latest > openssh. Things work well... for a while... then the tunnel goes > unidirectional.[...]> PING 169.254.253.10 (169.254.253.10): 56 data bytes > ping: sendto: No buffer space availableIt looks like the buffer space in your tunnel device is full. Is the SSH stream still running? If you stop transmitting traffic does it come good again? Are other normal connections to that machine affected? You might also like to try picking a fast cipher (arcfour is usually fastest of the standard ones) and see if that makes a difference. -- Darren Tucker (dtucker at zip.com.au) GPG key 8FF4FA69 / D9A3 86E9 7EEE AF4B B2D4 37C9 C982 80C7 8FF4 FA69 Good judgement comes with experience. Unfortunately, the experience usually comes from bad judgement.
On 1/24/06, Darren Tucker <dtucker at zip.com.au> wrote:> It looks like the buffer space in your tunnel device is full. Is the > SSH stream still running? If you stop transmitting traffic does it come > good again? Are other normal connections to that machine affected? > > You might also like to try picking a fast cipher (arcfour is usually > fastest of the standard ones) and see if that makes a difference.Thanks for the response. I think the buffer space exhaustion is a sympton and not a cause: the tunnel quits sending so its send buffers get full. The tunnel between lab2 -> lab1 fails. The only thing that would be going through the tunnel in this test is the ACK stream from the netcat. Other connections to lab[12] are not affected during the lock (even other ssh connections to the same daemon). AFAIK the tunnel never fixes itself. If I kill the ssh process that started the tunnel on lab1 and then restart it the tunnel works again... for a while. I will try using a faster encryption algorithm later today. -Corey Smith