Dear OpenSSH maintainers,
When analyzing the performance of sftp some weeks ago, I discovered that
the cause of the horrible performance was the use of the Nagle algorithm
on the ssh TCP connection, i.e. TCP_NODELAY was not used.
I propose that the following patch is applied, enabling TCP_NODELAY for
both ssh and sshd, unconditionally. Since the SSH channel is packet
bases, if we want to reduce the amount of data on the wire, we should use
a Nagle-type algorithm before making packets of the data. It is more
efficient to make one SSH packet of two small data "chunks" than to
make
two packets of the data chunks, even if both fit within one TCP packet.
I also have a patch which implements overlapping read and write requests
for the sftp client. The patch was posted to the list at the time, but I
see no reason to apply that patch until the TCP_NODELAY issue has been
resolved.
/Tobias
diff -ru openssh-3.0.2p1.orig/packet.c openssh-3.0.2p1.nodelay/packet.c
--- openssh-3.0.2p1.orig/packet.c Mon Nov 12 01:07:58 2001
+++ openssh-3.0.2p1.nodelay/packet.c Thu Jan 17 20:40:47 2002
@@ -1180,7 +1180,6 @@
int lowdelay = IPTOS_LOWDELAY;
int throughput = IPTOS_THROUGHPUT;
#endif
- int on = 1;
if (called)
return;
@@ -1198,7 +1197,7 @@
if (interactive) {
/*
* Set IP options for an interactive connection. Use
- * IPTOS_LOWDELAY and TCP_NODELAY.
+ * IPTOS_LOWDELAY.
*/
#if defined(IP_TOS) && !defined(IP_TOS_IS_BROKEN)
if (packet_connection_is_ipv4()) {
@@ -1208,9 +1207,6 @@
strerror(errno));
}
#endif
- if (setsockopt(connection_in, IPPROTO_TCP, TCP_NODELAY, (void *) &on,
- sizeof(on)) < 0)
- error("setsockopt TCP_NODELAY: %.100s", strerror(errno));
} else if (packet_connection_is_ipv4()) {
/*
* Set IP options for a non-interactive connection. Use
diff -ru openssh-3.0.2p1.orig/sshconnect.c openssh-3.0.2p1.nodelay/sshconnect.c
--- openssh-3.0.2p1.orig/sshconnect.c Wed Oct 10 07:07:45 2001
+++ openssh-3.0.2p1.nodelay/sshconnect.c Thu Jan 17 20:09:11 2002
@@ -370,6 +370,7 @@
linger.l_onoff = 1;
linger.l_linger = 5;
setsockopt(sock, SOL_SOCKET, SO_LINGER, (void *)&linger, sizeof(linger));
+ setsockopt(sock, IPPROTO_TCP, TCP_NODELAY, (void *) &on, sizeof(on));
/* Set keepalives if requested. */
if (options.keepalives &&
diff -ru openssh-3.0.2p1.orig/sshd.c openssh-3.0.2p1.nodelay/sshd.c
--- openssh-3.0.2p1.orig/sshd.c Mon Nov 12 01:07:12 2001
+++ openssh-3.0.2p1.nodelay/sshd.c Thu Jan 17 20:42:24 2002
@@ -1118,6 +1118,7 @@
linger.l_onoff = 1;
linger.l_linger = 5;
setsockopt(sock_in, SOL_SOCKET, SO_LINGER, (void *) &linger,
sizeof(linger));
+ setsockopt(sock_in, IPPROTO_TCP, TCP_NODELAY, (void *) &on, sizeof(on));
/* Set keepalives if requested. */
if (options.keepalives &&
On Sat, 19 Jan 2002, Tobias Ringstrom wrote: :When analyzing the performance of sftp some weeks ago, I discovered that :the cause of the horrible performance was the use of the Nagle algorithm :on the ssh TCP connection, i.e. TCP_NODELAY was not used. : :I propose that the following patch is applied, enabling TCP_NODELAY for :both ssh and sshd, unconditionally. Since the SSH channel is packet :bases, if we want to reduce the amount of data on the wire, we should use :a Nagle-type algorithm before making packets of the data. It is more :efficient to make one SSH packet of two small data "chunks" than to make :two packets of the data chunks, even if both fit within one TCP packet. can you provide some tcpdump traces to show nagle delays during sftp?
mouring at etoh.eviladmin.org
2002-Jan-20 20:26 UTC
[PATCH] Using TCP_NODELAY unconditionally
On Sun, 20 Jan 2002, Markus Friedl wrote:> On Sun, Jan 20, 2002 at 12:26:15AM +0100, Tobias Ringstrom wrote: > > It is easy to theoretically > > show how Nagle kills performance when you have a request-reply packet > > based protocol. > > not everybody thinks like you > > http://groups.google.com/groups?th=885385572a5a4219&seekm=990719183329.AA27312%40dojo.mi.org&frame=off > > but i guess, it would be better if ssh implements a nagle-like > algorithms itself. >I think it would be better to see if we can avoid reinventing the wheel and offer a NoDelay No/Yes option on the client side. Unsure about the correct thing to do on the server side. - Ben
On Sun, 20 Jan 2002 mouring at etoh.eviladmin.org wrote:> I think it would be better to see if we can avoid reinventing the wheel > and offer a NoDelay No/Yes option on the client side. Unsure about the > correct thing to do on the server side.Is there no way to pass options from the client to the server? If not, a work-around for the sftp case could be to enable TCP_NODELAY when starting a subsystem. /Tobias
On Sun, Jan 20, 2002 at 10:02:46PM +0100, Tobias Ringstrom wrote:> On Sun, 20 Jan 2002 mouring at etoh.eviladmin.org wrote: > > > I think it would be better to see if we can avoid reinventing the wheel > > and offer a NoDelay No/Yes option on the client side. Unsure about the > > correct thing to do on the server side. > > Is there no way to pass options from the client to the server?no, unless we define a "TCP_NODELAY at openssh.com"> If not, a work-around for the sftp case could be to enable TCP_NODELAY > when starting a subsystem. > > /Tobias >
On Sun, Jan 20, 2002 at 03:23:48PM +0100, Markus Friedl wrote:> On Sun, Jan 20, 2002 at 12:26:15AM +0100, Tobias Ringstrom wrote: > > It is easy to theoretically > > show how Nagle kills performance when you have a request-reply packet > > based protocol. > > not everybody thinks like you > > http://groups.google.com/groups?th=885385572a5a4219&seekm=990719183329.AA27312%40dojo.mi.org&frame=off > > but i guess, it would be better if ssh implements a nagle-like > algorithms itself.Ok agreed. But, in the meantime, what is the rationale for turning Nagle off for interactive sessions and leaving it on for all others? I would expect the reverse...> -mCheers, Nico -- -DISCLAIMER: an automatically appended disclaimer may follow. By posting- -to a public e-mail mailing list I hereby grant permission to distribute- -and copy this message.- Visit our website at http://www.ubswarburg.com This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. This message is provided for informational purposes and should not be construed as a solicitation or offer to buy or sell any securities or related financial instruments.