Avnish & OpenSSH Community,
I'm sorry this has taken so long to reply to. I've been out of the
office for the past 3 week dealing with nuptials and such.
Avnish Bhatnagar wrote:> Hello,
>
> I know this has come up before; but is the HPN patch (or elements thereof)
> currently being considered for integration in to the OpenSSH code base?
I don't know. I do know that the scope of my patch is pretty broad and
getting it integrated at this point would require a certain amount of
effort to have it properly vetted. I understand how little time people
have to deal with these sort of things and how different people and
users have different priorities. However, even if the OpenSSH dev team
doesn't like the exact code I and Mike Stevens developed I do believe
that the concept is worth pursuing and I'm willing to do what I can to
provide necessary test beds if anyone on the dev team is interested.
Personally, I don't have anything emotionally invested in the code
itself. I just believe that incorporating the concept would be useful to
a large number of users - both in the higher end environments and in the
growing number of home environment with 10Mb/s+ connections.
> Are
> there pending issues (buffer management, none cipher, etc) which still need
> to be addressed?
I believe most of these issues have been addressed but I might be
mistaken. The none cipher usage maybe a show stopper but I have always
seen that as a secondary feature to the performance enhancements.
> We have been using HPN-SSH for over a year now, and like others, have
> observed significant performance improvement over standard OpenSSH. I can
> scp a 1 GB test file between two HPN-SSH LAN hosts at 700 Mbps (<1 ms
> latency). And over a cross-country high-BDP WAN link, I'm able to
achieve
> over 500 Mbps (85 ms latency). These single-stream scp transfers were run
> on well-tuned Linux kernels 2.6.15 (or higher) with the arcfour cipher.
> (I'll be happy to provide more details about these tests upon request.)
I'm
> not sure how 'typical' my results are, but they represent an order
of
> magnitude improvement over stock OpenSSH. While the improvement tends to
> vary among different platforms, I have never observed a performance
> degradation.
Actually those increases are pretty typical. Its not perfect and there
are still some aspects that can prevent a user from seeing the rates
they might see with gridFTP or some other dedicated bulk data mover but
in terms of simplicity, functionality, and security SSH, and the OpenSSH
implementation, is tough to beat. We've actually been thinking about
extending our usage of it to make it more of a middleware transport
mechanism for some applications.
> We recommend HPN SSH to our users who need to (securely) transfer their
bulk
> scientific datasets ranging in size of hundreds megabytes to one terabyte;
> so naturally, performance is very important for them. But they (or their
> sysadmins) are often reluctant to deploy software which represents a
> deviation from a standard distribution...and the maintenance issues that
> follow.
Which is understandable and one of the questions I frequently field on
this. Is it compatible and maintainable? Is it secure? Will it be part
of the main code base? (yes, yes, and I don't know). Still, there are a
growing number of people that have started using it as their default
distribution. The Data Working Group of the TeraGrid recently made it a
mandatory component for data transport servers. So adoption is growing,
at least in the HPN community.
Chris Rapier
Pittsburgh Supercomputing Center