On multiple core systems OpenSSH is limited to using a single core for
all operations. On these systems this can result in a transfer being
processor bound even though additional CPU resources exist. In order to
open up this bottleneck we've developed a multi-threaded version of
the AES-CTR cipher. Unlike CBC mode, since there is no dependency
between cipher blocks in CTR mode we parallelize cipher block operations
among multiple threads. Furthermore, since the AES_encrypt operations do
not depend on the data to be ciphered we pre-generate the effective
keystream. The main thread still handles the packetization, MAC
computation, and associated I/O but the computationally expensive
AES_encrypt operations are offloaded to one or more additional cores. In
our tests this resulted in a performance improvement of up to 125% on
systems that were previously CPU bound. In fact, utilizing less than
four cores we were able to achieve near line rate on a GigE LAN
connection with 128, 192, and 256-bit AES.
More details on the implementation can be found at
http://www.internet2.edu/presentations/jt2008jan/20080122-rapier-bennett.htm
starting at slide 30. Results can be found on slide 46.
As the resulting cipher stream is indistinguishable from the original
single-threaded implementation of AES-CTR there are no known issues with
backward compatibility.
This patch should be thought of as experimental at this point. While it
has performed well in test environments it is not yet, to our knowledge,
deployed in critical production environments and the threading can
impose a performance penalty on single core systems (but only when using
AES-CTR). We're still exploring methods to have single-threaded and
multi-threaded implementations of CTR mode exist side by side.
The patch itself can be found at
http://www.psc.edu/networking/projects/hpn-ssh/
or more specifically
http://www.psc.edu/networking/projects/hpn-ssh/openssh4.7-CTR-threading.diff
Additionally, this patch will apply on top of the HPN-SSH12v20 patch. It
will, within a week or so, be incorporated into the HPN suite of patches
as HPN13. If you have any problems applying the patch please let us know.
Any comments, suggestions, or critiques you may have are welcome and
appreciated.
