Lustre discuss - Jun 2006 - MX-10G and Myri-10G Ethernet performance pages

Hi all,

I have added some performance pages for Myri-10G NICs in MX mode and  
in Ethernet mode. These are at:

https://mail.clusterfs.com/wikis/lustre/MX-10G
https://mail.clusterfs.com/wikis/lustre/Myri-10G_Ethernet

These tests measure network performance only (I do not have access to  
any disk arrays that can handle 1,175 MB/s) using the beta Myrinet  
Express (MX) LND and the stock SOCK LND with the TCP Zero-Copy patch.

Using MXLND, we can nearly saturate the wire using a single client  
for read and write. Using 3 clients slightly increases the read and  
write performance, but there is not much room for an increase to  
begin with. ;-)

Using SOCKLND, we get about 45% of wire rate (~520 MB/s) using a  
single client for both read and write. When using 3 clients, write  
increases to 620 MB/s (50% of wire rate) and read achieves near wire  
rate of 1,175 MB/s. The difference is due to the fact that the Zero- 
Copy patch eliminate a copy at the sender side only. The receiver  
still has to copy. For reads, the clients have to copy but that is  
spread over multiple clients. For writes, the server has to do the  
copying and that prevents much of a speedup in this case.

The MXLND requires MX 1.2.0, which should be out in June/July.

Scott

--
Scott Atchley
Myricom Inc.
http://www.myri.com