Scott Atchley
2006-Jun-18 11:18 UTC
[Lustre-discuss] MX-10G and Myri-10G Ethernet performance pages
Hi all, I have added some performance pages for Myri-10G NICs in MX mode and in Ethernet mode. These are at: https://mail.clusterfs.com/wikis/lustre/MX-10G https://mail.clusterfs.com/wikis/lustre/Myri-10G_Ethernet These tests measure network performance only (I do not have access to any disk arrays that can handle 1,175 MB/s) using the beta Myrinet Express (MX) LND and the stock SOCK LND with the TCP Zero-Copy patch. Using MXLND, we can nearly saturate the wire using a single client for read and write. Using 3 clients slightly increases the read and write performance, but there is not much room for an increase to begin with. ;-) Using SOCKLND, we get about 45% of wire rate (~520 MB/s) using a single client for both read and write. When using 3 clients, write increases to 620 MB/s (50% of wire rate) and read achieves near wire rate of 1,175 MB/s. The difference is due to the fact that the Zero- Copy patch eliminate a copy at the sender side only. The receiver still has to copy. For reads, the clients have to copy but that is spread over multiple clients. For writes, the server has to do the copying and that prevents much of a speedup in this case. The MXLND requires MX 1.2.0, which should be out in June/July. Scott -- Scott Atchley Myricom Inc. http://www.myri.com