konrad.gutkowski@ffs.pl
2007-Feb-22 13:17 UTC
[Lustre-discuss] improving wire speeds over ethernet
Hi, I''ve been using couple gigabit NIC''s (realtek 8169) with Lustre, was wondering if you can improve (probably can) the wire speeds by using jumbo frames ? Do i need to change anything else than mtu of the interface for this to work ? Also the driver supports frames only up to around 7K TiA Konrad Gutkowski
On Feb 22, 2007 21:16 +0100, konrad.gutkowski@ffs.pl wrote:> I''ve been using couple gigabit NIC''s (realtek 8169) with Lustre, > was wondering if you can improve (probably can) the wire speeds by using > jumbo frames ?Yes, this reduces the overhead of the TCP packets noticably.> Do i need to change anything else than mtu of the interface for this to > work? Also the driver supports frames only up to around 7KYou quite often need to restart the interface in order to change the MTU. Having an MTU slightly under 8kB means that a send or reply can fit into a 2-page allocation after the TCP/IP/eth headers are added (assuming there is no bug in the code as was the case with e1000 that always allocated 16kB). I don''t know the exact MTU size though. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Konrad, We have been using Jumbo frames with MTU=9000 for quite a while now so I can say that Lustre is very efficient at using the network even with MTU=1500, so you do not gain much in raw speed. You do gain by using less cpu on the OSS and clients which may help generally. You should change all hosts at the same time and, we have found up till now that Lustre appears to check the MTU size only at startup time so you have to do a complete restart on all hosts. John Belshaw CGGVeritas -----Original Message----- From: lustre-discuss-bounces@clusterfs.com [mailto:lustre-discuss-bounces@clusterfs.com] On Behalf Of konrad.gutkowski@ffs.pl Sent: 22 February 2007 20:16 To: lustre-discuss@clusterfs.com Subject: [Lustre-discuss] improving wire speeds over ethernet Hi, I''ve been using couple gigabit NIC''s (realtek 8169) with Lustre, was wondering if you can improve (probably can) the wire speeds by using jumbo frames ? Do i need to change anything else than mtu of the interface for this to work ? Also the driver supports frames only up to around 7K TiA Konrad Gutkowski _______________________________________________ Lustre-discuss mailing list Lustre-discuss@clusterfs.com https://mail.clusterfs.com/mailman/listinfo/lustre-discuss