Peter J. Braam
2006-Apr-27 23:44 UTC
[Lustre-devel] RE: [Lustre-discuss] Lustre over UDP ?
Hi Nir, Moved this topic to lustre-devel.... This would require coding a new LND - Lustre Network Driver, not a very complicated one probably. I am not sure how well this would work, because we only have run Lustre over reliable transports. But very few UDP packets are actually lost, so possibly this would work very well. CFS has no plans to do this. - Peter - ________________________________ From: lustre-discuss-bounces@clusterfs.com [mailto:lustre-discuss-bounces@clusterfs.com] On Behalf Of Nir Drang Sent: Thursday, April 27, 2006 7:16 AM To: lustre-discuss@clusterfs.com Subject: [Lustre-discuss] Lustre over UDP ? Hello All, I am quite new to Lustre so I hope I am not asking a dumb question. Is Lustre can be supported over UDP as well as TCP. Our client incorporates internal flow control from the application level and TCP processing Overhead can be minimized. Thanks, Nir. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-devel/attachments/20060427/1c9265a4/attachment.html
This goes along with the work I mentioned earlier that the group I''m with is doing. To cut a long story short, and to save anyone flipping through endless e-mails, there ARE schemes for doing reliable transport over UDP. Instead of having the packet verification done by the network layer, the verification is done at the next layer up, and instead of receiving the packets strictly in order, the packets are merely delivered to the application in order. There are Open Source implementations of such mechanisms out there, and I''m working on one for the company I work for. As for which one would be the best for Lustre - that I don''t know. I''ve not carried out enough research to guess as to which has the best performance or would require the least amount of effort to hook Lustre up to. If you merely want to use something with better performance than TCP, you might want to look at Linux'' DCCP implementation, as that is a protocol that is mid-way between TCP and UDP in behaviour. It has the definite benefit that it''s already in the kernel and should be fairly simple to move to. If you''re looking to squeeze every ounce of performance out of the system, are fairly good at coding, and want to be a little daring, you might try GAMMA - it''s a stack based on Active Messages that runs over raw ethernet. It''s said to be much faster than your typical IPv4 protocol and is supposed to have lower overheads on sending and receiving. Jonathan Day --- "Peter J. Braam" <braam@clusterfs.com> wrote:> Hi Nir, > > Moved this topic to lustre-devel.... > > This would require coding a new LND - Lustre Network > Driver, not a very > complicated one probably. > > I am not sure how well this would work, because we > only have run Lustre > over reliable transports. But very few UDP packets > are actually lost, > so possibly this would work very well. > > CFS has no plans to do this. > > - Peter - > > > ________________________________ > > From: lustre-discuss-bounces@clusterfs.com > [mailto:lustre-discuss-bounces@clusterfs.com] On > Behalf Of Nir Drang > Sent: Thursday, April 27, 2006 7:16 AM > To: lustre-discuss@clusterfs.com > Subject: [Lustre-discuss] Lustre over UDP ? > > > Hello All, > I am quite new to Lustre so I hope I am not asking > a dumb > question. > Is Lustre can be supported over UDP as well as TCP. > Our client incorporates internal flow control from > the > application level and TCP processing > Overhead can be minimized. > Thanks, > Nir. > > > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel@clusterfs.com >https://mail.clusterfs.com/mailman/listinfo/lustre-devel>__________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com