Hi, I was reading about Lustre roadmap and future releases in the net.. I also read about Lustre and ZFS integration which was quite interesting. After reading many web sites I am a little confused now! Here are my questions: 1. Integrating ZFS with Lustre: Does this mean that some features of ZFS are going to be integrated with Lustre? ( like optimized checksums of ZFS etc ) 1. Which version of Lustre will have end-to-end data interity and which checksum algorithm will be used (if not CRC32)? (I read in one document written by Peter Bojanic which said Lustre+ZFS = End-to-End Data Integrity) So is it ver. 3.0 and above? 1. I read in wikipedia under ZFS integration "Lustre 3.0 will allow users to choose between ZFS and ldiskfs as back-end storage". Why ZFS and ldiskfs are treated separately here even after integration here? Once integrated it is just Lustre 3.0 isn''t it? I would be glad to hear the answers from you which may solve my queries and confusion :) Thanking you in advance.... Regards Ravi O. Rattihalli| Project Lead - R&D Services | MindTree Ltd. |Phase 1, Global Village, RVCE Post, Mysore Road, Bangalore - 560059, INDIA | Voice: (Direct) +91 80 6706-4135 Extn: 64135 / (Board) +91 80 6706-4000 / Fax +91 80 6706 4100 | email: Ravi_Rattihalli at mindtree.com|www.mindtree.com<http://www.mindtree.com/> | ________________________________ http://www.mindtree.com/email/disclaimer.html -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090113/2b9eea22/attachment.html
Hi Ravi, On Ter, 2009-01-13 at 11:59 +0530, Ravi Rattihalli wrote:> Here are my questions:> 1. Integrating ZFS with Lustre: Does this mean that some features > of ZFS are going to be integrated with Lustre? ( like > optimized checksums of ZFS etc )Yes, we plan to integrate some ZFS features with Lustre. One of such features is the checksumming, like you mentioned. We are planning to make the Lustre clients compute and provide the checksums over the wire to the servers, and use them as the block checksums in ZFS. That will achieve two goals: 1) offload checksum computation to clients, which in total have more CPU available than servers and 2) achieve true end-to-end data integrity. There may also be some integration in terms of quotas, or some other features. We will also be developing some features in ZFS, to achieve either better performance in some cases (e.g., a zero-copy API), or to achieve new functionality (e.g., multi-mount protection, but this is not our highest priority right now).> > 1. Which version of Lustre will have end-to-end data interity and > which checksum algorithm will be used (if not CRC32)? > > (I read in one document written by Peter Bojanic which said Lustre+ZFS > = End-to-End Data Integrity) So is it ver. 3.0 and above?I believe so.> > I read in wikipedia under ZFS integration ?Lustre 3.0 will allow > users to choose between ZFS and ldiskfs as back-end storage?. > > Why ZFS and ldiskfs are treated separately here even after integration > here? Once integrated it is just Lustre 3.0 isn?t it?Yes, but I''m not sure what is your confusion here. With Lustre 3.0, you will be free to choose whether you wish to create ldiskfs or ZFS-formatted backend devices - both options should be available.> > I would be glad to hear the answers from you which may solve my > queries and confusionJI hope my answers clarify things a bit. Cheers, Ricardo
Dear Ricardo Thank you very much for the reply. I will get back in case I need more information. Regards Ravi O. Rattihalli| Project Lead - R&D Services | MindTree Ltd. |Phase 1, Global Village, RVCE Post, Mysore Road, Bangalore - 560059, INDIA | Voice: (Direct) +91 80 6706-4135 Extn: 64135 / (Board) +91 80 6706-4000 / Fax +91 80 6706 4100 | email: Ravi_Rattihalli at mindtree.com|www.mindtree.com | -----Original Message----- From: Ricardo.M.Correia at Sun.COM [mailto:Ricardo.M.Correia at Sun.COM] Sent: Friday, January 16, 2009 5:19 AM To: Ravi Rattihalli Cc: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Query Hi Ravi, On Ter, 2009-01-13 at 11:59 +0530, Ravi Rattihalli wrote:> Here are my questions:> 1. Integrating ZFS with Lustre: Does this mean that some features > of ZFS are going to be integrated with Lustre? ( like > optimized checksums of ZFS etc )Yes, we plan to integrate some ZFS features with Lustre. One of such features is the checksumming, like you mentioned. We are planning to make the Lustre clients compute and provide the checksums over the wire to the servers, and use them as the block checksums in ZFS. That will achieve two goals: 1) offload checksum computation to clients, which in total have more CPU available than servers and 2) achieve true end-to-end data integrity. There may also be some integration in terms of quotas, or some other features. We will also be developing some features in ZFS, to achieve either better performance in some cases (e.g., a zero-copy API), or to achieve new functionality (e.g., multi-mount protection, but this is not our highest priority right now).> > 1. Which version of Lustre will have end-to-end data interity and > which checksum algorithm will be used (if not CRC32)? > > (I read in one document written by Peter Bojanic which said Lustre+ZFS > = End-to-End Data Integrity) So is it ver. 3.0 and above?I believe so.> > I read in wikipedia under ZFS integration "Lustre 3.0 will allow > users to choose between ZFS and ldiskfs as back-end storage". > > Why ZFS and ldiskfs are treated separately here even after integration > here? Once integrated it is just Lustre 3.0 isn''t it?Yes, but I''m not sure what is your confusion here. With Lustre 3.0, you will be free to choose whether you wish to create ldiskfs or ZFS-formatted backend devices - both options should be available.> > I would be glad to hear the answers from you which may solve my > queries and confusionJI hope my answers clarify things a bit. Cheers, Ricardo http://www.mindtree.com/email/disclaimer.html