Is it still worth the effort to try and build mv_stata? when working with an x4500? stata_mv from RHEL4 does not appear to show some of the stability problems discussed online before. I am curious because the build system sun provides with the driver does not play nicely with the lustre kernel source packaging. If it is worth all the pain, if others have already figured it out. Any help would be grateful. Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985
Brock, It is recommended that mv_sata is used on the x4500. It has been a while since I have built this up myself and a few Lustre releases back but I do understand the pain. I hope that with Lustre 1.6.5.1 on RHEL 4.5 you can just build mv_sata against the provided Lustre kernel and alias it accordingly in modprobe.conf and create a new initrd, then update grub. I don''t have gear handy to give it try unfortunately. Please let me know your experiences with this if you pursue it. Enclosed is a somewhat dated document on what we have found to be the best configuration of the x4500 for use with Lustre. Ignore the N1SM parts. We optimized for performance and RAS with some sacrifices on capacity. Hopefully this is a useful reference. Regards, Mike Berg Sr. Lustre Solutions Engineer Sun Microsystems, Inc. Office/Fax: (303) 547-3491 E-mail: Mike.Berg at Sun.Com -------------- next part -------------- A non-text attachment was scrubbed... Name: X4500-preparation.pdf Type: application/pdf Size: 320945 bytes Desc: not available Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080807/447adfc0/attachment-0001.pdf -------------- next part -------------- On Aug 6, 2008, at 1:48 PM, Brock Palen wrote:> Is it still worth the effort to try and build mv_stata? when working > with an x4500? > stata_mv from RHEL4 does not appear to show some of the stability > problems discussed online before. > > I am curious because the build system sun provides with the driver > does not play nicely with the lustre kernel source packaging. > > If it is worth all the pain, if others have already figured it out. > Any help would be grateful. > > > Brock Palen > www.umich.edu/~brockp > Center for Advanced Computing > brockp at umich.edu > (734)936-1985 > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
Thanks, I might look into it. Right now the performance of the stock driver that comes with the kernel is more than the 4 1gig connections we will be using. I am having other issues now with the new filesystem that I did not have with our old one, that iwll be a new question though. Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985 On Aug 7, 2008, at 2:11 PM, Mike Berg wrote:> Brock, > > It is recommended that mv_sata is used on the x4500. > > It has been a while since I have built this up myself and a few > Lustre releases back but I do understand the pain. I hope that with > Lustre 1.6.5.1 on RHEL 4.5 you can just build mv_sata against the > provided Lustre kernel and alias it accordingly in modprobe.conf > and create a new initrd, then update grub. I don''t have gear handy > to give it try unfortunately. Please let me know your experiences > with this if you pursue it. > > Enclosed is a somewhat dated document on what we have found to be > the best configuration of the x4500 for use with Lustre. Ignore the > N1SM parts. We optimized for performance and RAS with some > sacrifices on capacity. Hopefully this is a useful reference. > > > Regards, > Mike Berg > Sr. Lustre Solutions Engineer > Sun Microsystems, Inc. > Office/Fax: (303) 547-3491 > E-mail: Mike.Berg at Sun.Com > > > <X4500-preparation.pdf> > > On Aug 6, 2008, at 1:48 PM, Brock Palen wrote: > >> Is it still worth the effort to try and build mv_stata? when working >> with an x4500? >> stata_mv from RHEL4 does not appear to show some of the stability >> problems discussed online before. >> >> I am curious because the build system sun provides with the driver >> does not play nicely with the lustre kernel source packaging. >> >> If it is worth all the pain, if others have already figured it out. >> Any help would be grateful. >> >> >> Brock Palen >> www.umich.edu/~brockp >> Center for Advanced Computing >> brockp at umich.edu >> (734)936-1985 >> >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >
Yes Brock -- as Mike has mentioned we also took this doc and provided this for our TACC customer: http://www.tacc.utexas.edu/resources/hpcsystems/ where we put 72 x4500s in place with this configuration with them. In addition, Sun''s recent Linux HPC Software http://www.sun.com/software/products/hpcsoftware/index.xml has the mv_sata driver and the SW configurations needed to put the x4500 together as an OSS one can build up upon and further configure with the SW RAID patches also included. HTH Mike Berg wrote On 08/07/08 11:11,:>Brock, > >It is recommended that mv_sata is used on the x4500. > >It has been a while since I have built this up myself and a few Lustre >releases back but I do understand the pain. I hope that with Lustre >1.6.5.1 on RHEL 4.5 you can just build mv_sata against the provided >Lustre kernel and alias it accordingly in modprobe.conf and create a >new initrd, then update grub. I don''t have gear handy to give it try >unfortunately. Please let me know your experiences with this if you >pursue it. > >Enclosed is a somewhat dated document on what we have found to be the >best configuration of the x4500 for use with Lustre. Ignore the N1SM >parts. We optimized for performance and RAS with some sacrifices on >capacity. Hopefully this is a useful reference. > > >Regards, >Mike Berg >Sr. Lustre Solutions Engineer >Sun Microsystems, Inc. >Office/Fax: (303) 547-3491 >E-mail: Mike.Berg at Sun.Com > > > >------------------------------------------------------------------------ > > > >On Aug 6, 2008, at 1:48 PM, Brock Palen wrote: > > > >>Is it still worth the effort to try and build mv_stata? when working >>with an x4500? >>stata_mv from RHEL4 does not appear to show some of the stability >>problems discussed online before. >> >>I am curious because the build system sun provides with the driver >>does not play nicely with the lustre kernel source packaging. >> >>If it is worth all the pain, if others have already figured it out. >>Any help would be grateful. >> >> >>Brock Palen >>www.umich.edu/~brockp >>Center for Advanced Computing >>brockp at umich.edu >>(734)936-1985 >> >> >> >>_______________________________________________ >>Lustre-discuss mailing list >>Lustre-discuss at lists.lustre.org >>http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> >>------------------------------------------------------------------------ >> >>_______________________________________________ >>Lustre-discuss mailing list >>Lustre-discuss at lists.lustre.org >>http://lists.lustre.org/mailman/listinfo/lustre-discuss >>-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080807/f6cad3bd/attachment.html