What is the process for integrating a Lustre+SAMFS solution into an existing customer environment. The plan is to have CRS build the Lustre component, but Lustre and SAMFS will need to configured and integrated into the customer computing environment. I am very familiar with the SAMFS integration, but not Lustre integration. Do we have resources in PS to provide the integration? Is this done by the CFS organization? Also, a small scale benchmark of the solution may be required. Which Benchmark center could provide Lustre support? Thanks, Jim Brennan Digital Media Systems Sun Systems Group Universal City, CA (310) 901-86777
Hi Brennan, Larry McIntosh of our Linux HPC team can advise you regarding our Lustre/SAM integration options. Cheers, Bojanic On 26-Mar-08, at 17:47, Brennan <James.E.Brennan at Sun.COM> wrote:> What is the process for integrating a Lustre+SAMFS solution into an > existing customer environment. The plan is to have CRS build the > Lustre component, but Lustre and SAMFS will need to configured and > integrated into the customer computing environment. I am very > familiar with the SAMFS integration, but not Lustre integration. > > Do we have resources in PS to provide the integration? Is this done > by the CFS organization? > > Also, a small scale benchmark of the solution may be required. Which > Benchmark center could provide Lustre support? > > Thanks, > > Jim Brennan > Digital Media Systems > Sun Systems Group > Universal City, CA > (310) 901-86777
We run Lustre benchmarks frequently in the SSC for HPC in Hillsboro. To request a benchmark or POC, go to http://ssc.west and click on Engage SSC. When completing the form, select HPC Benchmarking in Hillsboro, otherwise it''ll take an extra day or two to wander through different parts of SSC. Barbara On 03/26/08 02:47 PM, Brennan wrote:> What is the process for integrating a Lustre+SAMFS solution into an > existing customer environment. The plan is to have CRS build the > Lustre component, but Lustre and SAMFS will need to configured and > integrated into the customer computing environment. I am very > familiar with the SAMFS integration, but not Lustre integration. > > Do we have resources in PS to provide the integration? Is this done by > the CFS organization? > > Also, a small scale benchmark of the solution may be required. Which > Benchmark center could provide Lustre support? > > Thanks, > > Jim Brennan > Digital Media Systems > Sun Systems Group > Universal City, CA > (310) 901-86777-- Barbara Perz Phone: (503) 617-8594 (x46594) HPC Benchmark Specialist Fax : (503) 617-3363 (x57363) Sun Solution Center Email: barbara.perz at sun.com Sun Microsystems Inc. http://ssc.west/
ALl: We have in Hillsboro a test rig for our HPC Storage and Archive Solution, and are charaterizing performance as I write this. Currently, it''s the SAM-QFS part with multiple QFS data movers. WHen we finish that part (2-3 weeks), we plan to add the Lustre part (We''re going to need some technical assistance then) and test several configurations for bandwidth etc. Net: Stay tuned - We''ll have numbers and best practices to implement. CW Larry McIntosh wrote: Brennan, One needs SAM-QFS Linux client on a given node that is a lustre client. There can be multiples of these within a cluster. This client(s) can then move (copy data) between SAM and Lustre. This has been put in place at a number of places for Sun customers such as DKRZ in Germany. This is a very basic solution today and we would like to see a tighter integration between the Lustre and SAM development efforts to have this as a more robust offering and SAM SW support for newer Linux Kernels. It is my understanding that this is underway. However for the time being it will require the aforementioned type of client. Larry Brennan wrote On 03/26/08 14:47,: What is the process for integrating a Lustre+SAMFS solution into an existing customer environment. The plan is to have CRS build the Lustre component, but Lustre and SAMFS will need to configured and integrated into the customer computing environment. I am very familiar with the SAMFS integration, but not Lustre integration. Do we have resources in PS to provide the integration? Is this done by the CFS organization? Also, a small scale benchmark of the solution may be required. Which Benchmark center could provide Lustre support? Thanks, Jim Brennan Digital Media Systems Sun Systems Group Universal City, CA (310) 901-86777 -- ======================================Chris Wood CTO - Storage & Data Management Practice Sun Microsystems Office: +1 408 276 3773 Mobile: +1-408-218-7313 (Preferred) Email: chris.wood-xsfywfwIY+M@public.gmane.org ======================================= _______________________________________________ Lustre-discuss mailing list Lustre-discuss-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Chris, Just a tip on the Lustre help scenario. Don''t wait till the last minute to get the Lustre guy involved. If they can''t just drop in one of the pre-built kernels (like Redhat ur Suse) it''s going to take some work. They will need to know if there is any wiggle room on which OS and/or kernel revision is used. Some customers/demos have very specific requirements on this, some are more relaxed, as in anything that''s come out in the last year. So, they are going to need hardware details like the exact OS requirements & kernel revision requirements, type of disk storage (DDN, Engenio/LSI, other). They also need to know the IB stack (OFED, etc) and if it''s a switch dependant stack, then the brand of switch (i.e- voltaire, topspin, silverstorm) so they know what options to compile the Lustre source with. If you are using CentOS, then you may get by with a prebuilt redhat kernel that comes with an older version of OFED IB support in it. But, if the customer requires the latest version of OFED, then you''re in for some work. -Sam At 04:18 PM 4/28/2008, Chris Wood wrote:>ALl: > >We have in Hillsboro a test rig for our HPC Storage and Archive Solution, >and are charaterizing performance as I write this. Currently, it''s the >SAM-QFS part with multiple QFS data movers. > >WHen we finish that part (2-3 weeks), we plan to add the Lustre part >(We''re going to need some technical assistance then) and test >several configurations for bandwidth etc. > >Net: Stay tuned - We''ll have numbers and best practices to implement. > >CW > > >Larry McIntosh wrote: >> >>Brennan, >> >>One needs SAM-QFS Linux client on a given node that is a lustre client. There >>can be multiples of these within a cluster. >> >>This client(s) can then move (copy data) between SAM and Lustre. >> >>This has been put in place at a number of places for Sun customers such as DKRZ in Germany. >> >>This is a very basic solution today and we would like to see a tighter integration >>between the Lustre and SAM development efforts to have this as a more robust offering and >>SAM SW support for newer Linux Kernels. >> >>It is my understanding that this is underway. However for the time being it will require >>the aforementioned type of client. >> >>Larry >> >> >>Brennan wrote On 03/26/08 14:47,: >>> >>>What is the process for integrating a Lustre+SAMFS solution into an >>>existing customer environment. The plan is to have CRS build the >>>Lustre component, but Lustre and SAMFS will need to configured and >>>integrated into the customer computing environment. I am very >>>familiar with the SAMFS integration, but not Lustre integration. >>> >>>Do we have resources in PS to provide the integration? Is this done >>>by the CFS organization? >>> >>>Also, a small scale benchmark of the solution may be required. Which >>>Benchmark center could provide Lustre support? >>> >>>Thanks, >>> >>>Jim Brennan >>>Digital Media Systems >>>Sun Systems Group >>>Universal City, CA >>>(310) 901-86777 >>> > > >-- >======================================>Chris Wood >CTO - Storage & Data Management Practice >Sun Microsystems >Office: +1 408 276 3773 >Mobile: +1-408-218-7313 (Preferred) >Email: <mailto:chris.wood at sun.com>chris.wood at sun.com >=======================================-------------- next part --------------An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080428/fa4ba6e2/attachment-0001.html
On Mon, 2008-04-28 at 17:32 -0600, Sam Bigger wrote:> > Chris, > > Just a tip on the Lustre help scenario. Don''t wait till the last > minute to get the Lustre guy > involved.This is good advise. A lot of planning and decision making needs to be done far ahead of equipment purchase to make for a successful deployment.> If they can''t just drop in one of the pre-built kernels (like Redhat > ur Suse) it''s going > to take some work.Keep in mind, for RHEL 5 and SLES 10 clients, we support "patchless" mode in which you run lustre modules that are built for the kernel that comes with the vendor''s distribution. No need to build kernels for clients, and in many cases, no need to even build lustre. For servers (OSS and MDS), they should be mostly "sealed server" (i.e. no other software should run on it -- reducing the need for kernels other than the Sun provided Lustre kernels) installations anyway, so the kernels we provide should suffice.> Some customers/demos have very specific requirements on this,But see above re: patchless clients. We will run on the vendor supplied kernel for RHEL5 and SLES10.> They also need to know the IB stack (OFED, etc)OFED is the preferred I/B stack if possible.> If you are using CentOS, then you may get > by with a prebuilt redhat kernel that comes with an older version of > OFED IB support in it.RHEL5 (and thus CentOS 5) comes with an OFED stack included in the OS.> But, if > the customer requires the latest version of OFED, then you''re in for > some work.1.6.5 will ship with OFED 1.3''s kernel-ib RPMs and Lustre all built for it. So, IOW, 1.6 5 will work with OFED I/B compliant hardware "out of the box" b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080428/eca47722/attachment-0001.bin
On Mon, 28 Apr 2008, Brian J. Murrell wrote:> On Mon, 2008-04-28 at 17:32 -0600, Sam Bigger wrote:... snip> Keep in mind, for RHEL 5 and SLES 10 clients, we support "patchless" > mode in which you run lustre modules that are built for the kernel that > comes with the vendor''s distribution. No need to build kernels for > clients, and in many cases, no need to even build lustre.*** There must be qualifications here? Are all RHEL 5 and SLES 10 (SPx) up to being "patchless" level for clients? Is this really set in stone for all levels of future Lustre, too?> For servers (OSS and MDS), they should be mostly "sealed server" (i.e. > no other software should run on it -- reducing the need for kernels > other than the Sun provided Lustre kernels) installations anyway, so the > kernels we provide should suffice.Not if you do not supply RDAC (mpp) kernels, too, e.g. for 6140?> RHEL5 (and thus CentOS 5) comes with an OFED stack included in the OS.Old, especially for the linkes of ConnextX/Hermon HCA.>> But, if >> the customer requires the latest version of OFED, then you''re in for >> some work. > > 1.6.5 will ship with OFED 1.3''s kernel-ib RPMs and Lustre all built for > it. So, IOW, 1.6 5 will work with OFED I/B compliant hardware "out of > the box"You need to supply SRPMS/src too as some people (like Hillsboro HPC benchmark) need to produce compatible PathScale/INTEL/PGI MPI versions (and maybe gcc if you don''t build that per OS/gcc release level). Also the comments (above) about mpp (RDAC) apply - you''d need modules built against that. Regards, DM
[ this was in my Inbox to respond to but slipped behind the landslide of e-mail I have gotten recently. hopefully better late than never. ] On Mon, 2008-04-28 at 19:05 -0700, Mostyn Lewis wrote:> On Mon, 28 Apr 2008, Brian J. Murrell wrote: > > > On Mon, 2008-04-28 at 17:32 -0600, Sam Bigger wrote: > ... snip > > > Keep in mind, for RHEL 5 and SLES 10 clients, we support "patchless" > > mode in which you run lustre modules that are built for the kernel that > > comes with the vendor''s distribution. No need to build kernels for > > clients, and in many cases, no need to even build lustre. > > *** > There must be qualifications here? Are all RHEL 5 and SLES 10 (SPx) up > to being "patchless" level for clients?Yes. In fact, most any kernel beyond about 2.6.15 or 2.6.16 vintage that has not had invasive patches included such as Ubuntu''s recent kernels with AppArmor.> Is this really set in stone > for all levels of future Lustre, too?As in which versions of Linux the patchless client can be used with? I don''t know specific obligations and so on, but I would certainly think that as newer Linux kernel versions are released we will continue to support them. I''m not sure how recent we are currently supporting. Perhaps as recent as 2.6.22.> Not if you do not supply RDAC (mpp) kernels, too, e.g. for 6140?Can RDAC not be built as a module external to the kernel build? IOW, does RDAC require patching of the kernel? If it does support building "out of [kernel] tree" (i.e and does not require patching the kernel) then you still can use a Sun provided kernel. You just need to build RDAC against the also provided source tree. Perhaps this RDAC module is important enough that the Lustre Group provides it in or with our kernel builds. Perhaps not. I will leave that up to the business folks to decide.> > RHEL5 (and thus CentOS 5) comes with an OFED stack included in the OS. > > Old, especially for the linkes of ConnextX/Hermon HCA.I don''t know anything about those devices. They need OFED do they? What version works best for them? What version do you understand to be in the RHEL5 kernel such that it''s too old? 1.6.5 will support and we will supply packages for OFED 1.3 (on servers; the status of patchless clients is still to be determined).> You need to supply SRPMS/src tooWe do. We supply both kernel-lustre-source and lustre-source packages.> as some people (like Hillsboro HPC benchmark) > need to produce compatible PathScale/INTEL/PGI MPI versionsAgain, if these are all external module builds, it doesn''t require an actual binary kernel (re-)build. Our provided RPMs should suffice along with the kernel-lustre-source RPMs for building modules against.> (and maybe gcc > if you don''t build that per OS/gcc release level).We use whatever is current on the given OS.> Also the comments (above) about mpp > (RDAC) apply - you''d need modules built against that.Hopefully my comments answer to that too. If not we can clarify. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080429/9c14d4fd/attachment.bin