Hello So we are currently looking at upgrading one of our Lustre file system. The current filesystems: 2 MDS servers in a cold spare configuration over IB 2 Filesystems one /home and /scratch 8 OSS with 2 SAS JBODS each has 14x300GB 10K drives. 2 hot spares 2 OSTs per server. One segment is for /home and one for /scratch Proposed solution by hardware vendor. 8 OSS with 2 SAS JBODS each has 14x600GB 10K drives. 2 hot spares 2OSTs per server. Both segments would be added onto the /scratch file system. Max out the memory in the existing OSS systems to plan for future 1.8 upgrade I understand the potential administrative overhead in having OSTs that are larger than the old OSTs. I have not been supporting Lustre long enough to know if there are any major performance implications of this setup. My questions are: Will Lustre handle this type of configuration gracefully? Are there any other reasons that this may not be a good idea other than administrative overhead of having to watch the OSTs and make sure that they do not fill up? Are there any issues with having these new servers serve 2 segments of the same filesystem? Are there anything else I should consider that I may be missing? Thanks Sebastian
I would suggest to keep the OST size uniform that you migrate the existing OSTs to the new 600GB drive LUNs then combine pairs of (now unused) 300GB LUNs into double-sized OSTs to match the new ones. While the MDS will handle different-sized OSTs OK, it isn''t the ideal situation. Cheers, Andreas On 2010-04-09, at 20:17, Sebastian Gutierrez <gutseb at cs.stanford.edu> wrote:> Hello > So we are currently looking at upgrading one of our Lustre file > system. > > The current filesystems: > 2 MDS servers in a cold spare configuration over IB > 2 Filesystems one /home and /scratch > 8 OSS with 2 SAS JBODS each has 14x300GB 10K drives. 2 hot spares > 2 OSTs per server. One segment is for /home and one for /scratch > > Proposed solution by hardware vendor. > 8 OSS with 2 SAS JBODS each has 14x600GB 10K drives. 2 hot spares > 2OSTs per server. > Both segments would be added onto the /scratch file system. > Max out the memory in the existing OSS systems to plan for future 1.8 > upgrade > > I understand the potential administrative overhead in having OSTs that > are larger than the old OSTs. I have not been supporting Lustre long > enough to know if there are any major performance implications of this > setup. > > My questions are: > > Will Lustre handle this type of configuration gracefully? > Are there any other reasons that this may not be a good idea other > than > administrative overhead of having to watch the OSTs and make sure that > they do not fill up? > Are there any issues with having these new servers serve 2 segments of > the same filesystem? > Are there anything else I should consider that I may be missing? > > Thanks > Sebastian > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
On 2010-04-12, at 15:11, Norberto Meijome wrote:> On 12 April 2010 10:15, Andreas Dilger <andreas.dilger at oracle.com> > wrote: >> I would suggest to keep the OST size uniform that you migrate the >> existing OSTs to the new 600GB drive LUNs then combine pairs of (now >> unused) 300GB LUNs into double-sized OSTs to match the new ones. >> >> While the MDS will handle different-sized OSTs OK, it isn''t the ideal >> situation. > > Andreas, > could you please explain ( or point to information about) why it''s > better / ideal to have same-sized OSTs?It boils down to the fact that if there is uneven utilization of any resource in the system, then performance is being wasted to some extent. If we have to skip allocating objects on the smaller OSTs at some point (either pre-emptively because they are getting more full than the larger OSTs and we want to avoid over filling them, or because they have gotten full and we can no longer use them at all), then the OSS bandwidth will be underused at that time. That said, if you are not depending on the full bandwidth of all the OSTs all the time, then this may be OK. Cheers, Andreas -- Andreas Dilger Principal Engineer, Lustre Group Oracle Corporation Canada Inc.