A post a few days back recommended that the MGS be placed on its own disk for all but "toy" setups I think was the comment. How much space does the MGS require? Is it based on the number of hosts? Or just the number of lustre file systems (we expect only 1 but maybe more in future). Last what''s the IO requirements of the MGS? I don''t think it would be much, such that could it share spindles with the journal for the MDS file system? Thanks! Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985
On Tue, 2008-06-17 at 11:09 -0400, Brock Palen wrote:> A post a few days back recommended that the MGS be placed on its own > disk for all but "toy" setups I think was the comment.Yeah.> How much space does the MGS require?Very little. I think we have discussed here that 100MB should be lots for now and the future. Perhaps somebody who knows more about what changes could be happening to configuration logs might know better.> Is it based on the number of > hosts? Or just the number of lustre file systems (we expect only 1 > but maybe more in future).Well, it would be both, but the amount of data per host (where a host is an OSS, MDS) is relatively small.> Last what''s the IO requirements of the MGS?Again, small. It''s used when a new OST or MDT is added to record their addition and when nodes join the filesystem or when a configuration change with a --writeconf is done.> I don''t think it would > be much, such that could it share spindles with the journal for the > MDS file system?Hrm. Given it''s relatively low use, I''d think that would be fine. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080617/42ec825d/attachment.bin
>> I don''t think it would >> be much, such that could it share spindles with the journal for the >> MDS file system? > > Hrm. Given it''s relatively low use, I''d think that would be fine. >I have a question ... if the MGS is used so infrequently relative to the use of the MDS, why is it (is it?) problematic to locate it on the same volume as the MDT? thanks, Klaus
On Tue, 2008-06-17 at 12:40 -0700, Klaus Steden wrote:> > I have a question ... if the MGS is used so infrequently relative to the use > of the MDS, why is it (is it?) problematic to locate it on the same volume > as the MDT?Our mountconf expert engineers probably know all of the reasons but one is that if you have multiple filesystems and you need to reformat the MDT that is shared with the MGS for some reason, you end up losing configuration logs for all of your filesystems. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080617/1a4cc7c9/attachment-0001.bin
On Jun 17, 2008 12:40 -0700, Klaus Steden wrote:> I have a question ... if the MGS is used so infrequently relative to the use > of the MDS, why is it (is it?) problematic to locate it on the same volume > as the MDT?If you have multiple MDTs on the same MDS node (i.e. multiple Lustre filesystems) then it is difficult to start up the MGS separately from the MDT if it is co-located with one of the MDTs. It isn''t impossible (with some manual mounting of the underlying filesystems) to move a co-located MGS to a separate filesystem if needed. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
Hi, I am very interested in finding out how to move co-located MGS to separate disk. I will be moving my MDTs to new hardware soon and I would like to separate MGS from MDT. I will be grateful for some info on this subject please. Many thanks, Wojciech 2008/6/23 Andreas Dilger <adilger at sun.com>> On Jun 17, 2008 12:40 -0700, Klaus Steden wrote: > > I have a question ... if the MGS is used so infrequently relative to the > use > > of the MDS, why is it (is it?) problematic to locate it on the same > volume > > as the MDT? > > If you have multiple MDTs on the same MDS node (i.e. multiple Lustre > filesystems) then it is difficult to start up the MGS separately from > the MDT if it is co-located with one of the MDTs. It isn''t impossible > (with some manual mounting of the underlying filesystems) to move a > co-located MGS to a separate filesystem if needed. > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-- -- Wojciech Turek Assistant System Manager High Performance Computing Service University of Cambridge Email: wjt27 at cam.ac.uk Tel: (+)44 1223 763517 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091010/f132885c/attachment.html
Hello Wojciech, I already did this several times, here are the steps I so far used: 1) Remove MGS from MDT-device tunefs.lustre --nogms /dec/mdt_device 2) Create new MGS mkfs.lustre --mgs /dev/mgs_device 3) Make sure OSTs and MDTs re-register with the MGS: tunefs.lustre --writeconf /dev/device I''m not sure if writeconf is really necessary, but so far I always did it to make sure everything goes smoothly (clients shouldn''t have the filesystem mounted at this step). 5) Mount MGS, MDT, OSTs 4) Re-apply settings done with lctl. As you also wrote (private mail), it might be possible to just copy over the CONFIGS directory, but I never tried to do that. Hope it helps, Bernd On Saturday 10 October 2009, Wojciech Turek wrote:> Hi, > > I am very interested in finding out how to move co-located MGS to separate > disk. I will be moving my MDTs to new hardware soon and I would like to > separate MGS from MDT. I will be grateful for some info on this subject > please. > > Many thanks, > > Wojciech > > 2008/6/23 Andreas Dilger <adilger at sun.com> > > > On Jun 17, 2008 12:40 -0700, Klaus Steden wrote: > > > I have a question ... if the MGS is used so infrequently relative to > > > the > > > > use > > > > > of the MDS, why is it (is it?) problematic to locate it on the same > > > > volume > > > > > as the MDT? > > > > If you have multiple MDTs on the same MDS node (i.e. multiple Lustre > > filesystems) then it is difficult to start up the MGS separately from > > the MDT if it is co-located with one of the MDTs. It isn''t impossible > > (with some manual mounting of the underlying filesystems) to move a > > co-located MGS to a separate filesystem if needed. > > > > Cheers, Andreas > > -- > > Andreas Dilger > > Sr. Staff Engineer, Lustre Group > > Sun Microsystems of Canada, Inc. > > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss at lists.lustre.org > > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-- Bernd Schubert DataDirect Networks
On Sun, 2009-10-11 at 19:05 +0200, Bernd Schubert wrote:>Bernd,> I already did this several times, here are the steps I so far used: > > 1) Remove MGS from MDT-device > > tunefs.lustre --nogms /dec/mdt_device > > 2) Create new MGS > > mkfs.lustre --mgs /dev/mgs_device > > 3) Make sure OSTs and MDTs re-register with the MGS: > > tunefs.lustre --writeconf /dev/device > > I''m not sure if writeconf is really necessary, but so far I always did it to > make sure everything goes smoothly (clients shouldn''t have the filesystem > mounted at this step). > > 5) Mount MGS, MDT, OSTs > > 4) Re-apply settings done with lctl. > > > As you also wrote (private mail), it might be possible to just copy over the > CONFIGS directory, but I never tried to do that.Could you file a bug in our bugzilla to get this procedure added to the manual? Thanx, b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091013/d450d6e9/attachment.bin