Hello,
I would like to adjust the stripe count for our lustre filesystem. Would
it be better to:
1) Kill all jobs, unmount the lustre filesystem from all clients, and then
adjust the default stripe count for the lustre filesystem on the MDS
using lctl.
or
2) Use "find" and the "lfs setstripe" command to traverse
and set the
stripe count for all directories in a currently mounted lustre
filesystem.
Besides the traversal cost of the filesystem, are there other
disadvantages, performance reasons, or other reasons not to use option 2?
thanks
-k
I don''t think you need to do this. If i understand right, you can set the stripe size of the mount, and everything inside that directory inherits it, unless they them self''s were explicitly set. Also files that already are created will keep the stripe settings they were created with. You would need to copy them, and move over the old one to change to the new stripe settings. Check the lustre manual they have something about this. You can use ''getstripe'' to see what a file/directory use for their settings. Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985 On Apr 17, 2008, at 10:34 AM, Kaizaad Bilimorya wrote:> Hello, > > I would like to adjust the stripe count for our lustre filesystem. > Would > it be better to: > > 1) Kill all jobs, unmount the lustre filesystem from all clients, > and then > adjust the default stripe count for the lustre filesystem on > the MDS > using lctl. > > or > > 2) Use "find" and the "lfs setstripe" command to traverse and set the > stripe count for all directories in a currently mounted lustre > filesystem. > > > Besides the traversal cost of the filesystem, are there other > disadvantages, performance reasons, or other reasons not to use > option 2? > > thanks > -k > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss > >
On Thu, 17 Apr 2008, Brock Palen wrote:> I don''t think you need to do this. > If i understand right, you can set the stripe size of the mount, and > everything inside that directory inherits it, unless they them self''s were > explicitly set.Hi Brock, thanks for the reply. I have set the stripe count on the lustre mount using "lfs setstripe" but the problem is that any sub directories that already existed under this mount will have the default filesystem stripe count and not the new one I set, so any new files created under these existent sub directories will inherit their parent directory stripe count and not the newly set one from the lustre mount. eg: /lustremount -> "lfs setstripe /lustremount 0 -1 2" /lustremount/existing_dir -> has filesystem default stripe count (1 in this case) /lustremount/new_dir -> gets stripe count of parent (2 in this case) /lustremount/existing_dir/newfile -> has filesystem default stripe count of 1 So that is why I have to do either option 1 (change default) or 2 (traverse and set explicitly for all existing dirs) that I specified, but I would like to know if there are any performance or other reasons not to do option 2. thanks -k> Also files that already are created will keep the stripe settings they were > created with. You would need to copy them, and move over the old one to > change to the new stripe settings. Check the lustre manual they have > something about this. > > On Apr 17, 2008, at 10:34 AM, Kaizaad Bilimorya wrote: >> Hello, >> >> I would like to adjust the stripe count for our lustre filesystem. Would >> it be better to: >> >> 1) Kill all jobs, unmount the lustre filesystem from all clients, and then >> adjust the default stripe count for the lustre filesystem on the MDS >> using lctl. >> >> or >> >> 2) Use "find" and the "lfs setstripe" command to traverse and set the >> stripe count for all directories in a currently mounted lustre >> filesystem. >> >> >> Besides the traversal cost of the filesystem, are there other >> disadvantages, performance reasons, or other reasons not to use option 2? >> >> thanks >> -k >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> >> >
On Apr 17, 2008, at 10:48 PM, Kaizaad Bilimorya wrote:> > On Thu, 17 Apr 2008, Brock Palen wrote: >> I don''t think you need to do this. >> If i understand right, you can set the stripe size of the mount, >> and everything inside that directory inherits it, unless they them >> self''s were explicitly set. > > Hi Brock, thanks for the reply. > > I have set the stripe count on the lustre mount using "lfs > setstripe" but the problem is that any sub directories that already > existed under this mount will have the default filesystem stripe > count and not the new one I set, so any new files created under > these existent sub directories will inherit their parent directory > stripe count and not the newly set one from the lustre mount.Ahh I see, I really don''t know, I think try walking the system and changing all the old directories. I have not had to do this my self.> > eg: > > /lustremount -> "lfs setstripe /lustremount 0 -1 2" > /lustremount/existing_dir -> has filesystem default stripe count (1 > in this case) > /lustremount/new_dir -> gets stripe count of parent (2 in this case) > /lustremount/existing_dir/newfile -> has filesystem default stripe > count of 1 > > So that is why I have to do either option 1 (change default) or 2 > (traverse and set explicitly for all existing dirs) that I > specified, but I would like to know if there are any performance or > other reasons not to do option 2. > > thanks > -k > >> Also files that already are created will keep the stripe settings >> they were created with. You would need to copy them, and move >> over the old one to change to the new stripe settings. Check the >> lustre manual they have something about this. >> >> On Apr 17, 2008, at 10:34 AM, Kaizaad Bilimorya wrote: >>> Hello, >>> I would like to adjust the stripe count for our lustre >>> filesystem. Would >>> it be better to: >>> 1) Kill all jobs, unmount the lustre filesystem from all clients, >>> and then >>> adjust the default stripe count for the lustre filesystem on >>> the MDS >>> using lctl. >>> or >>> 2) Use "find" and the "lfs setstripe" command to traverse and set >>> the >>> stripe count for all directories in a currently mounted lustre >>> filesystem. >>> Besides the traversal cost of the filesystem, are there other >>> disadvantages, performance reasons, or other reasons not to use >>> option 2? >>> thanks >>> -k >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-discuss >> > >
On Apr 18, 2008 11:02 -0400, Brock Palen wrote:> On Apr 17, 2008, at 10:48 PM, Kaizaad Bilimorya wrote: > > > > On Thu, 17 Apr 2008, Brock Palen wrote: > >> I don''t think you need to do this. > >> If i understand right, you can set the stripe size of the mount, > >> and everything inside that directory inherits it, unless they them > >> self''s were explicitly set. > > > > Hi Brock, thanks for the reply. > > > > I have set the stripe count on the lustre mount using "lfs > > setstripe" but the problem is that any sub directories that already > > existed under this mount will have the default filesystem stripe > > count and not the new one I set, so any new files created under > > these existent sub directories will inherit their parent directory > > stripe count and not the newly set one from the lustre mount.Note that this is a new feature in 1.6. I''m not sure if it was in the 1.6.4 release or only 1.6.5, but setting the striping on the root directory will change the filesystem-wide default striping parameters, immediately affecting all directories that do not have their own striping set. This was considered a better interface for setting the default striping than having to rewrite the MDS configuration file using a different mechanism. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.