Hi. I sent this to the Lustre dist and I don''t see my entry.
Do I need to do anything else to enable my post?
Joseph
-------- Original Message --------
Subject: New OST lost quotas
Date: Fri, 10 Apr 2009 15:29:50 -0700
From: Joseph Farran <jfarran at uci.edu>
Reply-To: jfarran at uci.edu
To: lustre-discuss at lists.lustre.org
Hi.
I am currently using Lustre 1.6.6, Redhat 5, with one MDT and two OST
and all is working fine over Infiniband. Group and user quota is also
working on Lustre.
I just added a new OST with the command:
mkfs.lustre --ost --fsname=lustre --param quota_type=ug
--mgsnode=nas-5-4-ib at o2ib /dev/sdc
The new OST is working, but quotas are gone. My question is, can I
dismount this new OST and run tunefs.lustre on just this OST:
tunefs.lustre --param ost.quota_type=ug /dev/sdc
Or do I need to take Lustre complete off-line and run the tunefs.lustre
on all OST and MTD again?
Thanks,
Joseph
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090414/20b0b5fa/attachment.html
Joseph,
you have to run "lfs quotacheck -ug" to create the required quota
files on the
new OST.
There is no need to run tunefs additionally (since you have already specified
the quota_type setting), but usually you can change permanent parameters
of running lustre filesystem with "lctl conf_param" from mgs.
mgs> lctl conf_param ${FSNAME}-MDT0000.mdt.quota_type=ug
mgs> lctl conf_param ${FSNAME}-OST0000.ost.quota_type=ug
mgs> lctl conf_param ${FSNAME}-OST0001.ost.quota_type=ug
...
By the way, we are going to change this non-obvious interface in the future
lustre releases.
Andrew.
On Tuesday 14 April 2009 20:20:23 Joseph Farran wrote:> Hi. I sent this to the Lustre dist and I don''t see my entry.
>
> Do I need to do anything else to enable my post?
>
> Joseph
>
> -------- Original Message --------
> Subject: New OST lost quotas
> Date: Fri, 10 Apr 2009 15:29:50 -0700
> From: Joseph Farran <jfarran at uci.edu>
> Reply-To: jfarran at uci.edu
> To: lustre-discuss at lists.lustre.org
>
>
>
> Hi.
>
> I am currently using Lustre 1.6.6, Redhat 5, with one MDT and two OST
> and all is working fine over Infiniband. Group and user quota is also
> working on Lustre.
>
> I just added a new OST with the command:
>
> mkfs.lustre --ost --fsname=lustre --param quota_type=ug
> --mgsnode=nas-5-4-ib at o2ib /dev/sdc
>
> The new OST is working, but quotas are gone. My question is, can I
> dismount this new OST and run tunefs.lustre on just this OST:
>
> tunefs.lustre --param ost.quota_type=ug /dev/sdc
>
> Or do I need to take Lustre complete off-line and run the tunefs.lustre
> on all OST and MTD again?
>
> Thanks,
> Joseph
Hi Andrew. First, thank you so much. I searched and searched, but some things are not very obvious to me. Questions please: When I run that command, does the Lustre file system need to be off-line (no writes, no appends, only read mode). If I only need to take that ost off-line, can I simply umounted it? What is the recommended way? One last un-related question: When I reboot my OST(s) and/or MetaData server, the servers go into the "recovery mode" which for us takes around ~40 minutes to complete. When the OST(s) are recovering and I am watching them, they seem to be idle simply waiting for a time out. Not really doing a fsck type of operation. Can I reduce this time to say 5 minutes? Thanks again, Joseph UCI Andrew Perepechko wrote:> Joseph, > > you have to run "lfs quotacheck -ug" to create the required quota files on the > new OST. > > There is no need to run tunefs additionally (since you have already specified > the quota_type setting), but usually you can change permanent parameters > of running lustre filesystem with "lctl conf_param" from mgs. > > mgs> lctl conf_param ${FSNAME}-MDT0000.mdt.quota_type=ug > mgs> lctl conf_param ${FSNAME}-OST0000.ost.quota_type=ug > mgs> lctl conf_param ${FSNAME}-OST0001.ost.quota_type=ug > ... > > By the way, we are going to change this non-obvious interface in the future > lustre releases. > > Andrew. > > On Tuesday 14 April 2009 20:20:23 Joseph Farran wrote: > >> Hi. I sent this to the Lustre dist and I don''t see my entry. >> >> Do I need to do anything else to enable my post? >> >> Joseph >> >> -------- Original Message -------- >> Subject: New OST lost quotas >> Date: Fri, 10 Apr 2009 15:29:50 -0700 >> From: Joseph Farran <jfarran at uci.edu> >> Reply-To: jfarran at uci.edu >> To: lustre-discuss at lists.lustre.org >> >> >> >> Hi. >> >> I am currently using Lustre 1.6.6, Redhat 5, with one MDT and two OST >> and all is working fine over Infiniband. Group and user quota is also >> working on Lustre. >> >> I just added a new OST with the command: >> >> mkfs.lustre --ost --fsname=lustre --param quota_type=ug >> --mgsnode=nas-5-4-ib at o2ib /dev/sdc >> >> The new OST is working, but quotas are gone. My question is, can I >> dismount this new OST and run tunefs.lustre on just this OST: >> >> tunefs.lustre --param ost.quota_type=ug /dev/sdc >> >> Or do I need to take Lustre complete off-line and run the tunefs.lustre >> on all OST and MTD again? >> >> Thanks, >> Joseph >> > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090521/8ea1f34a/attachment.html
Joseph, current usages may go slightly out of sync when doing quotacheck in parallel with writes/creations/deletions. Unmounting the clients or remounting them r/o can help. Recovery period length depends on the obd_timeout value. What is the value that you are using? Are any of the clients missing when you experience those timeouts? Note that you can abort recovery either with the lctl utility or by mounting the target with the abort_recov option (mount -o abort_recov). Andrew. On Thursday 21 May 2009 19:46:21 Joseph Farran wrote:> Hi Andrew. > > First, thank you so much. I searched and searched, but some things > are not very obvious to me. > > Questions please: When I run that command, does the Lustre file > system need to be off-line (no writes, no appends, only read mode). > > If I only need to take that ost off-line, can I simply umounted it? > What is the recommended way? > > One last un-related question: When I reboot my OST(s) and/or MetaData > server, the servers go into the "recovery mode" which for us takes > around ~40 minutes to complete. When the OST(s) are recovering and I > am watching them, they seem to be idle simply waiting for a time out. > Not really doing a fsck type of operation. Can I reduce this time to > say 5 minutes? > > Thanks again, > Joseph > UCI > > Andrew Perepechko wrote: > > Joseph, > > > > you have to run "lfs quotacheck -ug" to create the required quota files > > on the new OST. > > > > There is no need to run tunefs additionally (since you have already > > specified the quota_type setting), but usually you can change permanent > > parameters of running lustre filesystem with "lctl conf_param" from mgs. > > > > mgs> lctl conf_param ${FSNAME}-MDT0000.mdt.quota_type=ug > > mgs> lctl conf_param ${FSNAME}-OST0000.ost.quota_type=ug > > mgs> lctl conf_param ${FSNAME}-OST0001.ost.quota_type=ug > > ... > > > > By the way, we are going to change this non-obvious interface in the > > future lustre releases. > > > > Andrew. > > > > On Tuesday 14 April 2009 20:20:23 Joseph Farran wrote: > >> Hi. I sent this to the Lustre dist and I don''t see my entry. > >> > >> Do I need to do anything else to enable my post? > >> > >> Joseph > >> > >> -------- Original Message -------- > >> Subject: New OST lost quotas > >> Date: Fri, 10 Apr 2009 15:29:50 -0700 > >> From: Joseph Farran <jfarran at uci.edu> > >> Reply-To: jfarran at uci.edu > >> To: lustre-discuss at lists.lustre.org > >> > >> > >> > >> Hi. > >> > >> I am currently using Lustre 1.6.6, Redhat 5, with one MDT and two OST > >> and all is working fine over Infiniband. Group and user quota is also > >> working on Lustre. > >> > >> I just added a new OST with the command: > >> > >> mkfs.lustre --ost --fsname=lustre --param quota_type=ug > >> --mgsnode=nas-5-4-ib at o2ib /dev/sdc > >> > >> The new OST is working, but quotas are gone. My question is, can I > >> dismount this new OST and run tunefs.lustre on just this OST: > >> > >> tunefs.lustre --param ost.quota_type=ug /dev/sdc > >> > >> Or do I need to take Lustre complete off-line and run the tunefs.lustre > >> on all OST and MTD again? > >> > >> Thanks, > >> Joseph