For some reason this eludes my searches through the archives. I keep seeing documentation on how to deactive the OST and copy files off it. In the manual it states removal of the OST permanently can be accomplished with: lctl conf_param <OST name>.osc.active=0 I have run this and in the proc info I now see active is now set to 0. However it still exists in proc and when running lfs df it shows there as an inactive device. I''m wanting it removed from existence. The OST no longer physically exists yet I am haunted by its persistence on the MGS/MDT. TIA, Larry
On Thu, 2010-01-21 at 16:21 -0500, Larry Brown wrote:> > I''m wanting it removed from existence.You can''t. There is no documented and supported procedure for doing this. b.
I might be mistaken, but I thought this exact feature was added back in 1.6.x -----Original Message----- From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J. Murrell Sent: Friday, January 22, 2010 2:18 PM To: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Permanently delete OST On Thu, 2010-01-21 at 16:21 -0500, Larry Brown wrote:> > I''m wanting it removed from existence.You can''t. There is no documented and supported procedure for doing this. b. _______________________________________________ Lustre-discuss mailing list Lustre-discuss at lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
On Fri, 2010-01-22 at 14:45 -0700, Lundgren, Andrew wrote:> I might be mistaken, but I thought this exact feature was added back in 1.6.xWhich feature exactly? _Delete_ or deactivate? You can permanently (probably "persistently" is the better word) an OST, but you can''t outright delete it from the configuration (i.e. such that you could add a new one and it would take up the "slot" the old one had). b.
Level 3 requested this feature be developed in 1.6. The after Sun did some work for us, the following is the procedure that we have setup for usage. IIRC the functionality was enabled in 1.6.7. We have tested it in 1.8.0. (We have done it in our development and test lab, but never in production, as we haven''t needed it yet.) *On the MDS, make the affected (missing) OST read only by deactivating it. *On a client, find all of the files that are now missing/damaged using the lfs getstripe --odb OST_NAME recursively on the impacted filesystem. (save for removal and restoration) *Remove all of the damaged files from luster using unlink on each filename found above. *On the MDS, permanently deactivate the OST that is gone using the --perm flag on the deactivate command. *Format out the replacement OST using the index number that you just deactivated. *On the MDS, activate the OST again using the --perm flag. *Mount the OST back into the filesystem. *Restore the files that were lost from the list created above.> -----Original Message----- > From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss- > bounces at lists.lustre.org] On Behalf Of Brian J. Murrell > Sent: Friday, January 22, 2010 2:53 PM > To: lustre-discuss at lists.lustre.org > Subject: Re: [Lustre-discuss] Permanently delete OST > > On Fri, 2010-01-22 at 14:45 -0700, Lundgren, Andrew wrote: > > I might be mistaken, but I thought this exact feature was added back > in 1.6.x > > Which feature exactly? _Delete_ or deactivate? You can permanently > (probably "persistently" is the better word) an OST, but you can''t > outright delete it from the configuration (i.e. such that you could add > a new one and it would take up the "slot" the old one had). > > b. > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss
On Mon, 2010-01-25 at 11:38 -0700, Lundgren, Andrew wrote:> Level 3 requested this feature be developed in 1.6. The after Sun did some work for us, the following is the procedure that we have setup for usage. IIRC the functionality was enabled in 1.6.7. We have tested it in 1.8.0.Do you recall what bug number, if any, this work was done under? I am searching but if you can save me some time... b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100125/c33eb2e3/attachment.bin
15345 iirc -----Original Message----- From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Brian J. Murrell Sent: Monday, January 25, 2010 1:20 PM To: lustre-discuss at lists.lustre.org Subject: Re: [Lustre-discuss] Permanently delete OST On Mon, 2010-01-25 at 11:38 -0700, Lundgren, Andrew wrote:> Level 3 requested this feature be developed in 1.6. The after Sun did some work for us, the following is the procedure that we have setup for usage. IIRC the functionality was enabled in 1.6.7. We have tested it in 1.8.0.Do you recall what bug number, if any, this work was done under? I am searching but if you can save me some time... b.
On Tue, 2010-01-26 at 08:23 -0700, Lundgren, Andrew wrote:> 15345 iircYes, I looked at that bug yesterday. I don''t see anything in there that provides any sort of "--perm" argument to completely purge an OST from the configuration. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100126/91046be5/attachment.bin
On Tue, 2010-01-26 at 11:29 -0500, Brian J. Murrell wrote:> Yes, I looked at that bug yesterday. I don''t see anything in there that > provides any sort of "--perm" argument to completely purge an OST from > the configuration. > > b.What is the latest on this? Also, after a system reboot, at the point the first "permanently" inactive OST would be listed in "lfs df" the output stops with the line "error: llapi_obd_statfs failed: Bad address (-14)". This looks like a bug mentioned earlier in the 1.6.6 version of Lustre. I am running 1.8.1.1. Larry
On Thu, 2010-01-28 at 15:22 -0500, Larry Brown wrote:> On Tue, 2010-01-26 at 11:29 -0500, Brian J. Murrell wrote: > > > Yes, I looked at that bug yesterday. I don''t see anything in there that > > provides any sort of "--perm" argument to completely purge an OST from > > the configuration. > > > > b. > > What is the latest on this?Well, I''ve searched and I cannot reconcile Andrew''s report of a "--perm" option with any of our code or bugs so I''m not sure where that came from. Given that, we have no supported method of permanently scrubbing the existence of an OST from the configuration. I think the closest you are going to get is what is detailed in the bug Andrew mentioned previously in this thread.> Also, after a system reboot, at the point > the first "permanently" inactive OST would be listed in "lfs df" the > output stops with the line "error: llapi_obd_statfs failed: Bad address > (-14)". This looks like a bug mentioned earlier in the 1.6.6 version of > Lustre. I am running 1.8.1.1.Sorry. This is not ringing any bells for me right now and I''m afraid I just don''t have the bandwidth to dig into it right now. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100128/fff29557/attachment.bin
Am Donnerstag 28 Januar 2010 21:33:28 schrieb Brian J. Murrell:> > Also, after a system reboot, at the point > > the first "permanently" inactive OST would be listed in "lfs df" the > > output stops with the line "error: llapi_obd_statfs failed: Bad address > > (-14)". This looks like a bug mentioned earlier in the 1.6.6 version of > > Lustre. I am running 1.8.1.1. > > Sorry. This is not ringing any bells for me right now and I''m afraid I > just don''t have the bandwidth to dig into it right now.https://bugzilla.lustre.org/show_bug.cgi?id=18329 According to dev team resolved and closed .... Regards Heiko
> https://bugzilla.lustre.org/show_bug.cgi?id=18329 > > According to dev team resolved and closed .... > > Regards > Heiko > _______________________________________________That shows the version being 1.6.6 but also that it was resolved 1/22/10 which is very recent. This bug must be in the 1.8 tree (as that is my current version) and was not noticed? Would this fix automatically be applied to the 1.8 tree procedurally or does a separate bug have to be reported? Also, if it has been fixed in 1.8 is there a scheduled release? Thanks, as this will be nice to be able to tell what disk capacity is available. Larry
I''m seeing the same messages on my OSS. Lustre 1.8.1.1 Erik On Fri, Jan 29, 2010 at 8:26 AM, Larry Brown < larry.brown at dimensionnetworks.com> wrote:> > > https://bugzilla.lustre.org/show_bug.cgi?id=18329 > > > > According to dev team resolved and closed .... > > > > Regards > > Heiko > > _______________________________________________ > > That shows the version being 1.6.6 but also that it was resolved 1/22/10 > which is very recent. This bug must be in the 1.8 tree (as that is my > current version) and was not noticed? Would this fix automatically be > applied to the 1.8 tree procedurally or does a separate bug have to be > reported? Also, if it has been fixed in 1.8 is there a scheduled > release? > > Thanks, as this will be nice to be able to tell what disk capacity is > available. > > Larry > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100129/c466fde2/attachment.html
> > https://bugzilla.lustre.org/show_bug.cgi?id=18329 > > > > According to dev team resolved and closed .... > > > > Regards > > HeikoBy the way, I got a hold of the source code and the patch listed for that bug listed in Heiko''s message. I introduced the changes to the lfs.c file and now I no longer get the error code. That patch does work but has not been applied to 1.8.1.1 in the released rpms. Still no option for permanent removal, but at least lfs df works again. Larry
I went back with our guys internally and reviewed the "--perm" parameter. It turns out that is a parameter that was put into our wrapper scripts to simplify the process. We are doing it today, but Brian is correct that the "perm" parameter doesn''t exist in the luster software. If needed, I can go back into our scripts and dig out the actual luster commands we are using to do this. It is worth noting that we are NOT leaving a dead point in the OST list and are replacing a bad OST id # with a new one. -- Andrew
How does the normal df -h behave? Even with the ost "permanently" deleted from the mds using the conf_param method, df -h hangs on the clients. After disabling the OST locally with the set_param method it stops hanging and reports the correct fs size. Oddly when I re-enable the OST on the mds the hang goes away even though the OST isn''t mounted anywhere. Erik On Fri, Jan 29, 2010 at 1:51 PM, Larry Brown < larry.brown at dimensionnetworks.com> wrote:> > > > https://bugzilla.lustre.org/show_bug.cgi?id=18329 > > > > > > According to dev team resolved and closed .... > > > > > > Regards > > > Heiko > > By the way, I got a hold of the source code and the patch listed for > that bug listed in Heiko''s message. I introduced the changes to the > lfs.c file and now I no longer get the error code. That patch does work > but has not been applied to 1.8.1.1 in the released rpms. > > Still no option for permanent removal, but at least lfs df works again. > > Larry > > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100129/5ff3f728/attachment.html