Lustre discuss - Mar 2013 - will obdfilter-survey destroy an already formatted file system

Hi,

I have read through the documentation for obdfilter-survey but could not found
any information on how invasive the test is. Will it destroy an already
formatted OST or render user data unusable?


Regards, Michael

--
Dr.-Ing. Michael Kluge

Technische Universit?t Dresden
Center for Information Services and
High Performance Computing (ZIH)
D-01062 Dresden
Germany

Contact:
Willersbau, Room WIL A 208
Phone:  (+49) 351 463-34217
Fax:    (+49) 351 463-37773
e-mail: michael.kluge at tu-dresden.de
WWW:    http://www.tu-dresden.de/zih

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130321/10c40633/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4630 bytes
Desc: not available
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20130321/10c40633/attachment.bin

Hi Michael,

You can safely run obdfilter-survey on an existing file system.

-cf

On 03/21/2013 04:09 AM, Michael Kluge wrote:
> Hi,
>
> I have read through the documentation for obdfilter-survey but could 
> not found any information on how invasive the test is. Will it destroy 
> an already formatted OST or render user data unusable?
>
>
> Regards, Michael
>
> --
> Dr.-Ing. Michael Kluge
>
> Technische Universit?t Dresden
> Center for Information Services and
> High Performance Computing (ZIH)
> D-01062 Dresden
> Germany
>
> Contact:
> Willersbau, Room WIL A 208
> Phone:  (+49) 351 463-34217
> Fax:    (+49) 351 463-37773
> e-mail: michael.kluge at tu-dresden.de <mailto:michael.kluge at
tu-dresden.de>
> WWW: http://www.tu-dresden.de/zih
>
>
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

On 2013/21/03 4:09 AM, "Michael Kluge" <Michael.Kluge at
tu-dresden.de> wrote:
>I have read through the documentation for obdfilter-survey but could not
>found any information on how invasive the test is. Will it destroy an
>already formatted OST or render user data unusable?
It shouldn''t - the obdfilter-survey uses a different object sequence
(2)
compared to normal filesystem objects (currently always 0), so the two do
not collide.

Cheers, Andreas
-- 
Andreas Dilger

Lustre Software Architect
Intel High Performance Data Division

Michael,

Unfortunately, the current Lustre Ops Manual indicates the opposite.  From
section 24.3 "Testing OST Performance (obdfilter_survey)":

"The obdfilter_survey script is destructive and should not be run on
devices that containing existing data that needs to be preserved. Thus,
tests using obdfilter_survey should be run before the Lustre file system
is placed in production."

I opened LUDOC-146 to track the issue previously and updated the details
to include Andreas'' explanation.

Thanks,
-Cory


On 3/21/13 7:18 PM, "Dilger, Andreas" <andreas.dilger at
intel.com> wrote:
>On 2013/21/03 4:09 AM, "Michael Kluge" <Michael.Kluge at
tu-dresden.de>
>wrote:
>>I have read through the documentation for obdfilter-survey but could not
>>found any information on how invasive the test is. Will it destroy an
>>already formatted OST or render user data unusable?
>
>It shouldn''t - the obdfilter-survey uses a different object
sequence (2)
>compared to normal filesystem objects (currently always 0), so the two do
>not collide.
>
>Cheers, Andreas
>-- 
>Andreas Dilger
>
>Lustre Software Architect
>Intel High Performance Data Division
>
>
>_______________________________________________
>HPDD-discuss mailing list
>HPDD-discuss at lists.01.org
>https://lists.01.org/mailman/listinfo/hpdd-discuss

Hi Cory,

I am running this stuff now since a few weeks. Only a few users are using the
file system up to now. Either I was lucky or Andreas is right. No one has
complained yet that data got lost. I am running integrity checks in parallel and
they did not find anything yet. So we can say it is "most probably
safe" :)


Regards, Michael

> Michael,
> 
> Unfortunately, the current Lustre Ops Manual indicates the opposite.  From
> section 24.3 "Testing OST Performance (obdfilter_survey)":
> 
> "The obdfilter_survey script is destructive and should not be run on
> devices that containing existing data that needs to be preserved. Thus,
> tests using obdfilter_survey should be run before the Lustre file system
> is placed in production."
> 
> I opened LUDOC-146 to track the issue previously and updated the details
> to include Andreas'' explanation.
> 
> Thanks,
> -Cory
> 
> 
> On 3/21/13 7:18 PM, "Dilger, Andreas" <andreas.dilger at
intel.com> wrote:
> 
>> On 2013/21/03 4:09 AM, "Michael Kluge" <Michael.Kluge at
tu-dresden.de>
>> wrote:
>>> I have read through the documentation for obdfilter-survey but
could not
>>> found any information on how invasive the test is. Will it destroy
an
>>> already formatted OST or render user data unusable?
>> 
>> It shouldn''t - the obdfilter-survey uses a different object
sequence (2)
>> compared to normal filesystem objects (currently always 0), so the two
do
>> not collide.
>> 
>> Cheers, Andreas
>> -- 
>> Andreas Dilger
>> 
>> Lustre Software Architect
>> Intel High Performance Data Division
>> 
>> 
>> _______________________________________________
>> HPDD-discuss mailing list
>> HPDD-discuss at lists.01.org
>> https://lists.01.org/mailman/listinfo/hpdd-discuss
>