Lustre discuss - Sep 2009 - Removing an OST from the filesystem

In the Lustre manual 4.2.10.1 "Removing an OST from the File System",
it
  tells you to how to deactivate the OSC on the MDT so no files are 
written to the OST, then says:

3 Use "lfs find" to discover all files that have objects residing on
the
deactivated OST.

4 Copy (not move) the files to a new directory in the file system.
Copying the files forces object re-creation on the active OSTs

5 Move (not copy) the files back to their original directory on in the 
file system.
Moving the files causes the original files to be deleted, as the copies 
replace them.

I can''t convince myself there isn''t a race condition here if a
file is
being written to during this process.

What happens if a file is being written to when I mark the OST offline, 
and continues to be written to during the copy and move?

Chris

On Sat, 2009-09-19 at 22:22 +0200, Christopher J. Walker
wrote:
> In the Lustre manual 4.2.10.1 "Removing an OST from the File
System", it
>   tells you to how to deactivate the OSC on the MDT so no files are 
> written to the OST, then says:
> I can''t convince myself there isn''t a race condition here
That''s good because there are race conditions in this operation.
> if a file is 
> being written to during this process.
Yup.
> What happens if a file is being written to when I mark the OST offline, 
> and continues to be written to during the copy and move?
When you mark the OST as inactive on the MDS, the only thing you are
doing is telling the MDS not to assign *new* objects to this OST.  That
does not at all prevent use of existing objects, whether that is for
read or write.

If you really want to prevent all I/O to the OST you need to deactivate
it on the clients too.  This of course will provide the clients with
EIOs to I/O operations to that OST.

This operation of moving files off an OST really is best done with the
filesystem "quiesced".

There were discussions at our last engineering summit of features which
would provide us the ability to close such race conditions, but they
were only discussions and I don''t know if any actual design and/or code
has resulted from them.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090920/111a7bd7/attachment.bin