Lustre discuss - Oct 2009 - Deactivated OSTs and object placement

Dear list,

when I

- create a file striped over all OSTs
- deactivate one OST (lctl set_param osc....active=0) on the MDS
- open, write, close that file

I have no more objects of that file on the deactivated OST. I also
noticed that all object IDs have changed.

Equivalent behavior when I re-activate the OST (and do the open,
write, close cycle): the file is fully striped again.

Is Lustre recreating my file, i.e. re-shuffling the objects?

TIA,
  Arne
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 6380 bytes
Desc: S/MIME Cryptographic Signature
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20091002/98c34b6b/attachment.bin

On Oct 02, 2009  16:27 +0200, Arne Wiebalck wrote:
> when I
>
> - create a file striped over all OSTs
> - deactivate one OST (lctl set_param osc....active=0) on the MDS
> - open, write, close that file
>
> I have no more objects of that file on the deactivated OST. I also
> noticed that all object IDs have changed.
>
> Equivalent behavior when I re-activate the OST (and do the open,
> write, close cycle): the file is fully striped again.
>
> Is Lustre recreating my file, i.e. re-shuffling the objects?
It depends on how you are doing the "open, write, close" step.

I would guess that you are using an editor or similar, and it is
creating a new file (which will not be striped over the deactivated
OST), writing the data to the new file, and then deleting the old
file when the new one is successfully written.

You could check this relatively easily by doing "ls -li {filename}"
before and after your test, and if the inode number is different
then your test has created a new file.

Lustre today will unfortunately not restripe a file after it is
created.  In Lustre 2.1 or so we will start having the ability to
restripe a file after it is created.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.