Lustre discuss - May 2010 - What would happen if

Hi,

My Lustre system has 7 OSTs on 3 OSSs.   2 of the OSTs have less then 200Gb free
while the other 5
have 3+ TB free.

I am not using striping.  Question, when creating a new file does Lustre try to
write to the OST
with the most free space?  If not, how does it choose which OST to write to?

Second question, if for whatever reason it picks the OST with 200Gb free and i
was writing a file
that would be 500Gb at finish, what would happen when that particular OST ran
out of space?  Would
it leave the file as incomplete and show no free space?

Thanks, trying to debug an odd problem.

On Wed, 2010-05-26 at 09:26 -0400, Scott wrote: 
> Hi,
Hi,

You don''t provide any particulars (lustre version, lfs df output, etc.)
so I will answer generically...
> I am not using striping.  Question, when creating a new file does Lustre
try to write to the OST
> with the most free space?
That depends on your lustre version and the values
in /proc/fs/lustre/lov/client-clilov-f385fc00/qos_*.  Please see the
manual for details on those files and how they affect object allocation,
if you are running a relevant version of Lustre.
> Second question, if for whatever reason it picks the OST with 200Gb free
and i was writing a file
> that would be 500Gb at finish, what would happen when that particular OST
ran out of space?
The client gets an ENOSPC.
> Would 
> it leave the file as incomplete and show no free space?
If the client application doesn''t do any cleanup on getting an error,
yes.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20100526/50177eee/attachment.bin

If you are creating files that are a significant fraction of the free 
space on your OSTs, you _should_ stripe
them across multiple OSTs (lfs setstripe).

Also note that ENOSPC is likely to be returned before the OST is 
actually "out" of space: due to the
OST pre-granting space to the clients, ENOSPC is returned when (free - 
grants) = 0.  The more Lustre
clients you have, the sooner this occurs.  See Bug 22498

Additionally, there are some grant leaks that can cause the server''s 
view of the grant to be inflated;
see Bug 22755

Kevin


Brian J. Murrell wrote:
> On Wed, 2010-05-26 at 09:26 -0400, Scott wrote: 
>   
>> Hi,
>>     
>
> Hi,
>
> You don''t provide any particulars (lustre version, lfs df output,
etc.)
> so I will answer generically...
>
>   
>> I am not using striping.  Question, when creating a new file does
Lustre try to write to the OST
>> with the most free space?
>>     
>
> That depends on your lustre version and the values
> in /proc/fs/lustre/lov/client-clilov-f385fc00/qos_*.  Please see the
> manual for details on those files and how they affect object allocation,
> if you are running a relevant version of Lustre.
>
>   
>> Second question, if for whatever reason it picks the OST with 200Gb
free and i was writing a file
>> that would be 500Gb at finish, what would happen when that particular
OST ran out of space?
>>     
>
> The client gets an ENOSPC.
>
>   
>> Would 
>> it leave the file as incomplete and show no free space?
>>     
>
> If the client application doesn''t do any cleanup on getting an
error,
> yes.
>
> b.
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>