Lustre discuss - Mar 2009 - Job fails opening 24k files and keeps them open during execution

Hello,

 

I am working with a user that has  Fortran code that opens 24.000 files and
keeps them open during execution.  We had to adjust our kernel parameters to
allow this to happen, since Linux cuts you off at 1024 by default.  This is job
runs successfully for him on the local disk of a Linux machine, but when he
moves the job to Lustre, it fails.  The metadata servers are running Red Hat ?do
they impose their own user limitations as well?  Or is there a limitation within
Lustre or a config file?

 

Thanks in advance,

Oz

 

 

 

 


No virus found in this outgoing message.
Checked by AVG. 
Version: 7.5.557 / Virus Database: 270.11.3/1975 - Release Date: 2/27/2009 7:05
AM
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090304/6b9a2337/attachment.html

On Wed, 2009-03-04 at 16:27 -0500, Osvaldo Rentas wrote:
> Hello,
> 
>  
> 
> I am working with a user that has  Fortran code that opens 24.000
> files and keeps them open during execution.  We had to adjust our
> kernel parameters to allow this to happen, since Linux cuts you off at
> 1024 by default.  This is job runs successfully for him on the local
> disk of a Linux machine, but when he moves the job to Lustre, it
> fails.  The metadata servers are running Red Hat ?do they impose their
> own user limitations as well?  Or is there a limitation within Lustre
> or a config file? 
Fails in what way?  You don''t say anything other than "it
fails".  We
can''t even begin to try to guess what the problem is without a better
description of what the failure looks like.

b.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090304/426a25e8/attachment-0001.bin

Lustre does not impose maximum number of open files, but practically it
depends on amount of RAM on the MDS. 

There are no "tables" for open files on the MDS, as they are only
linked
in a list to a given client''s export. 

Each client process probably has a limit of several thousands of open
files which depends on the ulimit.

? 2009-03-04?? 16:27 -0500?Osvaldo Rentas???
> Hello,
> 
>  
> 
> I am working with a user that has  Fortran code that opens 24.000
> files and keeps them open during execution.  We had to adjust our
> kernel parameters to allow this to happen, since Linux cuts you off at
> 1024 by default.  This is job runs successfully for him on the local
> disk of a Linux machine, but when he moves the job to Lustre, it
> fails.  The metadata servers are running Red Hat ?do they impose their
> own user limitations as well?  Or is there a limitation within Lustre
> or a config file? 
> 
>  
> 
> Thanks in advance,
> 
> Oz
> 
>  
> 
>  
> 
>  
> 
>  
> 
> 
> 
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.557 / Virus Database: 270.11.3/1975 - Release Date:
> 2/27/2009 7:05 AM
> 
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss