Paul Kapinos
2009-Aug-17 09:54 UTC
[Linux_hpc_swstack] The hard limit of file descriptors on Limix and SunMPI
Dear Sun HPC Software Stack folks, we see an eventually problem in the HPC Sw Stack comming in the close future. Currently, we use the Sun HPC Sw Stack running Sun Linux computers, including the Sun Cluster Tools 8.2 MPI (which is OpenMP 1.3.3). $ ulimit -n telly you the number of file descriptors per process to use. On our systems with the HPCSwStack kernel installed this limit is set to 1024 and this is a hard limit (only root may set it to a higher value). We found out that currently, maximally 84 MPI processes may be used within one box (a Linux OS instance) because of exceeding usage of file descriptors through the mpiexec process (about 12 file descriptors per child MPI process). Today it is not a really problem, but we have ordered 128-way systhems... We submitted an message on the OpenMPI mailing list and Rolf Vandervaart means he would look why the (over-)consumption of file descriptors arises. But, we knocked not once on the "sonic barrier" of hard! limit of only 1024 file descriptors in the past. (On Solaris, the hard limit is by 64k !!!). Is it not an option, to set the hard limit for the number of file descriptors to some higher value in the future releases of Sun HPC Software stack? (we know about the possibility to adjust the number over limits.conf but this works only if root rights are used somewhere somehow... and, why workarounding if there is a native way?) Best regards, Paul Kapinos RZ RWTH Aachen -------------- next part -------------- A non-text attachment was scrubbed... Name: kapinos.vcf Type: text/x-vcard Size: 330 bytes Desc: not available Url : http://lists.lustre.org/pipermail/linux_hpc_swstack/attachments/20090817/5b8f4c78/attachment.vcf -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 4230 bytes Desc: S/MIME Cryptographic Signature Url : http://lists.lustre.org/pipermail/linux_hpc_swstack/attachments/20090817/5b8f4c78/attachment.bin