Lustre discuss - May 2006 - What is the limitation of Lustre filesystem?

Do, Jong Gwan wrote:
> For example, what is the following limitations?
> 
> 1. The maximum filename and pathname length.
These limits are the same as in ext3, which I believe to be 255 bytes
and 4096 bytes, respectively.
> 2. The maximum capacity of a single lustre filesystem.
This is limited by available hardware.  Linux imposes a 2TB limit per
block device, but Lustre can use many (hundreds or thousands) of such
block devices to store data in a single file system.

One of our customers has 104 object storage servers in a single FS, and
all indications are that Lustre would scale just fine if more hardware
were available.
> 3. The maximum number of directories and files in a single directory.
We use the ext3 hashed directory code, which has a meaningless
theoretical limit of ~130 trillion files, at which point the directory
grows to more than 2GB.  The maximum number of subdirectories is
currently ~32,000.

More realistically, we have run tests with ten of million files in a
single directory.  On a properly-configured dual-CPU MDS with 4 GB of
ram, it is possible to do random lookups in this directory at a rate of
5,000/second.

A single MDS imposes an upper limit of 4B inodes, but the actual limit
is slightly less than the device size / 8kB, so about 256M inodes for a
2TB MDS filesystem.  This can be increased at initial filesystem
creation time by specifying mount options, but it can have a performance
impact if your files have large numbers of stripes.
> 4. The maximum size of a single file.
Individual files have a hard limit of nearly 16TB on 32-bit systems
imposed by the kernel memory subsystem.  On 64-bit systems this limit
does not exist and files can be fully 64-bits in size.

Lustre imposes an additional limit of 2TB * number_of_stripes; a single
file can have a maximum of 162 stripes in the 1.0 release of Lustre,
which gives an upper single file limit of 264TB for 64-bit systems.  The
actual amount of data that can be stored in a file is dependent upon the
amount of free space in each OST that a file is striped over.

-Phil

-------0ESF8G5J0ERQMQX39P4R5KAET8Q46XG
Content-Type: text/plain;
	charset=EUC-KR
Content-Transfer-Encoding: QUOTED-PRINTABLE

Hi everyone,

What is the limitation of Lustre filesystem?
For example, what is the following limitations?

1=2E The maximum filename and pathname length=2E
2=2E The maximum capacity of a single lustre filesystem=2E
3=2E The maximum number of directories and files in a single directory=2E
4=2E The maximum size of a single file=2E
5=2E Some other limitations related to numerical limits=2E

Regards=2E

J=2E G=2E
=0A=0A

-------0ESF8G5J0ERQMQX39P4R5KAET8Q46XG
Content-Type: text/html;
	charset=EUC-KR
Content-Transfer-Encoding: QUOTED-PRINTABLE

<style type=3D"text/css">
<!--
TT { font-size: 10pt}
-->
</style>
<TT>
Hi everyone,
<BR>=0A
<BR>=0AWhat is the limitation of Lustre filesystem?
<BR>=0AFor example, what is the following limitations?
<BR>=0A
<BR>=0A1=2E The maximum filename and pathname length=2E
<BR>=0A2=2E The maximum capacity of a single lustre filesystem=2E
<BR>=0A3=2E The maximum number of directories and files in a single
directory=2E
<BR>=0A4=2E The maximum size of a single file=2E
<BR>=0A5=2E Some other limitations related to numerical limits=2E
<BR>=0A
<BR>=0ARegards=2E
<BR>=0A
<BR>=0AJ=2E G=2E
<BR>=0A</TT>
=0A

-------0ESF8G5J0ERQMQX39P4R5KAET8Q46XG--