Hi Rob--
On 12/15/2004 12:13, Rob Martin wrote:>
> You know the ''.'' file in a directory? When you add more
files, the size of
> the ''.'' file grows - I assume is contains an index to the
inodes of the
> files in a directory but not sure on that.
This is correct. The directory contains a mapping from names to inodes; in
the version of ext3 that we include with Lustre, it uses a tree to make
lookups more efficient in large directories.
> At any rate, I''ve been testing my lustre-lite configuration for
the last
> week prior to a rollout of a small cluster. One of the tests I ran was to
> create 3.2 million files (actually, 6.4 million files, but only half of
> them remained on the drive at the end of the test. The others were created
> and deleted in the script), in a flat directory structure. Lustre worked
> fine, but of course I couldn''t ''rm'' the files.
I could have done an
> xargs, but chose to reformat instead.
For those who don''t know, this is a limitation of the shell -- most
shells
have a maximum number or size of command-line arguments, and "rm *"
would
exceed that -- not a limitation in Lustre.
> The next test called for the creation of fewer, larger files - I think I
> made 800,000 or so. After this test, I did ''rm'' them
(with xargs). Again,
> no problem. However, after deleting them I still ended up with that
''.''
> file being very large, and processing the directory being very slow:
>
> snippet:
> root@as2 10:56:53: [/space] $ time ls -alh /cluster/
> total 18M
> drwxr-xr-x 2 root root 18M Dec 14 22:56 .
> drwxr-xr-x 23 root root 536 Dec 14 22:31 ..
This is a side-effect of the fact that we use ext3 as our backend file
system today. ext3 never shrinks directories, only grows them.
It''s possible that that could be fixed, but I think it''s minor
enough that
most people would prefer we spend that time on other things.
> For comparison, I created and deleted 50,000 files on a non-Lustre
> directory on the same machine. After deleting the files, the size of the
> ''.'' file and the performance were fine:
This is probably not an ext3 file system. Am I right?
> Is there something I should be doing differently (other than using
> subdirectories?)
If the directories that don''t shrink are a problem for you, then yes, I
think using more subdirectories will be your only option.
Thanks--
-Phil