Hi Sophana--
On 12/7/2004 7:33, sophana wrote:>
> I''ve read in this mailing list that lustre''s performance
for small files
> is very bad (compared to nfs for example) (was true for version 1.0)
>
> Is it still true?
I think the situation is somewhat better than that, actually.
We have run many benchmarks comparing many things, between Lustre and some
expensive enterprise NFS hardware that shall remain nameless. Compared to
NFS, and somewhat to our surprise, Lustre performed very well in almost
every test we can think of, including tests that involve many tiny files.
But you''re probably right; we may not have started these comparisons
until
Lustre 1.2.
> my application is a compute cluster for vaious job sizes. Lot of these
> jobs are compilation jobs with rather small source files.
One such comparison that we did was to unpack and compile a Linux kernel
tarball. Both the unpack and the compilation were about 25% faster in our
TCP/IP-based Lustre installation, compared to the NFS hardware.
When we ran the same test over Elan 3, just to see, performance almost
doubled. This is not very surprising, given the exceptionally low latency
of Elan; tests which are quite metadata-heavy will benefit from the lower
latency.
I don''t know if Lustre will be faster than NFS for your particular
application or not -- I recommend that you evaluate Lustre 1.4, run a test,
and let us know how it turns out!
If you only have time to do this test once, then you may wish to wait a
couple of releases -- Lustre 1.4.2 or 1.4.3 will contain some metadata
locking optimizations that will reduce the amount of metadata lock
contention in many situations.
> Has anybody measured the read latency of lustre (when the system is not
> loaded)?
Not specifically, no. But I know that some of people are using Lustre
precisely because it can open and read small files relatively quickly (to do
some real-time video processing, with one frame per file).
Regards--
-Phil