Marcos Montero wrote:>
> Hello,
>
> I''m trying to deploy a Lustre installation as a storage solution
for
> an e-mail cluster: Several nodes with a cyrus e-mail server sharing
> their data over the lustre file-system. The idea is to guarantee high
> availability and scalability in terms of power, (more clients), and
> storage, (with lustre offering large file systems).
>
> The Lustre installation consists of two nodes: one of them serving the
> MDT and the other one with two OSTs for the same file system. Each
> node is capable of acting as MDS and OSS at the same time in case of
> failure of the other node.
> The file system now has 3Gb but i''d like to increase the size to
7Gb
> in the future.
>
> The OSS and MDS nodes are two Itanium II servers with 2 CPU and 4Gb of
> RAM each one. The network connexion is a standard 1Gbit TCP/IP. The
> disk is provided by a Bull FDA2400 fiberchannel disk array.
>
> I have everything working but i have a very big problem with the I/O
> performance. The read and write ratios with "real e-mail data"
are
> extremely poor. I have I/O ratios around 200Kb/seg on readings and
> perhaps around 300Kb/seg on writings.
> I assume this is due to the fact that the standard configuration
> Lustre is oriented to work with very big files, but i don''t know
what
> can i expect in terms of performance with small files and i know for
> sure that most of the files in my e-mail server are really small files
> (i.e. size between 2Kb and 30Kb). Every user has some big files in his
> e-mail account. But these represent a very minimal part of the whole
> account in terms of messages.
>
>
> I have several questions: :-)
>
> - Anybody of you has ever tried to use Lustre for an e-mail cluster
> like this (or similar)? What are your experiences? Do you think is it
> worth to use Lustre for this kind of service? or only recommend it
> for big files in hpc scenarios?
>
> - I assume the I/O ratios are possible to improve. But, does anybody
> know the reasonable speeds than can i achieve with small files?
> 1Mb/seg? 10Mb/seg? 50Mb/seg?
>
> - Is there any specific tunning for the file system to work properly
> with small files?
>
>
I have not done a large scale mail server, but I do have some experience
with using Lustre for large file/small random i/o, which is similar to
your i/o pattern. Lustre is not suited for small i/o. If you can live
with the slower i/o speed, you can certainly meet your scalability
goals, however. There are tunable parameters for both client and server
(mostly client side for this application though), I don''t have a
running
Lustre server available right now so someone else will need to give you
the specifics. By default the client will keep a large local data
cache, which will affect write performance.
ME