Hi,
about your DB usage of the Lustre FS, I would strongly advise you to follow
Andreas advice and test first.
Note also that you must use a recent Percona-released MySQL to avoid a nasty
POSIX-related bug in Innodb that crashes MySQL...
In my case, performance showed to be dramatically insufficent. small random I/O
and meta-data performances are a killer for MySQL.
I could probably optimze in a number of other ways that what I did, or throw a
whole lot of additional hardware at it, but after several month of tweaking
(both lustre and mysql), we''ve just got away from it and returned to
plain direct-attached storage (and sharding).
Nevertheless, I took the time to run post-mortem benchs with mysql+lustre (for
other untested optimzations, notably RAID configs of the underlying storage).
And also with standard tools (dd, sgp_dd, bonnie++) on other DFS.
To make a long story short :
1/ our usage of MySQL is not compatible with Lustre. You should check your is !
2/ Lustre is the best-performing DFS I could test, given then hardware we had
(tested PVFS2/OrangeFS, GlusterFS, FHGFS/FraunhoferFS)
(NB : FHGFS is second-best by far. OrangeFS showed stability issues, Gluster is
stable but is a joke...)
Hardware :
- a pair of HA MDS/MGS with a shared MDT on a iscsi array of RAID10 15k disks
(note : dedicated gigE switch)
- 3 OSSes with 2 RAID1 ''os'' disks + 6 RAIDed 7.2K SAS disks
(originally was a RAID10)
- connected with DDR IB (+ gigE for ''legacy'' clients)
Hope that helps.
Fran?ois Chassaing Directeur Technique - CTO
weborama.com - fch at weborama.com
----- Mail Original -----
De: "Andreas Dilger" <adilger at whamcloud.com>
?: "Jon Yeargers" <yeargers at ohsu.edu>
Cc: lustre-discuss at lists.lustre.org
Envoy?: Jeudi 8 Novembre 2012 02h19:30 GMT +01:00 Amsterdam / Berlin / Berne /
Rome / Stockholm / Vienne
Objet: Re: [Lustre-discuss] Lustre - usage / best practices
On 2012-11-04, at 8:54 AM, Jon Yeargers wrote:> I?ll soon be retiring a number of old file servers and have the opportunity
to move to a DFS based system. Im wondering about using Lustre for ?mixed
purposes? and what the recommended setup is.
>
> More specifically: lets say there are three groups that want to use the
space. One group is primarily storing Tb of data in large numbers of flat files.
So, perfect use case for Lustre.
> A second group is using a DB like MySQL that has fewer but larger files.
This winds up being small random IO to the file. Not a common workload for
Lustre, so I would really recommend testing it out on a small system first.
> A third may have some other process. Would I divide up my DFS space somehow
or is it ok for all three groups to comingle? Would I do something on the Lustre
clients to force them apart or just rely on Linux security to restrict access to
the client folders (on the lustre client)?
The Lustre servers have essentially the same security model as NFS today, so
they depend on the clients to enforce user identification. If malicious users
can configure their own client nodes and connect them to the filesystem, then
they could access any files in the filesystem. If the client nodes are secure,
then Lustre will properly enforce user access permissions/ACLs to the files.
> I can imagine creating a single Lustre client and then NIS mounting the
various volumes on separate machines for each group. This seems like adding an
extra step but perhaps it?s the normal pattern?
Do you mean "NFS" when you wrote "NIS"? I''m not
sure why this would be better, but I can definitely think of reasons why it
would be worse.
Cheers, Andreas
--
Andreas Dilger Whamcloud, Inc.
Principal Lustre Engineer http://www.whamcloud.com/
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss at lists.lustre.org
http://lists.lustre.org/mailman/listinfo/lustre-discuss