Lustre discuss - Nov 2006 - Lustre as general purpose cluster filesystem

I wanted to know if Lustre can be used as a general purpose cluster
filesystem? Has anybody done any benchmarking of Lustre with workload like
fileserver(postmark), varmail, oltp etc? My intention is to use lustre for
scalable web services (mostly read only data), to serve files (postmark kind
of load) and try to run Oracle RAC instances on 4-8 Lustre clients. As far
as I know Lustre has been used mostly for HPC environment for scientific
calculations, what about standard industry applications like scalable
apache, Oracle RAC etc?

Regards,
-Atul
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061128/7fa0afbb/attachment.html

On Tue, 28 Nov 2006, Atul Vidwansa wrote:
> I wanted to know if Lustre can be used as a general purpose cluster 
> filesystem? Has anybody done any benchmarking of Lustre with workload 
> like fileserver(postmark), varmail, oltp etc?
I tried postmark, with rather bad performance:
   https://mail.clusterfs.com/pipermail/lustre-discuss/2006-November/002371.html

It seems to me that Lustre is not yet ready to serve this kind of 
workload, but read the whole thread to make your own opinion. :)
> My intention is to use lustre for scalable web services (mostly read 
> only data), to serve files (postmark kind of load) and try to run Oracle 
> RAC instances on 4-8 Lustre clients. As far as I know Lustre has been 
> used mostly for HPC environment for scientific calculations, what about 
> standard industry applications like scalable apache, Oracle RAC etc?
Depends on workload, but I would expect better results with other cluster 
filesystems such as OCFS2 or GFS. As of now, good metadata performance 
with Lustre requires many clients.


Cheers,

-- 
Jean-Marc Saffroy - jean-marc.saffroy@ext.bull.net

We are evaluating Lustre with the same intent - to use it as a general
purpose file system. Lustre seems to perform well (compared to NFS)
with,
 
- large number of clients
- relatively bigger file sizes (roughly above 512KB)
- dedicated storage servers hosting OSTs
 
But its performance doesn''t measure up with,
 
- small files sizes (the ones you typically find in /usr/bin, /etc, ...)
- handful of clients
- general purpose PCs hosting OSTs along with other applications
 
Atleast this is our understanding trying out Lustre for the past couple
of weeks. Other than that it is pretty straight forward to
configure/install and its code is quite well written. I wish it had
better performance for small files. Here are few more email threads on
this subject,

https://mail.clusterfs.com/pipermail/lustre-discuss/2006-June/001619.htm
l
https://mail.clusterfs.com/pipermail/lustre-discuss/2005-December/001036
.html


thanks,
Sridhar
 
________________________________

	From: lustre-discuss-bounces@clusterfs.com
[mailto:lustre-discuss-bounces@clusterfs.com] On Behalf Of Atul Vidwansa
	Sent: Tuesday, November 28, 2006 9:05 AM
	To: lustre-discuss@clusterfs.com
	Subject: [Lustre-discuss] Lustre as general purpose cluster
filesystem
	
	
	I wanted to know if Lustre can be used as a general purpose
cluster filesystem? Has anybody done any benchmarking of Lustre with
workload like fileserver(postmark), varmail, oltp etc? My intention is
to use lustre for scalable web services (mostly read only data), to
serve files (postmark kind of load) and try to run Oracle RAC instances
on 4-8 Lustre clients. As far as I know Lustre has been used mostly for
HPC environment for scientific calculations, what about standard
industry applications like scalable apache, Oracle RAC etc? 
	
	Regards,
	-Atul