Hi Joe,
We've been seeing the same thing here with several GlusterFS setups. It
is really bad with GigE under load. From looking at our setups, it appears to
be a network latency/contention issue. It is definitely not a disk contention
issue as there is not a lot of load on the disks at the time of the metadata
performance issues.
Like you, we've tried a number of different combinations to help improve
the performance. statprefetch will help with subsequent accesses to the same
metadata, but it doesn't help with the initial access.
We have not tried Infiniband yet to see if it improves the performance, but I
am concerned that in a high IOps environment (like we run here), that even IB
will not be able to sustain performance.
- Mike
Michael Moore
Staff Engineer - Network
SOLiD Advanced Research and Collaborations
500 Cummings Center
Suite 2400
Beverly, MA 01915
T: 978-232-7886
Michael.Moore at lifetech.com
Mike.Moore at appliedbiosystems.com
-----Original Message-----
From: gluster-users-bounces at gluster.org [mailto:gluster-users-bounces at
gluster.org] On Behalf Of Joe Landman
Sent: Tuesday, April 27, 2010 7:19 PM
To: gluster-users at gluster.org
Subject: [Gluster-users] heavy metadata issues?
Hi folks
Seeing some metadata slowdowns in some cases on GlusterFS 3.0.3.
These metadata slowdowns manifest in terms of long ls, long stat, long
rm times for directories with a few thousand files.
We tried statprefetch at the end of the stack, as well as right after
the protocol/client, tried increasing read-ahead, etc.
Is there anything in particular we should be looking to do, in order
to get reasonable metadata performance under load?
Thanks!
Joe
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web : http://scalableinformatics.com
http://scalableinformatics.com/jackrabbit
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users