On Mar 01, 2006 13:33 +0100, Arjan Wijnveen wrote:> When i initially setup the system i ran some tests on all disks in a striped > configuration and the performance was blistering. However when i try to read > many many small files the performance drops and ''ps aux'' shows the client > application hanging on disk.Does your application run many threads on the same client? What are the open flags for the read-only clients (use strace if you don''t have the source code)? How much RAM does the MDS have, and how much is used?> From what i gathered reading through the mailling list the MDS or client > locks the directory when there is a file request thus basically killing > concurrent access.The MDS only ever grants read locks to clients, and it is possible for many clients to have read locks on the same item, so that should not be the issue.> I also found some references to a ''pdirops'' patch for ext3 which supposedly > tackled this issue but i''ve been unable to find out if it is integrated in > any of the releases or if it should be manually patched in.The pdirops patches are not in any released version of Lustre. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
I currently have lustre 1.4.6 running on 6 RHEL 4 machines using the default 2.6.9-22 lustre kernel. However i''m experiencing some serious performance issues in regards to the metadata. The situation i''m using luster in is as follows : There are 4 OSS''es with each 2 OBD''s. There is 1 MDS and a seperate client machine. The data is stored non-striped and consists of about 500k files with each between 10kb and 1mb. The client reads the small files at random from the cluster at about 600 req/s. The client needs read-only access as the writing is done in batches from another client. When i initially setup the system i ran some tests on all disks in a striped configuration and the performance was blistering. However when i try to read many many small files the performance drops and ''ps aux'' shows the client application hanging on disk.>From what i gathered reading through the mailling list the MDS or client locks the directory whenthere is a file request thus basically killing concurrent access. I also found some references to a ''pdirops'' patch for ext3 which supposedly tackled this issue but i''ve been unable to find out if it is integrated in any of the releases or if it should be manually patched in. Is there any way to speed up the concurrent access to those files which all reside in the same directory ? Since the access needs to be read only i wondered if it would be possible to disable the locking all together ? The manual did mention ''disable locking for direct i/o access'' but this section has not yet been written. Hopefully someone on this list can help me out as i''m all out of ideas. Regards, Arjan Wijnveen BounceTek