vilobh meshram
2011-Jun-22 23:06 UTC
[Lustre-discuss] Queries regarding Lustre Throughput Numbers with mdtest benchmark
Hi, I have a query regarding Lustre Throughput Numbers with mdtest benchmark.I am running mdtest benhmark with following options :- /home/meshram/mpich2-new/mpich2-1.4/mpich2-install/bin/mpirun -np 256 -hostfile ./hostfile ./mdtest -z 3 -b 10 -I 5 -v -d /tmp/l66 where , mdtest - is the standard benchmark to test metadata operations. [ https://computing.llnl.gov/?set=code&page=sio_downloads ] /tmp/l66 is my Lustre mount. I am using 1Gige Network with TCP transport. hostfile has 8 host nodes I am varying the number of processes as we can see in following table . I was amazed by the throughput which I got. I think this is too huge. Can someone please let me know if these numbers are correct ? 4 Process : File creation : 0.501 sec, 44392.140 ops/sec 8 Process : File creation : 0.685 sec, 64890.598 ops/sec 16 Process : File creation : 1.426 sec, 62318.798 ops/sec 32 Process : File creation : 2.947 sec, 60312.766 ops/sec 64 Process : File creation : 5.630 sec, 63142.760 ops/sec 128 Process : File creation : 13.208 sec, 53835.707 ops/sec 256 Process : File creation : 24.601 sec, 57804.777 ops/sec Thanks, Vilobh Graduate Research Associate Department of Computer Science The Ohio State University Columbus Ohio -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20110622/9d5c9fda/attachment.html
Andreas Dilger
2011-Jul-07 02:25 UTC
[Lustre-discuss] Queries regarding Lustre Throughput Numbers with mdtest benchmark
On 2011-06-22, at 5:06 PM, vilobh meshram wrote:> I have a query regarding Lustre Throughput Numbers with mdtest benchmark.I am running mdtest benhmark with following options :- > > /home/meshram/mpich2-new/mpich2-1.4/mpich2-install/bin/mpirun -np 256 -hostfile ./hostfile ./mdtest -z 3 -b 10 -I 5 -v -d /tmp/l66 > > > where , > mdtest - is the standard benchmark to test metadata operations. [https://computing.llnl.gov/?set=code&page=sio_downloads ] > /tmp/l66 is my Lustre mount. > I am using 1Gige Network with TCP transport. > hostfile has 8 host nodes > I am varying the number of processes as we can see in following table . > I was amazed by the throughput which I got. I think this is too huge. Can someone please let me know if these numbers are correct ? > > > > 4 Process : File creation : 0.501 sec, 44392.140 ops/sec > 8 Process : File creation : 0.685 sec, 64890.598 ops/sec > 16 Process : File creation : 1.426 sec, 62318.798 ops/sec > 32 Process : File creation : 2.947 sec, 60312.766 ops/sec > 64 Process : File creation : 5.630 sec, 63142.760 ops/sec > 128 Process : File creation : 13.208 sec, 53835.707 ops/sec > 256 Process : File creation : 24.601 sec, 57804.777 ops/secSeems nobody has responded to your email, and I just found it buried in my inbox. I agree that the numbers are quite high, especially for GigE networking. More typical numbers are in the 5-20k creates/sec (depending on network and MDS hardware). That said, the above may not be completely impossible for small file counts (allowing all of the creates to be served from cache), or if the client+MDS+OSS are all on the same node (which avoids any network latency). You also didn''t describe the Lustre filesystem, nor what version you are testing. This workload has gotten a faster with Lustre 2.1, and it stands to get faster in Lustre 2.2 also. Cheers, Andreas -- Andreas Dilger Principal Engineer Whamcloud, Inc.
Nathan Rutman
2011-Jul-07 22:32 UTC
[Lustre-discuss] [Lustre-devel] Queries regarding Lustre Throughput Numbers with mdtest benchmark
I suspect that you are running this test against local hard drives instead of a shared Lustre mount point. Are you sure Lustre is mounted at /tmp/l66 on all clients in hostfile? On Jul 6, 2011, at 7:25 PM, Andreas Dilger wrote:> On 2011-06-22, at 5:06 PM, vilobh meshram wrote: >> I have a query regarding Lustre Throughput Numbers with mdtest benchmark.I am running mdtest benhmark with following options :- >> >> /home/meshram/mpich2-new/mpich2-1.4/mpich2-install/bin/mpirun -np 256 -hostfile ./hostfile ./mdtest -z 3 -b 10 -I 5 -v -d /tmp/l66 >> >> >> where , >> mdtest - is the standard benchmark to test metadata operations. [https://computing.llnl.gov/?set=code&page=sio_downloads ] >> /tmp/l66 is my Lustre mount. >> I am using 1Gige Network with TCP transport. >> hostfile has 8 host nodes >> I am varying the number of processes as we can see in following table .. >> I was amazed by the throughput which I got. I think this is too huge. Can someone please let me know if these numbers are correct ? >> >> >> >> 4 Process : File creation : 0.501 sec, 44392.140 ops/sec >> 8 Process : File creation : 0.685 sec, 64890.598 ops/sec >> 16 Process : File creation : 1.426 sec, 62318.798 ops/sec >> 32 Process : File creation : 2.947 sec, 60312.766 ops/sec >> 64 Process : File creation : 5.630 sec, 63142.760 ops/sec >> 128 Process : File creation : 13.208 sec, 53835.707 ops/sec >> 256 Process : File creation : 24.601 sec, 57804.777 ops/sec > > Seems nobody has responded to your email, and I just found it buried in my > inbox. > > I agree that the numbers are quite high, especially for GigE networking. > More typical numbers are in the 5-20k creates/sec (depending on network > and MDS hardware). > > That said, the above may not be completely impossible for small file > counts (allowing all of the creates to be served from cache), or if the > client+MDS+OSS are all on the same node (which avoids any network latency). > > You also didn''t describe the Lustre filesystem, nor what version you are > testing. This workload has gotten a faster with Lustre 2.1, and it stands > to get faster in Lustre 2.2 also. > > Cheers, Andreas > -- > Andreas Dilger > Principal Engineer > Whamcloud, Inc. > > > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel______________________________________________________________________ This email may contain privileged or confidential information, which should only be used for the purpose for which it was sent by Xyratex. No further rights or licenses are granted to use such information. If you are not the intended recipient of this message, please notify the sender by return and delete it. You may not use, copy, disclose or rely on the information contained in it. Internet email is susceptible to data corruption, interception and unauthorised amendment for which Xyratex does not accept liability. While we have taken reasonable precautions to ensure that this email is free of viruses, Xyratex does not accept liability for the presence of any computer viruses in this email, nor for any losses caused as a result of viruses. Xyratex Technology Limited (03134912), Registered in England & Wales, Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA. The Xyratex group of companies also includes, Xyratex Ltd, registered in Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in The People''s Republic of China and Xyratex Japan Limited registered in Japan. ______________________________________________________________________