I test touch and ls command of Lustre on Linux 2.6.9-67.0.7.EL_lustre.
1.6.5smp (x86, 64bit, mem:4G,two CPU).
Lustre Filesystem:
one MDS
one OSS(one OST)
I found it some problems:
With the increase in the number of files, Lustre system performance
decrease too much.
1. fileid.sh
#!/bin/bash
for ((i=0; $i<10000; i++))
do
fileid="/mnt/webfile/test/$i";
touch $fileid
done
time ./fileid.sh
eal 0m20.493s
user 0m1.930s
sys 0m9.340s
time ls
real 0m2.748s
user 0m0.050s
sys 0m0.610s
touch:10000/20.493=487.97
ls:10000/2.748=3639.01
2.
#!/bin/bash
for ((i=0; $i<1000000; i++))
do
fileid="/mnt/webfile/test/testm/$i";
touch $fileid
done
time ./fileid.sh
real 60m56.666s
user 3m11.690s
sys 20m37.190s
real 15m48.615s
user 0m14.530s
sys 3m52.960s
touch:1000000/(60*60+56.666)=273.47
ls:1000000/(15*60+48.615)=1054.16
ext3 Filesystem test:
#!/bin/bash
for ((i=0; $i<1000000; i++))
do
fileid="/home/test/testm/$i";
touch $fileid
done
time ./fileid.sh
real 16m44.438s
user 3m0.860s
sys 12m35.920s
time ls
real 1m54.691s
user 0m13.770s
sys 1m0.220s
touch:1000000/(16*60+44.438)=995.58
ls:1000000/(1*60+54.691)=8719.08
Lustre 1.6.5 slower than normal file system?
Add:
The network of Lustre Filesystem is Giga network.
On Thu, 2008-07-17, at 03:50 PM, Johnlya <john... at gmail.com>
wrote:> I test touch and ls command of Lustre on Linux 2.6.9-67.0.7.EL_lustre.
> 1.6.5smp (x86, 64bit, mem:4G,two CPU).
> Lustre Filesystem:
> one MDS
> one OSS(one OST)
> I found it some problems:
> With the increase in the number of files, Lustre system performance
> decrease too much.
> 1. fileid.sh
> #!/bin/bash
> for ((i=0; $i<10000; i++))
> do
> ? ? ? ? ? ? ? ? fileid="/mnt/webfile/test/$i";
> ? ? ? ? ? ? ? ? touch $fileid
> done
> time ./fileid.sh
> eal ? ?0m20.493s
> user ? ?0m1.930s
> sys ? ? 0m9.340s
>
> time ls
> real ? ?0m2.748s
> user ? ?0m0.050s
> sys ? ? 0m0.610s
> touch:10000/20.493=487.97
> ls:10000/2.748=3639.01
> 2.
> #!/bin/bash
> for ((i=0; $i<1000000; i++))
> do
> ? ? ? ? ? ? ? ? fileid="/mnt/webfile/test/testm/$i";
> ? ? ? ? ? ? ? ? touch $fileid
> done
> time ./fileid.sh
> real ? ?60m56.666s
> user ? ?3m11.690s
> sys ? ? 20m37.190s
>
> real ? ?15m48.615s
> user ? ?0m14.530s
> sys ? ? 3m52.960s
>
> touch:1000000/(60*60+56.666)=273.47
> ls:1000000/(15*60+48.615)=1054.16
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-disc... at
lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss
Lustre doesn''t perform well with these kinds of tiny file i/o and metadata intensive operations. If you search back through the lists this issue has been covered a bunch of times. Small i/o is not efficient for a large filesystem and lustre wasn''t designed with this kind of workload in mind. It was designed for a proper HPC workload. On Jul 17, 2008, at 4:45 AM, Johnlya wrote:> ext3 Filesystem test: > #!/bin/bash > for ((i=0; $i<1000000; i++)) > do > fileid="/home/test/testm/$i"; > touch $fileid > done > > time ./fileid.sh > real 16m44.438s > user 3m0.860s > sys 12m35.920s > > time ls > real 1m54.691s > user 0m13.770s > sys 1m0.220s > > touch:1000000/(16*60+44.438)=995.58 > ls:1000000/(1*60+54.691)=8719.08 > > Lustre 1.6.5 slower than normal file system? > > Add: > The network of Lustre Filesystem is Giga network. > > On Thu, 2008-07-17, at 03:50 PM, Johnlya <john... at gmail.com> wrote: >> I test touch and ls command of Lustre on Linux >> 2.6.9-67.0.7.EL_lustre. >> 1.6.5smp (x86, 64bit, mem:4G,two CPU). >> Lustre Filesystem: >> one MDS >> one OSS(one OST) >> I found it some problems: >> With the increase in the number of files, Lustre system performance >> decrease too much. >> 1. fileid.sh >> #!/bin/bash >> for ((i=0; $i<10000; i++)) >> do >> fileid="/mnt/webfile/test/$i"; >> touch $fileid >> done >> time ./fileid.sh >> eal 0m20.493s >> user 0m1.930s >> sys 0m9.340s >> >> time ls >> real 0m2.748s >> user 0m0.050s >> sys 0m0.610s >> touch:10000/20.493=487.97 >> ls:10000/2.748=3639.01 >> 2. >> #!/bin/bash >> for ((i=0; $i<1000000; i++)) >> do >> fileid="/mnt/webfile/test/testm/$i"; >> touch $fileid >> done >> time ./fileid.sh >> real 60m56.666s >> user 3m11.690s >> sys 20m37.190s >> >> real 15m48.615s >> user 0m14.530s >> sys 3m52.960s >> >> touch:1000000/(60*60+56.666)=273.47 >> ls:1000000/(15*60+48.615)=1054.16 >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-disc... at lists.lustre.orghttp://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss