On Jan 14, 2006 11:11 -0800, Ressa wrote:> i''m reading the lustre howto, in there is a topic > about performance measurement. And there is a commang > like below this: > #time dd if=/dev/zero of=/mnt/lustre/test bs=8k count=65536 > > 65536+0 records in > 65536+0 records out > > real 1m22.017s > user 0m0.292s > sys 0m11.904s > > what i want to ask is about th output of that command, > does anyone know what the meaning of output command??It means you wrote 65536 * 8kB records in 82 seconds, so 6.24MB/s which isn''t so great, though it depends on what config you have. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
----- Original Message ----- From: "Ressa" <ressa4299@yahoo.com> To: <lustre-discuss@clusterfs.com> Sent: Saturday, January 14, 2006 1:11 PM Subject: [Lustre-discuss] Performance Measurement> i''m reading the lustre howto, in there is a topic > about performance measurement. And there is a commang > like below this: > #time dd if=/dev/zero of=/mnt/lustre/test bs=8k > count=65536 > > 65536+0 records in > 65536+0 records out > > real 1m22.017s > user 0m0.292s > sys 0m11.904s > > what i want to ask is about th output of that command, > does anyone know what the meaning of output command?? > > Regards, > > > Ressa > Registered Linux User Number 336566 > Linux Newbie >Ressa, The command "time dd if=/dev/zero of=/mnt/lustre/test bs=8k count=65536 time ... command to measure the real, user, and system time of the command that follows. if .... input file of ... output file bs ...Block size for transfers (8k = 8192 byte sized transfers) count.. Number of transfers to do. Why this is not such a good idea... The command "dd" is going to read from /dev/zero and write to /mnt/lustre/test. Thus you are not only measuring the performance of writing to /mnt/lustre/test but also measuring several other things at the same time. Such as: a. Reading from /dev/zero b. Page faults caused by executing "dd" and the page faults of the internal buffers inside of dd. c. The CPU time consumed by the command dd for all of its internal operations. d. The loading of dd, and process tear down of the command dd. e. The fork, and exit time of a process. f. The open/create is also in the measurement. g. The test only produces results for an initial write, which includes metadata overhead. h. The open time for opening /dev/zero. i. The time it took for dd to write its output to the screen is also in the measurement. This is why you might want to consider a benchmark, instead of using a system utility that was not designed to be a benchmark. Naturally, I favor Iozone, but there are many other I/O benchmarks as well that would not have all of these side effects inside of your measurement. Enjoy, Don Capps
Ok, I can accept that on small IO jobs ''dd'' may not be the
best
benchmark, but If you use a large enough dataset, it should still give
you reliable numbers. You can pretty much ignore the
''overhead'' of
starting and stopping the command. From your point below:
a. Reading from /dev/zero
Dev zero is really fast. REALLY fast. On my little laptop here I can
copy 35.1 GB/s when we get to the point that reading from /dev/zero is
slower than network speed, I may buy this argument.
b. Page faults caused by executing "dd" and the page faults
of the internal buffers inside of dd.
If your memory load on the benchmark client is so bad that we start page
faulting the DD, is any benchmark going to work well?
c. The CPU time consumed by the command dd for all of its
internal operations.
And Benchmarks like IOZone don''t do anything internally?
They all probably do.
d. The loading of dd, and process tear down of the command dd.
Same with any benchmark
e. The fork, and exit time of a process.
Overhead. Inconsequential if you dataset is sufficiently
large
f. The open/create is also in the measurement.
Overhead.
g. The test only produces results for an initial write, which
includes
metadata overhead.
Again Overhead.
h. The open time for opening /dev/zero.
Overhead..
i. The time it took for dd to write its output to the screen is
also
in the measurement.
Sure this takes a little time, but its very small, and
is not necessarily going to a screen...
Anyhow after all that. I get better benchmarks using synchronized
parallel DD than I do with Iozone, or IOR. And its easy to run, most
systems have it etc... Now I need large data sets and time to run, but
I get better results.
I also think that dd represents better what a ''user'' will use,
I''ve
copied files with DD, but never with Iozone.
Evan Felix
PNNL
On Sat, 2006-01-14 at 11:11 -0800, Ressa wrote:> i''m reading the lustre howto, in there is a topic > about performance measurement. And there is a commang > like below this: > #time dd if=/dev/zero of=/mnt/lustre/test bs=8k > count=65536 > > 65536+0 records in > 65536+0 records out > > real 1m22.017s > user 0m0.292s > sys 0m11.904s > > what i want to ask is about th output of that command, > does anyone know what the meaning of output command??The command wrote 512MB to the test file. The time, as measured on a clock, was 82.017s, so the throughput was about 5.6MB/s. 0.292s were spent using the CPU in the dd command itself, and 11.904s was spent running in the kernel. The remaining 69.821s were spent waiting for the network, or waiting for other processes, etc. -jwb
----- Original Message ----- From: "Jeffrey W. Baker" <jwbaker@acm.org> To: "Ressa" <ressa4299@yahoo.com> Cc: <lustre-discuss@clusterfs.com> Sent: Saturday, January 14, 2006 2:12 PM Subject: Re: [Lustre-discuss] Performance Measurement> On Sat, 2006-01-14 at 11:11 -0800, Ressa wrote: >> i''m reading the lustre howto, in there is a topic >> about performance measurement. And there is a commang >> like below this: >> #time dd if=/dev/zero of=/mnt/lustre/test bs=8k >> count=65536 >> >> 65536+0 records in >> 65536+0 records out >> >> real 1m22.017s >> user 0m0.292s >> sys 0m11.904s >> >> what i want to ask is about th output of that command, >> does anyone know what the meaning of output command?? > > The command wrote 512MB to the test file. The time, as measured on a > clock, was 82.017s, so the throughput was about 5.6MB/s. 0.292s were > spent using the CPU in the dd command itself, and 11.904s was spent > running in the kernel. The remaining 69.821s were spent waiting for the > network, or waiting for other processes, etc. > > -jwb > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@lists.clusterfs.com > https://lists.clusterfs.com/mailman/listinfo/lustre-discuss >The command "dd" is not actually a benchmark tool and can produce somewhat contaminated results, due to the internal details. You might want to try an I/O benchmark like Iozone ( See: http://www.iozone.org ) Iozone is a very detailed tool for making I/O measurements. If you just want a wet-thumb, try Bonnie. Enjoy, Don Capps
is lustre have their own benchmark or measurement ?? Regards, Ressa Registered Linux User Number 336566 Linux Newbie The information is provided as is without warranty of any kind. In no event shall the writer be liable for any incidental, indirect or consequential damages of any kind, including, but not limited to : loss of business profits, police knocking on your door, computer crashes, sharks attack, temporary short-term memory loss (some cases reported recently), death of your pet or alien invasion... __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
Ressa wrote:> is lustre have their own benchmark or measurement ??We have a kit of IO tests available on our web, but using it is not trivial. For real-world numbers, it''s probably best to use iozone, or other standard tests that you are familiar with. cliffw> > > Regards, > > > Ressa > Registered Linux User Number 336566 > Linux Newbie > > The information is provided as is without warranty of any kind. In no event shall the writer be liable for any incidental, indirect or consequential damages of any kind, including, but not limited to : loss of business profits, police knocking on your door, computer crashes, sharks attack, temporary short-term memory loss (some cases reported recently), death of your pet or alien invasion... > > __________________________________________________ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@lists.clusterfs.com > https://lists.clusterfs.com/mailman/listinfo/lustre-discuss
Felix,
Corrections and comments below:
----- Original Message ----- >From: "Felix, Evan J" <Evan.Felix@pnl.gov>
>To: "Don Capps" <Don.Capps@hp.com>; "Ressa"
<ressa4299@yahoo.com>;
><lustre-discuss@clusterfs.com>
>Sent: Monday, January 16, 2006 10:40 AM
>Subject: RE: [Lustre-discuss] Performance Measurement
>
>Ok, I can accept that on small IO jobs ''dd'' may not be the
best
>benchmark, but If you use a large enough dataset, it should still give
>you reliable numbers. You can pretty much ignore the
''overhead'' of
>starting and stopping the command. From your point below:
>
> a. Reading from /dev/zero
>Dev zero is really fast. REALLY fast. On my little laptop here I can
>copy 35.1 GB/s when we get to the point that reading from /dev/zero is
>slower than network speed, I may buy this argument.
Perhaps Ok, if using large files, not so OK if lots of open/closes.
> b. Page faults caused by executing "dd" and the page
faults
> of the internal buffers inside of dd.
>If your memory load on the benchmark client is so bad that we start page
>faulting the DD, is any benchmark going to work well?
The "DD" application must page fault in from disk. This again
is not part of the I/O you were attempting to measure.
Iozone''s page-in''s are handled outside of the measurement.
> c. The CPU time consumed by the command dd for all of its
> internal operations.
>And Benchmarks like IOZone don''t do anything internally?
>They all probably do.
Iozone is designed to do I/O and use as small amount of
CPU as possible in doing so. DD is not.
> d. The loading of dd, and process tear down of the command dd.
>Same with any benchmark
No... Iozone measures the I/O while the benchmark is running,
it does not include any process startup, or teardown, in the
measurement. /bin/time dd will include startup and teardown times
in the measurment.
Note: Some systems have very long startup/teardown times. I''ve
seen some that are in seconds.
> e. The fork, and exit time of a process.
>Overhead. Inconsequential if you dataset is sufficiently
>large
Agreed. If the dataset is sufficiently large as to amortize the
overhead away, and the system doesn''t have abnormally
high fork/exit times. (as do some systems)
> f. The open/create is also in the measurement.
>Overhead.
Yes, and the open() system call is one of the most
expensive system calls that exist.
> g. The test only produces results for an initial write, which
> includes metadata overhead.
>Overhead.
Initial writes may not be what your application does. So it
makes sense to measure both initial writes, (with meta-data overhead)
and re-writes (without the meta-data overhead)
>Again Overhead.
> h. The open time for opening /dev/zero.
Again, if one is trying to measure file I/O, one would
not desire to have un-intended events inside the measurement.
>Overhead..
> i. The time it took for dd to write its output to the screen is
Again, if one is trying to measure file I/O, one would
not desire to have tty I/O in the measurment.
>
>also in the measurement.
>Sure this takes a little time, but its very small, and
>is not necessarily going to a screen...
True, it could be going to 300 baud line printer, or
a socket on a dirty network :-)
>
>
>Anyhow after all that. I get better benchmarks using synchronized
>parallel DD than I do with Iozone, or IOR. And its easy to run, most
>systems have it etc... Now I need large data sets and time to run, but
>I get better results.
It would depend on the implementation of "sychronized" as to
if this measurment was achieving what one thinks it is achieving.
Iozone uses a barrier sync mechanism so that it guarantees that
the measurment of the throughput in the parallel region, was only
taken during the time that the test was actually running in parallel.
(Removing straggler effects, from the measurement)
The definition of the word "better" is what I question :-)
>
>I also think that dd represents better what a ''user'' will
use, I''ve
>copied files with DD, but never with Iozone.
I would think that one would use "cp" instead of dd. :-) But I
would agree with the statement that the end user''s real
application is the best tool for measuring performance, be that
cp, dd, Oracle, Sybase, or perl. :-)
Enjoy,
Don Capps
i''m reading the lustre howto, in there is a topic about performance measurement. And there is a commang like below this: #time dd if=/dev/zero of=/mnt/lustre/test bs=8k count=65536 65536+0 records in 65536+0 records out real 1m22.017s user 0m0.292s sys 0m11.904s what i want to ask is about th output of that command, does anyone know what the meaning of output command?? Regards, Ressa Registered Linux User Number 336566 Linux Newbie The information is provided as is without warranty of any kind. In no event shall the writer be liable for any incidental, indirect or consequential damages of any kind, including, but not limited to : loss of business profits, police knocking on your door, computer crashes, sharks attack, temporary short-term memory loss (some cases reported recently), death of your pet or alien invasion... __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com