Gluster users - Mar 2013 - How to evaluate the glusterfs performance with small file workload?

Hi guys

    I have met some troubles when I want to evaluate the glusterfs performance
with small file workload.

    1: What kind of benchmark should I use to test the small file operation ?

        As we all know, we can use iozone tools to test the large file
operation, while for the sake of memory cache,

       if we testing small file operation with iozone, the result will not
correct. Then, what kind of benchmark should

       I use? 

       How about "dd oflag=direct"?

   2: How to simulate the the large scale clients concurrence accessing
operation?

       When we use iozone, there is a cluster can help us doing multiple clients
testing. While, if the number of

       clients is about hounds , it's difficult for us to deploy so many
clients at the same time? Could we

       deploy multiple processes on one client at the same time to simulate
multiple clients concurrence?

    3: For the small file operation, how to increase the workload of a single
client?

    4: From the glusterfs document, I get that in order to avoid the cache
coherency there is no write cache feature.

        Does it mean that there is no inference of memory cache for small file
write performance of glusterfs?

        So, when we testing glusterfs with:

        "dd if=/dev/zero of=test.img bs=10k count=1 oflag=direct" and

        "dd if=/dev/zero of=test.img bs=10k count=1"

        These two commands should get the same write performance.

        While when I do this, the results of these two commands are not same
each other. and the gap is big.

        How to explain?

    5: How to tuning for the small file operation on glusterfs?

    

        If you know, please let me know, many thanks

        

        Best Regards

        Lixin Niu

    

   
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130318/5a59df1d/attachment.html>

On Mon, Mar 18, 2013 at 11:27 AM, nlxswig <nlxswig at 126.com>
wrote:
> Hi guys
>     1: What kind of benchmark should I use to test the small file operation
> ?
I've been wondering a bit about the same thing.
I was thinking it would be nice to have something record and
synthesize IO patterns.
One could record a process which does a lot of handling of small
files, for example Dovecot,
and be able to replay those IO patterns on top of any filesystem.

A quick look around revealed ioreplay[1].
It seems to work by replaying strace output, which is cool idea.
I haven't tried it, but it looks to be a nice testing tool.

[1]: https://code.google.com/p/ioapps/wiki/ioreplay
>     4: From the glusterfs document, I get that in order to avoid the cache
> coherency there is no write cache feature.
>
>         Does it mean that there is no inference of memory cache for small
> file write performance of glusterfs?
>
>         So, when we testing glusterfs with:
>
>         "dd if=/dev/zero of=test.img bs=10k count=1 oflag=direct"
and
>
>         "dd if=/dev/zero of=test.img bs=10k count=1"
>
>         These two commands should get the same write performance.
>
>         While when I do this, the results of these two commands are not
same
> each other. and the gap is big.
>
>         How to explain?
My impression is that there are write caching features,
but Gluster tries hard to maintain coherency and correctness regarding writes.
For one type of cache, see the write-behind translator that is enabled
by default.

AFAIK, the difference between the to dd invocations is that the first
one disables
all caches, while the last one doesn't even wait for the sync before
finishing.
My understanding leads me to say that the first one can't use cache at all,
while the second one uses all the cache there is.

Try to run the last one with "conv=fsync".
This will sync the file at the end of writing, ensuring that when dd
returns the data should be on disk. This will probably even out the
run time for the two invocations.



--
Vennlig hilsen
Torbj?rn Thorsen
Utvikler / driftstekniker

Trollweb Solutions AS
- Professional Magento Partner
www.trollweb.no

Telefon dagtid: +47 51215300
Telefon kveld/helg: For kunder med Serviceavtale

Bes?ksadresse: Luramyrveien 40, 4313 Sandnes
Postadresse: Maurholen 57, 4316 Sandnes

Husk at alle v?re standard-vilk?r alltid er gjeldende

nlxswig <nlxswig at 126.com> writes:
> Hi guys
>
> I have met some troubles when I want to evaluate the glusterfs
> performance with small file workload.
>
> 1: What kind of benchmark should I use to test the small file
> operation ?
I use fio (http://freecode.com/projects/fio).

Here's an old tutorial:
https://www.linux.com/learn/tutorials/442451-inspecting-disk-io-performance-with-fio/

You'll need to read the docs to use it well, but the tutorial gives
some idea of the operation.

-- 
Shawn Nock (OpenPGP: 0x65118FA5)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130318/c4b71b6e/attachment.sig>