Gluster users - Feb 2015 - Poor Gluster performance

----- Original Message -----
> From: "Lars Hanke" <debian at lhanke.de>
> To: gluster-users at gluster.org
> Sent: Wednesday, February 18, 2015 3:01:54 PM
> Subject: [Gluster-users] Poor Gluster performance
> 
> I set up a distributed, replicated volume consisting of just 2 bricks on
> two physical nodes. The nodes are peered using a dedicated GB ethernet
> and can be accessed from the clients using a separate GB ethernet NIC.
> 
> Doing a simple dd performance test I see about 11 MB/s for read and
> write. Running a local setup, i.e. both bricks on the same machine and
> local mount, I saw even 500 MB/s. So network sould be the limiting
> factor. But using NFS or CIFS on the same network I see 110 MB/s.
> 
> Is gluster 10 times slower than NFS?
Something is going on there.  On my gigabit setups I see 100-120 MB / sec writes
for pure distribute and about 45-55 MB / sec with replica 2.  What block size
are you using?  I could see that if you were writing something like 4k or under
but 64k and up you should be getting about what I said.  Can you tell me more
about your test?

-b
> 
> Gluster version is 3.6.2.
> 
> Thanks for your help,
>   - lars.
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>

Am 18.02.2015 um 22:05 schrieb Ben Turner:
> ----- Original Message -----
>> From: "Lars Hanke" <debian at lhanke.de>
>> To: gluster-users at gluster.org
>> Sent: Wednesday, February 18, 2015 3:01:54 PM
>> Subject: [Gluster-users] Poor Gluster performance
>>
>> I set up a distributed, replicated volume consisting of just 2 bricks
on
>> two physical nodes. The nodes are peered using a dedicated GB ethernet
>> and can be accessed from the clients using a separate GB ethernet NIC.
>>
>> Doing a simple dd performance test I see about 11 MB/s for read and
>> write. Running a local setup, i.e. both bricks on the same machine and
>> local mount, I saw even 500 MB/s. So network sould be the limiting
>> factor. But using NFS or CIFS on the same network I see 110 MB/s.
>>
>> Is gluster 10 times slower than NFS?
>
> Something is going on there.  On my gigabit setups I see 100-120 MB / sec
writes for pure distribute and about 45-55 MB / sec with replica 2.  What block
size are you using?  I could see that if you were writing something like 4k or
under but 64k and up you should be getting about what I said.  Can you tell me
more about your test?
Block size is 50M:

root at gladsheim:/# mount -t glusterfs node2:/test ~/mnt
root at gladsheim:/# dd if=/dev/zero of=~/mnt/testfile.null bs=50M count=10
10+0 records in
10+0 records out
524288000 bytes (524 MB) copied, 46.6079 s, 11.2 MB/s
root at gladsheim:/# dd if=~/mnt/testfile.null of=/dev/null bs=50M count=10
10+0 records in
10+0 records out
524288000 bytes (524 MB) copied, 45.7487 s, 11.5 MB/s

It doesn't depend on whether I use node1 or node2 for the mount.

BTW: does the cut of the bandwidth to half in replicated mode mean that 
the client writes to both nodes, i.e. doubles the network load on the 
client side network? I hoped that replication would be run on the server 
side network.

Regards,
  - lars.