Gluster users - Nov 2015 - GlusterFS Performance tuning

Hi Lindsay Mathieson and all,


Le 24/11/2015 21:09, Lindsay Mathieson a ?crit :
> More details on your setup[ would be useful:
> - Number of Nodes
> - CPU & RAM on Each Node
> - What else is running on the nodes
> - Number of bricks
> - output of "gluster  volume info" & "gluster volume
status"
>
> - ZFS config for each Node
>   * number of disks and rai arrangement
>   * log and cache SSD?
>   * zpool status
>
OK I have tested that kind of configuration, and the result depend of 
what you are waiting for :

  * zfsonlinux is now efficient, but you will not have access to the ACL ;
  * on a volume with seven disk we get the maximum of the PCI Express
    bandwidth ;
  * so you can mount a distributed gluster volume with your zfsonlinux
    nodes. The bandwidth will depend of the kind of glusterfs volume you
    want to build : distributed, stripped, replicated ;
      o replicated : bad, because of the synchronism write for the files
        replication ;
      o striped is the best because it allow you to get an average
        bandwidth on a file whatever the node you R/W the file ;
  * then the last, for me, is the Ethernet access between each nodes. If
    you have 1Gb, get back to your sand box, At the year you need 10Gbs
    and as the minimal Ethernet access is two port you need to bound them ;
  * have 10 Gbs Ethernet switch ;

That the expressions of the needs for the now and future necessities.

sincerely

Pierre L?onard


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20151124/377ab9cc/attachment.html>

you can do the following:

# gluster volume set $vol performance.o-thread-count 64
Today?s CPU are powerful enough to handle 64 threads per volume.

# gluster volume set $vol client.event-threads XX
XX depend on the number of connections from the FUSE client to the server, you
can get this number by running netstat and grep on the server IP and count the
number of connections.

# gluster volume set $vol server.event-threads XX

XX depend on the number of connections from the server to the client(s), you can
get this number by running netstat and grep on ?gluster" and count the
number of connections.

also, you can follow the instructions in the following page:
http://gluster.readthedocs.org/en/release-3.7.0/Administrator%20Guide/Linux%20Kernel%20Tuning/


-Bishoy
> On Nov 24, 2015, at 1:31 PM, Pierre MGP Pro <pierre-mgp-jouy.inra at
agsic.net> wrote:
> 
> Hi Lindsay Mathieson and all,
> 
> 
> Le 24/11/2015 21:09, Lindsay Mathieson a ?crit :
>> More details on your setup[ would be useful:
>> - Number of Nodes
>> - CPU & RAM on Each Node
>> - What else is running on the nodes
>> - Number of bricks
>> - output of "gluster  volume info" & "gluster volume
status"
>> 
>> - ZFS config for each Node
>>   * number of disks and rai arrangement
>>   * log and cache SSD?
>>   * zpool status
>> 
> OK I have tested that kind of configuration, and the result depend of what
you are waiting for :
> zfsonlinux is now efficient, but you will not have access to the ACL ;
> on a volume with seven disk we get the maximum of the PCI Express bandwidth
;
> so you can mount a distributed gluster volume with your zfsonlinux nodes.
The bandwidth will depend of the kind of glusterfs volume you want to build :
distributed, stripped, replicated ;
> replicated : bad, because of the synchronism write for the files
replication ;
> striped is the best because it allow you to get an average bandwidth on a
file whatever the node you R/W the file ;
> then the last, for me, is the Ethernet access between each nodes. If you
have 1Gb, get back to your sand box, At the year you need 10Gbs and as the
minimal Ethernet access is two port you need to bound them ;
> have 10 Gbs Ethernet switch ;
> That the expressions of the needs for the now and future necessities.
> sincerely
> Pierre L?onard
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20151124/b58cab26/attachment.html>