Gluster users - Mar 2021 - any one uses all flash GlusterFS setup?

Thanks Strahil,
good point on choose-local, definitely we will try.
the connection is: 10Gbit, also FDR Infiniband( ipoib will be used).
we are still experimenting with 2 buildings and 8 nodes ovirt+ changing the
bricks number on glusterfs.

Strahil Nikolov <hunter86_bg at yahoo.com> schrieb am So., 28. M?rz 2021,
00:35:
> It's worth mentioning that if your network bandwidth is smaller than
the
> raid bandwidth, you can consider to enable the cluster.choose-local (which
> oVirt's optimizations disable) for faster reads.
>
> Some people would also consider going with JBOD (replica 3) mode. I guess
> you can test both prior moving to prod phase
>
>
> P.S.: Don't forget to align the LVM/FS layer to the hardware raid.
>
> Best Regards,
> Strahil Nikolov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20210329/b18cc244/attachment.html>

I guess there is no need to mention that lattency is the real killer of Gluster.
What is the DC-to-DC lattency and MTU (ethernet) ?Also, if you use SSDs ,
consider using noop/none I/O schedulers.
Also, you can obtain the tuned profiles used in Red Hat Gluster Storage via this
source rpm:
http://ftp.redhat.com/redhat/linux/enterprise/7Server/en/RHS/SRPMS/redhat-storage-server-3.5.0.0-7.el7rhgs.src.rpm
You can combine the settings from the tuned profile for Hypervisor and combine
it with the gluster random I/O tuned profile.
Also worth mentioning, RHGS uses 512M shard size, while default in upstream
gluster is just 64M. Some oVirt users have reported issues and suspect is
gluster's inability to crwate enough shards.
WARNING: ONCE SHARDING IS ENABLED, NEVER EVER DISABLE IT.
Best Regards,Strahil Nikolov
 
 
  On Mon, Mar 29, 2021 at 11:03, Arman Khalatyan<arm2arm at gmail.com>
wrote:   Thanks Strahil,good point on choose-local, definitely we will try.the
connection is: 10Gbit, also FDR Infiniband( ipoib will be used).we are still
experimenting with 2 buildings and 8 nodes ovirt+ changing the bricks number on
glusterfs.?

Strahil Nikolov <hunter86_bg at yahoo.com> schrieb am So., 28. M?rz 2021,
00:35:

It's worth mentioning that if your network bandwidth is smaller than the
raid bandwidth, you can consider to enable the cluster.choose-local (which
oVirt's optimizations disable) for faster reads.
Some people would also consider going with JBOD (replica 3) mode. I guess you
can test both prior moving to prod phase?

P.S.: Don't forget to align the LVM/FS layer to the hardware raid.
Best Regards,Strahil Nikolov
  
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20210329/f13a994b/attachment.html>