Gluster users - Jun 2020 - Latest NFS-Ganesha Gluster Integration docs

For many years, we have maintained a number of standalone, 
hyperconverged Gluster/Libvirt clusters? Replica 2 + Arbiter using Fuse 
mount and Sharding.

Performance has been mostly acceptable. The clusters have high 
availability and we have had very zero problems over the years as long 
as we do green field upgrades to new major versions.

So that is the beauty of Gluster in our opinion. It is easy to setup and 
use and it simply works without much thought.

As we ask more of our VMs, we see that disk i/o sometimes is a 
bottleneck and I am looking at improving things.

On this list I keep on seeing comments that VM performance is better on 
NFS and a general dissatisfaction with Fuse. So we are looking to see 
for ourselves if NFS would be an improvement.

We looked into gfapi but our hosts are mostly Ubuntu and gfapi is not 
built in. We would prefer to stay with stock components for the critical 
underbelly of the infrastructure.

So in looking into NFS,? I find instructions for NFS-Ganesha as a 
standalone, and I see mentions of Gluster Drivers but then I see 
mentions of StorHaug which sort of seems to be worked on. I also see 
pacemaker type failover instructions.

Is there a set of instructions for replacing our Fuse Mount setup with 
NFS/Ganesha and how HA is achieved? (with or without StorHaug).

Any advice in this area would be appreciated.

-wk

Last time I did storhaug+NFS-Ganesha I used
https://github.com/gluster/storhaug/wiki .
I guess you can setup NFS-Ganesha without HA and check the performance before 
proceeding further.

Have you tuned your I/O scheduler,  tuned profile , aligned your PV ,etc ? There
are  alot of stuff that can improve your Gluster.

Also, you can check the settings in /var/lib/glusterd/groups/virt  . The 
settings  are used by oVirt/RHV and are the optimal settings for a
Virtualization.

P.S.: Red Hat support Hyperconverged Infrastructure with 512MB shards, while the
default shard size is 64MB. You can test on another volume setting a bigger
shard size.


Best Regards,
Strahil Nikolov

?? 29 ??? 2020 ?. 5:00:22 GMT+03:00, "wkmail at bneit.com" <wkmail
at bneit.com> ??????:
>For many years, we have maintained a number of standalone, 
>hyperconverged Gluster/Libvirt clusters? Replica 2 + Arbiter using Fuse
>
>mount and Sharding.
>
>Performance has been mostly acceptable. The clusters have high 
>availability and we have had very zero problems over the years as long 
>as we do green field upgrades to new major versions.
>
>So that is the beauty of Gluster in our opinion. It is easy to setup
>and 
>use and it simply works without much thought.
>
>As we ask more of our VMs, we see that disk i/o sometimes is a 
>bottleneck and I am looking at improving things.
>
>On this list I keep on seeing comments that VM performance is better on
>
>NFS and a general dissatisfaction with Fuse. So we are looking to see 
>for ourselves if NFS would be an improvement.
>
>We looked into gfapi but our hosts are mostly Ubuntu and gfapi is not 
>built in. We would prefer to stay with stock components for the
>critical 
>underbelly of the infrastructure.
>
>So in looking into NFS,? I find instructions for NFS-Ganesha as a 
>standalone, and I see mentions of Gluster Drivers but then I see 
>mentions of StorHaug which sort of seems to be worked on. I also see 
>pacemaker type failover instructions.
>
>Is there a set of instructions for replacing our Fuse Mount setup with 
>NFS/Ganesha and how HA is achieved? (with or without StorHaug).
>
>Any advice in this area would be appreciated.
>
>-wk
>
>
>
>________
>
>
>
>Community Meeting Calendar:
>
>Schedule -
>Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>Bridge: https://bluejeans.com/441850968
>
>Gluster-users mailing list
>Gluster-users at gluster.org
>https://lists.gluster.org/mailman/listinfo/gluster-users

Hi,

On 6/29/20 7:30 AM, wkmail at bneit.com wrote:
> For many years, we have maintained a number of standalone, 
> hyperconverged Gluster/Libvirt clusters? Replica 2 + Arbiter using Fuse 
> mount and Sharding.
> 
> Performance has been mostly acceptable. The clusters have high 
> availability and we have had very zero problems over the years as long 
> as we do green field upgrades to new major versions.
> 
> So that is the beauty of Gluster in our opinion. It is easy to setup and 
> use and it simply works without much thought.
> 
> As we ask more of our VMs, we see that disk i/o sometimes is a 
> bottleneck and I am looking at improving things.
> 
> On this list I keep on seeing comments that VM performance is better on 
> NFS and a general dissatisfaction with Fuse. So we are looking to see 
> for ourselves if NFS would be an improvement.
> 
> We looked into gfapi but our hosts are mostly Ubuntu and gfapi is not 
> built in. We would prefer to stay with stock components for the critical 
> underbelly of the infrastructure.
> 
> So in looking into NFS,? I find instructions for NFS-Ganesha as a 
> standalone, and I see mentions of Gluster Drivers but then I see 
> mentions of StorHaug which sort of seems to be worked on. I also see 
> pacemaker type failover instructions.
There are gluster docs available to manually setup NFS-Ganesha on top of 
gluster -

Standalone-

https://docs.gluster.org/en/latest/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/#step-by-step-procedures-to-exporting-glusterfs-volume-via-nfs-ganesha


HA (using Pacemaker) -

https://docs.gluster.org/en/latest/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/#using-highly-available-active-active-nfs-ganesha-and-glusterfs-cli

Its been a while since this document was added. So please let us know if 
you run into any errors or any details are missing.

Also there are ansible playbooks written automating all the above manual 
steps to configure NFS-Ganesha HA on Gluster. Maybe you could try that.

https://github.com/gluster/gluster-ansible-features

Thanks,
Soumya

> 
> Is there a set of instructions for replacing our Fuse Mount setup with 
> NFS/Ganesha and how HA is achieved? (with or without StorHaug).
> 
> Any advice in this area would be appreciated.
> 
> -wk
> 
> 
> 
> ________
> 
> 
> 
> Community Meeting Calendar:
> 
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
> 
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users

Dear Users,

> On this list I keep on seeing comments that VM performance is better
> on NFS and a general dissatisfaction with Fuse. So we are looking to
> see for ourselves if NFS would be an improvement

Does anyone can provide information (performance test) how big the
improvement is using ganesha nfs instead of glusterfs in case of small
file performance?

Actually, due to an issue with a geo-replication issue we stick to fuse,
but switching on the long run from fuse to ganesha nfs obviously make
sense only if

the benefit is big enough.


Regards,

Felix