Gluster users - Apr 2015 - NFS vs native fuse

On 2015-04-28 14:44, Kingsley wrote:
> I've seen a lot of talk about mounting gluster volumes via NFS, but
> despite searching around, I can't find (apart from performance in some
> cases) why you'd want to do this.
Performance seems like the big one in situations where it's applicable. 
Compatibility where the native FUSE client can't be installed or 
maintained also seems like a reasonable reason to use NFS, since the NFS 
client may already exist.

Bandwidth is also a consideration, the FUSE client will upload multiple 
copies based on the replica setting for the volume, so if the client is 
connected at 100Mb/s or over wifi, and the servers are cross-connected 
on a 10Gb/s backplane, having the client upload multiple copies vs 
having the NFS server handle the replicas may have an impact on very 
large files.

Finally, NFS seems to have a lighter CPU footprint on the client, at the 
possible cost of higher server CPU load, although this is anecdotal 
(from my own experience), and probably a mixed bag.

I have switched back and forth (and can re-mount my entire gluster 
infrastructure with the flip of a single distributed config file) and 
for me, NFS ends up performing better, the native gluster client is just 
too slow.

-- 
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren

On Tue, 2015-04-28 at 17:09 -0700, Dave Warren wrote:
> Bandwidth is also a consideration, the FUSE client will upload multiple 
> copies based on the replica setting for the volume, so if the client is 
> connected at 100Mb/s or over wifi, and the servers are cross-connected 
> on a 10Gb/s backplane, having the client upload multiple copies vs 
> having the NFS server handle the replicas may have an impact on very 
> large files.
> 
> Finally, NFS seems to have a lighter CPU footprint on the client, at the 
> possible cost of higher server CPU load, although this is anecdotal 
> (from my own experience), and probably a mixed bag.
Oh I think I get it - the Gluster server daemons are also NFS server
daemons, so mounting via NFS you're still talking to a Gluster daemon on
the server ... and when writing to the server, the servers then handle
the replication themselves?

I thought people were just putting the path to a brick in /etc/exports
and then just using nfsd that came with the OS, which I presume would
then break things.

In that case, would this be a reasonable summary of mounting via NFS
instead of using the native fuse client?

NFS pros:
      * faster
      * lighter weight on client resources (CPU, bandwidth)
      * available to clients that have a standard NFS client but cannot
        install gluster-fuse

NFS cons:
      * more load on servers
      * writes complete on the client before the data is replicated
        (which is faster, but less secure)
      * if the server that the client happens to be connected to goes
        down, the client loses access to the volume (whereas the fuse
        client recovers and continues writing to the remaining notes)

Does that seem about right?

-- 
Cheers,
Kingsley.