Gluster users - Apr 2015 - Poor performance of NFS client for large writes compared to native client

----- Original Message -----
> From: "Vijay Bellur" <vbellur at redhat.com>
> To: "Behrooz Shafiee" <shafiee01 at gmail.com>,
"Gluster-users at gluster.org List" <gluster-users at
gluster.org>
> Sent: Thursday, April 30, 2015 6:44:11 AM
> Subject: Re: [Gluster-users] Poor performance of NFS client for large
writes compared to native client
> 
> On 04/30/2015 06:49 AM, Behrooz Shafiee wrote:
> > Hi,
> >
> >   I was comparing GlusterFS native and NFS clients and I noticed, NFS
> > client is significantly slower for large writes. I wrote about 200,
1GB
> > files using a 1MB block sizes and NFS throughput was almost half of
> > native client. Can anyone explain why is that?
> >
> 
> Depending on where the file gets scheduled, NFS might need an additional
> network hop. That can contribute to additional latency and less
> throughput than the native client.
To tag on here GlusterFS mounts use the hash algorithm to know which server to
write directly to.  NFS is not aware of this so all files get routed through the
server that is mounted just like Vijay said.  The server relaying the file adds
this extra hop and contributes to the latency / slowdown.  I estimate
performance like:

10G interface with 12 disk RAID 6:

GFS Read(replica 1 or 2) = 720 MB/s
GFS Write(replica 1) = 820 MB/s
GFS Write(replica 2) = 410 MB/s

NFS Read(replica 1 or 2) = 535 MB/s
NFS Write(replica 1) = 400 MB/s
NFS Write(replica 2) = 250 MB/s

So with replica 2 Gluster FS I would expect ~410 MB / sec writes and on the same
volume over NFS I would expect 250 MB / sec.  Its not a full 50% but its close.

HTH!

-b

> -Vijay
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>

Thanks, it clarifies write slowdown! But my reads with NFS are as fast as
GlusterFS native client. Does it mean the server which NFS was mounted with
is actually hosting those files so no extra hop and same performance?

Thanks,
On 30 Apr 2015 8:27 am, "Ben Turner" <bturner at redhat.com>
wrote:
> ----- Original Message -----
> > From: "Vijay Bellur" <vbellur at redhat.com>
> > To: "Behrooz Shafiee" <shafiee01 at gmail.com>,
"Gluster-users at gluster.org
> List" <gluster-users at gluster.org>
> > Sent: Thursday, April 30, 2015 6:44:11 AM
> > Subject: Re: [Gluster-users] Poor performance of NFS client for large
> writes compared to native client
> >
> > On 04/30/2015 06:49 AM, Behrooz Shafiee wrote:
> > > Hi,
> > >
> > >   I was comparing GlusterFS native and NFS clients and I noticed,
NFS
> > > client is significantly slower for large writes. I wrote about
200, 1GB
> > > files using a 1MB block sizes and NFS throughput was almost half
of
> > > native client. Can anyone explain why is that?
> > >
> >
> > Depending on where the file gets scheduled, NFS might need an
additional
> > network hop. That can contribute to additional latency and less
> > throughput than the native client.
>
> To tag on here GlusterFS mounts use the hash algorithm to know which
> server to write directly to.  NFS is not aware of this so all files get
> routed through the server that is mounted just like Vijay said.  The server
> relaying the file adds this extra hop and contributes to the latency /
> slowdown.  I estimate performance like:
>
> 10G interface with 12 disk RAID 6:
>
> GFS Read(replica 1 or 2) = 720 MB/s
> GFS Write(replica 1) = 820 MB/s
> GFS Write(replica 2) = 410 MB/s
>
> NFS Read(replica 1 or 2) = 535 MB/s
> NFS Write(replica 1) = 400 MB/s
> NFS Write(replica 2) = 250 MB/s
>
> So with replica 2 Gluster FS I would expect ~410 MB / sec writes and on
> the same volume over NFS I would expect 250 MB / sec.  Its not a full 50%
> but its close.
>
> HTH!
>
> -b
>
>
> > -Vijay
> >
> > _______________________________________________
> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150430/25742268/attachment.html>