[Gluster-users] Tuning for small files

Ben Turner
Mon, 28 Sep 2015 21:37:49 -0000

----- Original Message -----
> From: hmlth at t-hamel.fr
> To: abauer at magix.net
> Cc: gluster-users at gluster.org
> Sent: Monday, September 28, 2015 7:40:52 AM
> Subject: Re: [Gluster-users] Tuning for small files
>
> I'm also quite interested by small files performances optimization, but
> I'm a bit confused about the best option between 3.6/3.7.
>
> Ben Turner was saying that 3.6 might give the best performances:
> http://www.gluster.org/pipermail/gluster-users/2015-September/023733.html
>
> What kind of gain is expected (with consistent-metadata) if this
> regression is solved?

Just to be clear, the issue I am talking about is metadata only(think ls -l or
file browsing).  It doesn't affect small file perf(well not that much,
I'm sure a little, but I have never quantified it), with server and client
event threads set to 4 + lookup optimize I see between a 200-300% gain on my
systems on 3.7 vs 3.6 builds.  If I needed fast metadata I would go with 3.6, if
I need fast smallfile I would go with 3.7.  If I needed both I would pick the
less of the two evils and go with that one and upgrade when the fix is released.

-b


>
> I tried 3.6.5 (last version for debian jessie), and it's a bit better
> than 3.7.4 but not by much (10-15%).
>
> I was also wondering if there is recommendations for the underlying file
> system of the bricks (xfs, ext4, tuning...).
>
>
> Regards
>
> Thomas HAMEL
>
> On 2015-09-28 12:04, Andr? Bauer wrote:
> > If you're not already on Glusterfs 3.7.x i would recommend an
update
> > first.
> >
> > Am 25.09.2015 um 17:49 schrieb Thibault Godouet:
> >> Hi,
> >>
> >> There are quite a few tuning parameters for Gluster (as seen in
> >> Gluster
> >> volume XYZ get all), but I didn't find much documentation on
those.
> >> Some people do seem to set at least some of them, so the knowledge
> >> must
> >> be somewhere...
> >>
> >> Is there a good source of information to understand what they
mean,
> >> and
> >> recommendation on how to set them to get a good small file
> >> performance?
> >>
> >> Basically what I'm trying to optimize is for svn operations
(e.g. svn
> >> checkout, or svn branch) on a replicated 2 x 1 volume (hosted on 2
> >> VMs,
> >> 16GB ram, 4 cores each, 10Gb/s network tested at full speed),
using a
> >> NFS mount which appears much faster than fuse in this case (but
still
> >> much slower than when served by a normal NFS server).
> >> Any recommendation for such a setup?
> >>
> >> Thanks,
> >> Thibault.
> >>
> >>
> >>
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://www.gluster.org/mailman/listinfo/gluster-users
> >>
> >
> >
> > --
> > Mit freundlichen Gr??en
> > Andr? Bauer
> >
> > MAGIX Software GmbH
> > Andr? Bauer
> > Administrator
> > August-Bebel-Stra?e 48
> > 01219 Dresden
> > GERMANY
> >
> > tel.: 0351 41884875
> > e-mail: abauer at magix.net
> > abauer at magix.net <mailto:Email>
> > www.magix.com <http://www.magix.com/>
> >
> >
> > Gesch?ftsf?hrer | Managing Directors: Dr. Arnd Schr?der, Michael Keith
> > Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205
> >
> > Find us on:
> >
> > <http://www.facebook.com/MAGIX>
<http://www.twitter.com/magix_de>
> > <http://www.youtube.com/wwwmagixcom>
<http://www.magixmagazin.de>
> > ----------------------------------------------------------------------
> > The information in this email is intended only for the addressee named
> > above. Access to this email by anyone else is unauthorized. If you are
> > not the intended recipient of this message any disclosure, copying,
> > distribution or any action taken in reliance on it is prohibited and
> > may be unlawful. MAGIX does not warrant that any attachments are free
> > from viruses or other defects and accepts no liability for any losses
> > resulting from infected email transmissions. Please note that any
> > views expressed in this email may be those of the originator and
do> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users