Gluster users - Oct 2015 - Tuning for small files

> Where you run into problems with smallfiles on gluster is latency of
sending
> data over the wire.  For every smallfile create there are a bunch of
different
> file opetations we have to do on every file.  For example we will have
to do
> at least 1 lookup per brick to make sure that the file doesn't exist
anywhere
> before we create it.  We actually got it down to 1 per brick with lookup
> optimize on, its 2 IIRC(maybe more?) with it disabled.
Is this lookup optimize something that needs to be enabled manually with
3.7, and if so, how?

Thanks

----- Original Message -----
> From: "Iain Milne" <glusterfs at noognet.org>
> To: gluster-users at gluster.org
> Sent: Wednesday, September 30, 2015 2:48:57 AM
> Subject: Re: [Gluster-users] Tuning for small files
> 
> > Where you run into problems with smallfiles on gluster is latency of
> sending
> > data over the wire.  For every smallfile create there are a bunch of
> different
> > file opetations we have to do on every file.  For example we will have
> to do
> > at least 1 lookup per brick to make sure that the file doesn't
exist
> anywhere
> > before we create it.  We actually got it down to 1 per brick with
lookup
> > optimize on, its 2 IIRC(maybe more?) with it disabled.
> 
> Is this lookup optimize something that needs to be enabled manually with
> 3.7, and if so, how?
Here are all 3 of the settings I was talking about:

    gluster v set testvol client.event-threads 4
    gluster v set testvol server.event-threads 4
    gluster v set testvol performance.lookup-optimize on

Yes, lookup optimize needs to be enabled.

-b
> Thanks
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>

Are these options enabled by default, at least for new volumes in 3.7?
If not, why?

Regards
Andr?

Am 30.09.2015 um 16:22 schrieb Ben Turner:
> ----- Original Message -----
>> From: "Iain Milne" <glusterfs at noognet.org>
>> To: gluster-users at gluster.org
>> Sent: Wednesday, September 30, 2015 2:48:57 AM
>> Subject: Re: [Gluster-users] Tuning for small files
>>
>>> Where you run into problems with smallfiles on gluster is latency
of
>> sending
>>> data over the wire.  For every smallfile create there are a bunch
of
>> different
>>> file opetations we have to do on every file.  For example we will
have
>> to do
>>> at least 1 lookup per brick to make sure that the file doesn't
exist
>> anywhere
>>> before we create it.  We actually got it down to 1 per brick with
lookup
>>> optimize on, its 2 IIRC(maybe more?) with it disabled.
>>
>> Is this lookup optimize something that needs to be enabled manually
with
>> 3.7, and if so, how?
> 
> Here are all 3 of the settings I was talking about:
> 
>     gluster v set testvol client.event-threads 4
>     gluster v set testvol server.event-threads 4
>     gluster v set testvol performance.lookup-optimize on
> 
> Yes, lookup optimize needs to be enabled.
> 
> -b
> 
>> Thanks
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://www.gluster.org/mailman/listinfo/gluster-users
>>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
> 

-- 
Mit freundlichen Gr??en
Andr? Bauer

MAGIX Software GmbH
Andr? Bauer
Administrator
August-Bebel-Stra?e 48
01219 Dresden
GERMANY

tel.: 0351 41884875
e-mail: abauer at magix.net
abauer at magix.net <mailto:Email>
www.magix.com <http://www.magix.com/>


Gesch?ftsf?hrer | Managing Directors: Dr. Arnd Schr?der, Michael Keith
Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205

Find us on:

<http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de>
<http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
----------------------------------------------------------------------
The information in this email is intended only for the addressee named
above. Access to this email by anyone else is unauthorized. If you are
not the intended recipient of this message any disclosure, copying,
distribution or any action taken in reliance on it is prohibited and
may be unlawful. MAGIX does not warrant that any attachments are free
from viruses or other defects and accepts no liability for any losses
resulting from infected email transmissions. Please note that any
views expressed in this email may be those of the originator and do
not necessarily represent the agenda of the company.
----------------------------------------------------------------------