Gluster users - Feb 2017 - Poor Gluster performance and standard Linux cron jobs.

An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170227/012232c8/attachment.html>

On Mon, Feb 27, 2017 at 12:34:23PM -0800, Ernie Dunbar
wrote:
>    Hi everyone!
> 
>    After a bit of an ordeal with our Gluster servers last week, I
discovered
>    some very important coincidences that can very badly affect Gluster
>    performance when they occur.
> 
>    Should the Mlocate updater start when Gluster is going through the
>    self-heal process, especially one that is very long (ours can take a few
>    days sometimes), Gluster's performance can become extremely poor
with
>    symptoms of high server load, moderate to high CPU usage, high disk IO
>    activity, and occasional to lengthy times where there is 100% disk
>    utilization. IO Wait times on any clients can be high to extreme. Memory
>    usage might also be higher than normal, and on servers with less than
say,
>    8 GB of RAM, this might cause RAM to be swapped to disk, exacerbating
the
>    problem further.
> 
>    The Mlocate update program "updatedb" is installed by default
on most
>    Linux implementations, and is run nightly by default in the daily cron
>    jobs. I have added a new file to the Gluster installation documentation
>    (named "post-installation") with the recommendation that this
daily cron
>    job be removed. Removing this job would have little to no impact on the
>    operation of Gluster's operating system.?
> 
>    If anyone would like to test and verify this in a non-production
>    environment, be my guest.
You are absolutely correct. We have seen problems with this before, and
I have filed a bug for Fedora to get this fixed in mlocate:
  https://bugzilla.redhat.com/show_bug.cgi?id=1331944

Other distributions probably need a similar fix. I recommend other users
to check the mlocate package for their distribution and report bugs
against them too. It will help others if everyone that reports a bug
replies to this email so that all links are archived in the mailinglist.

Thanks!
Niels
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20170227/fa70db68/attachment.sig>