Gluster users - Aug 2019 - Brick Reboot => VMs slowdown, client crashes

You also need to make sure your volume is setup properly for best performance.
Did you apply the gluster virt group to your volumes, or at least features.shard
= on on your VM volume?
> On Aug 19, 2019, at 11:05 AM, Carl Sirotic <csirotic at
evoqarchitecture.com> wrote:
> 
> Yes, I made sure there was no heal.
> This is what I am suspecting thet shutting down a host isn't the right
way to go.
> Hi Carl, Did you check for any pending heals before rebooting the gluster
server? Also, it was discussed that shutting down the node, does not stop the
bricks properly and thus the clients will eait for a timeout before restoring
full functionality.  You can stop your glusterd and actually all processes by
using a script in /usr/share/gluster/scripts (the path is based on memory and
could be wrong). Best Regards, Strahil NikllovOn Aug 19, 2019 18:34, Carl
Sirotic wrote: > > Hi, > > we have a replicate 3 cluster. > >
2 other servers are clients that run VM that are stored on the Gluster >
volumes. > > I had to reboot one of the brick for maintenance. > >
The whole VM setup went super slow and some of the client crashed. > > I
think there is some timeout setting for KVM/Qemu vs timeout of > Glusterd
that could fix this. > > Do anyone have an idea ? > > The whole
point of having gluster for me was to be able to shut down one > of the host
while the vm stay running. > > > Carl > >
_______________________________________________ > Gluster-users mailing list
> Gluster-users at gluster.org >
https://lists.gluster.org/mailman/listinfo/gluster-users AVIS DE CONFIDENTIALIT?
: Ce courriel peut contenir de l'information privil?gi?e et confidentielle.
Nous vous demandons de le d?truire imm?diatement si vous n'?tes pas le
destinataire. CONFIDENTIALITY NOTICE: This email may contain information that is
privileged and confidential. Please delete immediately if you are not the
intended recipient. _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users

On 2019-08-19 12:08 p.m., Darrell Budic wrote:
> You also need to make sure your volume is setup properly for best
performance. Did you apply the gluster virt group to your volumes, or at least
features.shard = on on your VM volume?
That's what we did here:


gluster volume set W2K16_Rhenium cluster.quorum-type auto
gluster volume set W2K16_Rhenium network.ping-timeout 10
gluster volume set W2K16_Rhenium auth.allow \*
gluster volume set W2K16_Rhenium group virt
gluster volume set W2K16_Rhenium storage.owner-uid 36
gluster volume set W2K16_Rhenium storage.owner-gid 36
gluster volume set W2K16_Rhenium features.shard on
gluster volume set W2K16_Rhenium features.shard-block-size 256MB
gluster volume set W2K16_Rhenium cluster.data-self-heal-algorithm full
gluster volume set W2K16_Rhenium performance.low-prio-threads 32

tuned-adm profile random-io??? ??? (a profile i added in CentOS 7)


cat /usr/lib/tuned/random-io/tuned.conf
==========================================[main]
summary=Optimize for Gluster virtual machine storage
include=throughput-performance

[sysctl]

vm.dirty_ratio = 5
vm.dirty_background_ratio = 2


Any more optimization to add to this?


Guy

-- 
Guy Boisvert, ing.
IngTegration inc.
http://www.ingtegration.com
https://www.linkedin.com/in/guy-boisvert-8990487

AVIS DE CONFIDENTIALITE : ce message peut contenir des
renseignements confidentiels appartenant exclusivement a
IngTegration Inc. ou a ses filiales. Si vous n'etes pas
le destinataire indique ou prevu dans ce  message (ou
responsable de livrer ce message a la personne indiquee ou
prevue) ou si vous pensez que ce message vous a ete adresse
par erreur, vous ne pouvez pas utiliser ou reproduire ce
message, ni le livrer a quelqu'un d'autre. Dans ce cas, vous
devez le detruire et vous etes prie d'avertir l'expediteur
en repondant au courriel.

CONFIDENTIALITY NOTICE : Proprietary/Confidential Information
belonging to IngTegration Inc. and its affiliates may be
contained in this message. If you are not a recipient
indicated or intended in this message (or responsible for
delivery of this message to such person), or you think for
any reason that this message may have been addressed to you
in error, you may not use or copy or deliver this message to
anyone else. In such case, you should destroy this message
and are asked to notify the sender by reply email.