Gluster users - Jul 2019 - Graceful gluster server retire/poweroff

Hi list,
I have a replica 3 test cluster and I have a question about how clients 
behave to an host shutdown.

If I suddenly switch off one of the gluster server, the connected 
clients see a ~42s stall in I/O: this is expected, as it is the default 
client timeout.

However, it is possible to *gracefully* shutdown a running gluster 
server, *without* impacting a running client? Ie: putting the affected 
server in a "maintenance mode" of sort.

Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti at assyoma.it - info at assyoma.it
GPG public key ID: FF5F32A8

On 16/07/19 3:16 PM, Gionatan Danti wrote:
> Hi list,
> I have a replica 3 test cluster and I have a question about how 
> clients behave to an host shutdown.
>
> If I suddenly switch off one of the gluster server, the connected 
> clients see a ~42s stall in I/O: this is expected, as it is the 
> default client timeout.
>
> However, it is possible to *gracefully* shutdown a running gluster 
> server, *without* impacting a running client? Ie: putting the affected 
> server in a "maintenance mode" of sort.
Yes, if you simply pkill the gluster brick processes of the node before 
switching it off, you won't observe the hang on the clients because they 
will receive the disconnect notification immediately. But before that, 
you would need to check if there are no pending heals etc. You can use 
the script [1] which does all these checks in the graceful mode.

HTH,
Ravi

[1] 
https://github.com/gluster/glusterfs/blob/master/extras/stop-all-gluster-processes.sh
>
> Thanks.
>

On 16/07/2019 15:27, Ravishankar N wrote:
> Yes, if you simply pkill the gluster brick processes of the node before 
> switching it off, you won't observe the hang on the clients because
they
> will receive the disconnect notification immediately. But before that, 
> you would need to check if there are no pending heals etc. You can use 
> the script [1] which does all these checks in the graceful mode.
Hi Ravi,
thanks for your reply. I tried killing the glusterfsd process and I 
confirm that the client does not see any pause, indeed.

I was thinking that during the shutdown procedure systemd would kill the 
processes by itself; which it *does*, but only if the glusterfsd.service 
is enabled/started.

By default, the systemd service installed on CentOS 7 + Gluster 6.0 SIG 
does not start glusterfsd.service and so, in the shutdown phase, it does 
not stop it.

Thanks very much for your information.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti at assyoma.it - info at assyoma.it
GPG public key ID: FF5F32A8