Gluster users - Apr 2018 - lstat & readlink calls during glusterfsd process startup

Hi all,

I am on gluster 3.10.5 with one EC volume 16+4.
One of the machines go down previous night and I just fixed it and powered on.
When glusterfsd processes started they consume all CPU on the server.
strace shows every process goes over in bricks directory and do a
lstat & readlink calls.
Each brick directory is 8TB, %60 full. I waited for 24 hours for it to
finish but it did not.
I stopped glusterd and restarted it but same thing happens again. Why
on startup glusterfsd processes traverse brick directory? Is it
related to self heal?

This happened one time before and I somehow prevent it happening with
glusterd stop&start or some other way I cannot remember right now.

Any thoughts how to solve this issue?

This is an example from one of the glusterfsd processes, strace -f -c
-p pid_of_glusterfsd

%time seconds usecs/call calls       errors     syscall
68       36.2       2131         17002     4758      futex
13        7           5783         1206                   epoll_wait
11        5.4        360545       15                      select
...

---------- Forwarded message ----------
From: Serkan ?oban <cobanserkan at gmail.com>
Date: Mon, Apr 16, 2018 at 9:20 AM
Subject: lstat & readlink calls during glusterfsd process startup
To: Gluster Users <gluster-users at gluster.org>


Hi all,

I am on gluster 3.10.5 with one EC volume 16+4.
One of the machines go down previous night and I just fixed it and powered on.
When glusterfsd processes started they consume all CPU on the server.
strace shows every process goes over in bricks directory and do a
lstat & readlink calls.
Each brick directory is 8TB, %60 full. I waited for 24 hours for it to
finish but it did not.
I stopped glusterd and restarted it but same thing happens again. Why
on startup glusterfsd processes traverse brick directory? Is it
related to self heal?

This happened one time before and I somehow prevent it happening with
glusterd stop&start or some other way I cannot remember right now.

Any thoughts how to solve this issue?