Gluster users - Oct 2018 - Gluster 3.12.14: slow ls -l command + "Another transaction is in progress. Please try again after sometime" message

Hi, 

Slow "ls -l" command which is also the reason behind TAB and df -h
issue.
I would suggest to check "disperse.other-eager-lock" option and see if
it is "ON" or "OFF"

gluster v get <volname> all | grep other 

If it is ON, change it to OFF by following command 
gluster v set <volname> disperse.other-eager-lock off 

It should solve the problem. 

--- 
Ashish 




----- Original Message -----

From: "Mauro Tridici" <mauro.tridici at cmcc.it> 
To: "gluster-users" <gluster-users at gluster.org> 
Sent: Wednesday, October 24, 2018 8:20:48 PM 
Subject: [Gluster-users] Gluster 3.12.14: slow ls -l command + "Another
transaction is in progress. Please try again after sometime" message

Dear All, 

during last two days, our distributed dispersed gluster volume is showing some
strange behaviors:

1) "df -h" and "ls -l" linux commands are very slow 
2) ?TAB? key usage causes the hang of the shell 
3) nagios monitoring system is showing some alerts related to QUOTA and VOLUME
UTILIZATION similar to the following one:

Notification Type: PROBLEM 

Service: Volume Quota - tier2 
Host: tier2-gluster 
Address: tier2-gluster 
State: WARNING 

Date/Time: Wed Oct 24 16:27:13 CEST 2018 

Additional Info: 

QUOTA: Quota status could not be determined. 

4) on some gluster servers, ?df -h? command shows a new mount
?/run/gluster/tier2_quota_list?
5) gluster vol status returns this message: ?Another transaction is in progress.
Please try again after sometime?.

I just restarted the ?glusterd? service on each server, one by one. 
Also, I executed ?umount -l /run/gluster/tier2_quota_list? on each server. 
Now, everything seems to be ok, but I would like to know if there is a way to
avoid this kind of problem.

Last question: do you think that it is a good practice executing ?service
glusterd restart? command on each server when the problem appears?
In attachment you can find the glusterd.log file 

Thank you in advance, 
Mauro 







_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
https://lists.gluster.org/mailman/listinfo/gluster-users 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20181025/a04698a3/attachment.html>

Hi Ashish,

"gluster v get  tier2 all | grep other? command returned empty output.

In attachment you can find the output of ?gluster v get tier2 all? command.

Thank you very much,
Mauro



> Il giorno 25 ott 2018, alle ore 09:11, Ashish Pandey <aspandey at
redhat.com> ha scritto:
> 
> 
> Hi,
> 
> Slow "ls -l" command which is also the reason behind TAB and df
-h issue.
> I would suggest to check "disperse.other-eager-lock" option and
see if it is "ON" or "OFF"
> 
> gluster v get  <volname> all | grep other
> 
> If it is ON, change it to OFF by following command
> gluster v set <volname> disperse.other-eager-lock off
> 
> It should solve the problem.
> 
> ---
> Ashish
> 
> 
> 
> 
> From: "Mauro Tridici" <mauro.tridici at cmcc.it>
> To: "gluster-users" <gluster-users at gluster.org>
> Sent: Wednesday, October 24, 2018 8:20:48 PM
> Subject: [Gluster-users] Gluster 3.12.14: slow ls -l command +
"Another        transaction is in progress. Please try again after
sometime" message
> 
> Dear All,
> 
> during last two days, our distributed dispersed gluster volume is showing
some strange behaviors:
> 
> 1) "df -h" and "ls -l" linux commands are very slow
> 2) ?TAB? key usage causes the hang of the shell
> 3) nagios monitoring system is showing some alerts related to QUOTA and
VOLUME UTILIZATION similar to the following one:
> 
> Notification Type: PROBLEM
> 
> Service: Volume Quota - tier2
> Host: tier2-gluster
> Address: tier2-gluster
> State: WARNING
> 
> Date/Time: Wed Oct 24 16:27:13 CEST 2018
> 
> Additional Info:
> 
> QUOTA: Quota status could not be determined.
> 
> 4) on some gluster servers, ?df -h? command shows a new mount
?/run/gluster/tier2_quota_list?
> 5) gluster vol status returns this message: ?Another transaction is in
progress. Please try again after sometime?.
> 
> I just restarted the ?glusterd? service on each server, one by one.
> Also, I executed ?umount -l /run/gluster/tier2_quota_list? on each server.
> Now, everything seems to be ok, but I would like to know if there is a way
to avoid this kind of problem.
> 
> Last question: do you think that it is a good practice executing ?service
glusterd restart? command on each server when the problem appears?
> In attachment you can find the glusterd.log file
> 
> Thank you in advance,
> Mauro
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20181025/cb9d7b75/attachment.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: get_all_options.txt
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20181025/cb9d7b75/attachment.txt>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20181025/cb9d7b75/attachment-0001.html>