Gluster users - Jun 2015 - Help: Folder hangs on concurrent accesses (Gluster 3.7 / Ubuntu 14.04)

Hi,

 

we got a setup of 3 machines running gluster and acting as both client and
server. We upgraded from gluster 3.4 to 3.7 yesterday evening /using the ppa
from Launchpad for Ubuntu 14.04). We got a web application, which creates a
.locked file in a shared folder to synchronize. This file is created and
deleted continuously.

 

Since upgrading, hit the following problem 3 times. We first placed our file
in a "config" folder and later in a new folder called
"settings"  but the
situation is always the same: All of the sudden, the full folder (Containing
3 files including our .locked) is not readable anymore. 

 

We can make an ls ./config/*  and the process never stops on the shell.
Removing the folder via rm -R ./config  doesn't return either. The CPU usage
doesn't change and none of the logs at /var/log/gluster/ contains any
additional/unusual entries when we try to ls or rm the folder. We can,
however, move/rename the folder but afterwards we're still not able to
delete or ls it.

 

However, when restarting gluster via 

/etc/init.d/glusterfs-server restart

on one of the 3 servers, the folder can be accessed and deleted normally
once, the gluster server is back again.

 

Before restarting we checked lsof  but neither the folder nor any files
inside where in use. We also closed all processed before restarting. I
checked the raw data of gluster (the data folder) and saw that the .locked
file was not inside. However, after restarting, the folder was accessible as
explained above AND the .locked file appeared inside. The timestamp is the
time of the gluster restart.

 

For me the first impression is that gluster hangs when creating the .locked
file, which is then rewritten after restart. However, the processes writing
the .locked file had been killed so it looks like the .locked file is indeed
created by gluster.

 

 

 

Best greetings,

 

Sven

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150615/527dcf91/attachment.html>