I seem to be running into an issue where gluster reports files are missing and the real problem seems to happen when it refers to these files via gfid rather than path. I see instances in my logs where it referred to files by path and while this may be a problem, services were still running. Right now the apache processes serving files off gluster are hung in an uninterruptible state holding open a file which lsof lists as deleted and my gluster client logs show missing files referenced by gfid. Anyone have any idea how I might go about troubleshooting this? [2013-01-22 08:05:22.874323] W [client3_1-fops.c:474:client3_1_stat_cbk] 0-WEBSTATS-client-1: remote operation failed: No such file or directory [2013-01-22 08:05:22.874884] W [client3_1-fops.c:474:client3_1_stat_cbk] 0-WEBSTATS-client-0: remote operation failed: No such file or directory [2013-01-22 08:05:22.877501] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-WEBSTATS-client-1: remote operation failed: No such file or directory. Path: <gfid:9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981> (9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981). Key: trusted.glusterfs.dht.linkto [2013-01-22 08:05:22.878042] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-WEBSTATS-client-0: remote operation failed: No such file or directory. Path: <gfid:9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981> (9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981). Key: trusted.glusterfs.dht.linkto [2013-01-22 08:05:22.885954] W [client3_1-fops.c:474:client3_1_stat_cbk] 0-WEBSTATS-client-1: remote operation failed: No such file or directory [2013-01-22 08:05:22.886492] W [client3_1-fops.c:474:client3_1_stat_cbk] 0-WEBSTATS-client-0: remote operation failed: No such file or directory [2013-01-22 08:05:22.889113] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-WEBSTATS-client-1: remote operation failed: No such file or directory. Path: <gfid:9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981> (9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981). Key: trusted.glusterfs.dht.linkto [2013-01-22 08:05:22.889809] W [client3_1-fops.c:1059:client3_1_getxattr_cbk] 0-WEBSTATS-client-0: remote operation failed: No such file or directory. Path: <gfid:9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981> (9c1ab9dc-26e6-4e59-8f5c-2a926ee1a981). Key: trusted.glusterfs.dht.linkto -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130122/48adb3ea/attachment.html>