I am seeing the following on one of my FUSE clients (indy.rst and
indy.rst.old has ??? ???).
Has anyone seen this before? Any idea what causes this for a given
client?
If I try to access the file, I get a stale file handle..
# cp indy.rst dfr.rst
cp: cannot stat `indy.rst': Stale file handle
Bottom of the log has:
[2015-02-10 22:20:34.913632] I [dht-rename.c:1344:dht_rename]
0-homegfs-dht: renaming
/hpc_shared/motorsports/gmics/Raven/p4/133/dc3.tmp
(hash=homegfs-replicate-2/cache=homegfs-replicate-2) =>
/hpc_shared/motorsports/gmics/Raven/p4/133/data_collected3
(hash=homegfs-replicate-3/cache=homegfs-replicate-2)
[2015-02-10 22:40:04.138594] W
[client-rpc-fops.c:504:client3_3_stat_cbk] 0-homegfs-client-1: remote
operation failed: Stale file handle
[2015-02-10 22:40:04.158855] W [MSGID: 108008]
[afr-read-txn.c:221:afr_read_txn] 0-homegfs-replicate-0: Unreadable
subvolume -1 found with event generation 2. (Possible split-brain)
[2015-02-10 22:40:04.202696] W [fuse-bridge.c:779:fuse_attr_cbk]
0-glusterfs-fuse: 1396664: STAT()
/hpc_shared/motorsports/gmics/Raven/p3/70_sst_r4_1em3/indy.rst.old => -1
(Stale file handle)
The message "W [MSGID: 108008] [afr-read-txn.c:221:afr_read_txn]
0-homegfs-replicate-0: Unreadable subvolume -1 found with event
generation 2. (Possible split-brain)" repeated 14 times between
[2015-02-10 22:40:04.158855] and [2015-02-10 22:41:00.610296]
[2015-02-10 22:41:45.339419] W [MSGID: 108008]
[afr-read-txn.c:221:afr_read_txn] 0-homegfs-replicate-0: Unreadable
subvolume -1 found with event generation 2. (Possible split-brain)
The message "W [MSGID: 108008] [afr-read-txn.c:221:afr_read_txn]
0-homegfs-replicate-0: Unreadable subvolume -1 found with event
generation 2. (Possible split-brain)" repeated 31 times between
[2015-02-10 22:41:45.339419] and [2015-02-10 22:43:11.483421]
[2015-02-10 22:43:37.498720] W [MSGID: 108008]
[afr-read-txn.c:221:afr_read_txn] 0-homegfs-replicate-0: Unreadable
subvolume -1 found with event generation 2. (Possible split-brain)
However, the same files on my other FUSE clients look fine:
>From the storage system:
>From another client:
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150210/fca38a72/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: zgcmbmkb.png
Type: image/png
Size: 7158 bytes
Desc: not available
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150210/fca38a72/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0qmo2cgm.png
Type: image/png
Size: 12985 bytes
Desc: not available
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150210/fca38a72/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tboeksdo.png
Type: image/png
Size: 19955 bytes
Desc: not available
URL:
<http://www.gluster.org/pipermail/gluster-users/attachments/20150210/fca38a72/attachment-0002.png>