Carlo Rodrigues
2024-Jul-05 15:06 UTC
[Gluster-users] Problems creating or renaming directories and files in gluster volume, via SAMBA
Hello all,
We have a distributed volume running in 7 hosts and 28 bricks.
We've been experiencing some strange behaviors over the time, some of them
are solved when the gluster services are restarted, but other problems persist.
In the case below, and this one happens many times, some people access the
gluster filesystem via Samba.
They have a mapped drive in their windows machine.
Most of the time things work as expected, but a recurrent issue is that
sometimes when directories are created, they get a "permission denied"
message. Apparently, the directories are created.
This sometimes happens also when they are renaming directories. And other times
they get a message telling them that the files/directories do not exist.
There are some logs below.
[2024-06-18 14:02:52.931157 +0000] W [MSGID: 109037]
[dht-common.c:4711:dht_getxattr_get_real_filename_cbk] 0-tank-dht: Failed to get
real filename. [No such file or directory]
[2024-06-18 14:02:52.934395 +0000] W [MSGID: 114031]
[client-rpc-fops_v2.c:2097:client4_0_create_cbk] 0-tank-client-5: remote
operation failed. [{path=/dir59/BallData.mat}, {errno=2}, {error=No such file or
directory}]
[2024-06-18 14:04:52.221379 +0000] W [MSGID: 114031]
[client-rpc-fops_v2.c:2097:client4_0_create_cbk] 0-tank-client-5: remote
operation failed. [{path=/dir59/BallData.mat}, {errno=2}, {error=No such file or
directory}]
Here we have multiple attempts to rename a file
[2024-06-18 14:05:54.992761 +0000] I [MSGID: 109064]
[dht-rename.c:1955:dht_rename] 0-tank-dht: renaming /dir59/BallData1.mat
(ae1ddf78-5e3d-470c-8a4c-cedca17829b8) (hash=tank-client-4/cache=tank-client-4)
=> /dir59/BallData.mat ((null)) (hash=tank-client-5/cache=<nul>)
[2024-06-18 14:05:55.527004 +0000] I [MSGID: 109064]
[dht-rename.c:1955:dht_rename] 0-tank-dht: renaming /dir59/BallData1.mat
(ae1ddf78-5e3d-470c-8a4c-cedca17829b8) (hash=tank-client-4/cache=tank-client-4)
=> /dir59/BallData.mat ((null)) (hash=tank-client-5/cache=<nul>)
The message "I [MSGID: 109064] [dht-rename.c:1955:dht_rename] 0-tank-dht:
renaming /dir59/BallData1.mat (ae1ddf78-5e3d-470c-8a4c-cedca17829b8)
(hash=tank-client-4/cache=tank-client-4) => /dir59/BallData.mat ((null))
(hash=tank-client-5/cache=<nul>) " repeated 2 times between
[2024-06-18 14:05:55.527004 +0000] and [2024-06-18 14:06:11.390068 +0000]
[2024-06-18 14:06:13.417710 +0000] I [MSGID: 109064]
[dht-rename.c:1955:dht_rename] 0-tank-dht: renaming /dir59/BallData1.mat
(ae1ddf78-5e3d-470c-8a4c-cedca17829b8) (hash=tank-client-4/cache=tank-client-4)
=> /dir59/BallData.mat ((null)) (hash=tank-client-5/cache=<nul>)
There are many messages in the logs like the ones below
[2024-06-16 12:28:07.720713 +0000] W [dict.c:1532:dict_get_with_ref]
(-->/usr/lib64/glusterfs/9.4/xlator/performance/md-cache.so(+0x11683)
[0x7f998d68f683] -->/lib64/libglusterfs.so.0(dict_get_iatt+0x40)
[0x7f999507bd10] -->/lib64/libglusterfs.so.0(dict_get_with_ref+0x85)
[0x7f99950799b5] ) 0-dict: dict OR key (virt-gf-prestat) is NULL [Invalid
argument]
[2024-06-16 12:28:29.099386 +0000] W [MSGID: 114061]
[client-common.c:3327:client_pre_readdirp_v2] 0-tank-client-0: remote_fd is -1.
EBADFD [{gfid=8def2ad4-72fd-49ed-af12-4b1f9878b2a4}, {errno=77}, {error=File
descriptor in bad state}]
[2024-06-16 12:37:56.662296 +0000] I [MSGID: 109061]
[dht-layout.c:641:dht_layout_normalize] 0-tank-dht: Found anomalies
[{path=/dir59}, {gfid=3e58f9ed-8565-4ce2-bbbb-77b18b841929}, {holes=3},
{overlaps=0}]
[2024-06-16 12:37:56.663670 +0000] W [MSGID: 109005]
[dht-selfheal.c:1064:dht_selfheal_dir_mkdir_cbk] 0-tank-dht: Healing of path
failed [{path=/dir59}, {gfid=3e58f9ed-8565-4ce2-bbbb-77b18b841929}, {errno=2},
{error=No such file or directory}]
[2024-06-16 12:37:56.672373 +0000] W [dict.c:1532:dict_get_with_ref]
(-->/usr/lib64/glusterfs/9.4/xlator/cluster/distribute.so(+0x516c6)
[0x7f998e3426c6] -->/lib64/libglusterfs.so.0(dict_get_uint32+0x3b)
[0x7f999507a5bb] -->/lib64/libglusterfs.so.0(dict_get_with_ref+0x85)
[0x7f99950799b5] ) 0-dict: dict OR key (trusted.glusterfs.dht.commithash) is
NULL [Invalid argument]
Did any of you been through this before? The "permission denied"
happens randomly.
About the renaming of files/directories, sometimes I think that the renames are
being done, but the windows client obtains an error and is not informed of the
success of the operation, and then, when they try again to rename, the original
filename does not exist anymore, and they get "File not Found". But
I'm not sure of this.
Thanks.
Carlo Rodrigues
Cloud & Infra Engineer
carlo.rodrigues at claranet.com
T +351 21 319 92 00
[Logo Description automatically generated]
Claranet Portugal
www.claranet.pt<http://www.claranet.pt/>
Todas as informa??es contidas nesta mensagem eletr?nica da Claranet est?o
abrangidas pelo disclaimer dispon?vel em
https://www.claranet.pt/email-disclaimer
All the information contained within this electronic message from Claranet is
covered by the disclaimer at https://www.claranet.pt/email-disclaimer-ENG
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20240705/927245e2/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 7171 bytes
Desc: image001.png
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20240705/927245e2/attachment.png>