search for: client4_0_lookup_cbk

...in bad state}] [2024-01-20 07:23:58.561177 +0000] E [MSGID: 108028] [afr-open.c:361:afr_is_reopen_allowed_cbk] 0-workdata-replicate-3: Failed getlk for faf59566-10f5-4ddd-8b0c-a87bc6a334fb [File descriptor in bad state] [2024-01-20 07:23:58.562151 +0000] W [MSGID: 114031] [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-11: remote operation failed. [{path=<gfid:faf59566-10f5-4ddd-8b0c-a87bc6a334fb>}, {gfid=faf59566-10f5-4ddd-8b0c-a87b c6a334fb}, {errno=2}, {error=No such file or directory}] [2024-01-20 07:23:58.562296 +0000] W [MSGID: 114061] [client-common.c:530:client_pre_flush_v2] 0-wor...

ok, finally managed to get all servers, volumes etc runnung, but took a couple of restarts, cksum checks etc. One problem: a volume doesn't heal automatically or doesn't heal at all. gluster volume status Status of volume: workdata Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------

...in bad state}] [2024-01-20 07:23:58.561177 +0000] E [MSGID: 108028] [afr-open.c:361:afr_is_reopen_allowed_cbk] 0-workdata-replicate-3: Failed getlk for faf59566-10f5-4ddd-8b0c-a87bc6a334fb [File descriptor in bad state] [2024-01-20 07:23:58.562151 +0000] W [MSGID: 114031] [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-11: remote operation failed. [{path=<gfid:faf59566-10f5-4ddd-8b0c-a87bc6a334fb>}, {gfid=faf59566-10f5-4ddd-8b0c-a87b c6a334fb}, {errno=2}, {error=No such file or directory}] [2024-01-20 07:23:58.562296 +0000] W [MSGID: 114061] [client-common.c:530:client_pre_flush_v2] 0-wor...

...0 07:23:58.561177 +0000] E [MSGID: 108028] > [afr-open.c:361:afr_is_reopen_allowed_cbk] 0-workdata-replicate-3: > Failed getlk for faf59566-10f5-4ddd-8b0c-a87bc6a334fb [File descriptor > in bad state] > [2024-01-20 07:23:58.562151 +0000] W [MSGID: 114031] > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-11: > remote operation failed. > [{path=<gfid:faf59566-10f5-4ddd-8b0c-a87bc6a334fb>}, > {gfid=faf59566-10f5-4ddd-8b0c-a87b > c6a334fb}, {errno=2}, {error=No such file or directory}] > [2024-01-20 07:23:58.562296 +0000] W [MSGID: 114061] > [client-common.c:...

...have any suggestions on how to go about doing this? I've built 3 vms to be a new test cluster, but if I copy over a file from the 3 nodes and try to read it, I can't and get errors in /var/log/glusterfs/foo.log: [2023-08-12 03:50:47.638134 +0000] W [MSGID: 114031] [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-gv-client-0: remote operation failed. [{path=/helmetpart.scad}, {gfid=00000000-0000-0000-0000-000000000000} , {errno=61}, {error=No data available}] [2023-08-12 03:50:49.834859 +0000] E [MSGID: 122066] [ec-common.c:1301:ec_prepare_update_cbk] 0-gv-disperse-0: Unable to get config xattr. FOP : ...

...0 07:23:58.561177 +0000] E [MSGID: 108028] > [afr-open.c:361:afr_is_reopen_allowed_cbk] 0-workdata-replicate-3: > Failed getlk for faf59566-10f5-4ddd-8b0c-a87bc6a334fb [File descriptor > in bad state] > [2024-01-20 07:23:58.562151 +0000] W [MSGID: 114031] > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-11: > remote operation failed. > [{path=<gfid:faf59566-10f5-4ddd-8b0c-a87bc6a334fb>}, > {gfid=faf59566-10f5-4ddd-8b0c-a87b > c6a334fb}, {errno=2}, {error=No such file or directory}] > [2024-01-20 07:23:58.562296 +0000] W [MSGID: 114061] > [client-common.c:...

...t; 10.4, but doesn't with version 11. What could be wrong? > > > > > > I only see these errors on one of the "good" servers in glustershd.log: > > > > > > [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] > > > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: > > > remote operation failed. > > > [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, > > > {gfid=cb39a1e4-2a4c-4727-861d-3ed9e > > > f00681b}, {errno=2}, {error=No such file or directory}] > > > [2024-01-18 06:08:57.59405...

...stantly e.g. after server reboot with version > 10.4, but doesn't with version 11. What could be wrong? > > I only see these errors on one of the "good" servers in glustershd.log: > > [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: > remote operation failed. > [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, > {gfid=cb39a1e4-2a4c-4727-861d-3ed9e > f00681b}, {errno=2}, {error=No such file or directory}] > [2024-01-18 06:08:57.594051 +0000] W [MSGID: 114031] > [client-rpc-fops_v...

...er brick. Heal was starting instantly e.g. after server reboot with version 10.4, but doesn't with version 11. What could be wrong? I only see these errors on one of the "good" servers in glustershd.log: [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: remote operation failed. [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, {gfid=cb39a1e4-2a4c-4727-861d-3ed9e f00681b}, {errno=2}, {error=No such file or directory}] [2024-01-18 06:08:57.594051 +0000] W [MSGID: 114031] [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk]...

...stantly e.g. after server reboot with version > 10.4, but doesn't with version 11. What could be wrong? > > I only see these errors on one of the "good" servers in glustershd.log: > > [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: > remote operation failed. > [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, > {gfid=cb39a1e4-2a4c-4727-861d-3ed9e > f00681b}, {errno=2}, {error=No such file or directory}] > [2024-01-18 06:08:57.594051 +0000] W [MSGID: 114031] > [client-rpc-fops_v...

...boot with version > > 10.4, but doesn't with version 11. What could be wrong? > > > > I only see these errors on one of the "good" servers in glustershd.log: > > > > [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] > > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: > > remote operation failed. > > [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, > > {gfid=cb39a1e4-2a4c-4727-861d-3ed9e > > f00681b}, {errno=2}, {error=No such file or directory}] > > [2024-01-18 06:08:57.594051 +0000] W [MSGID: 114031...

...>>> 10.4, but doesn't with version 11. What could be wrong? >>> >>> I only see these errors on one of the "good" servers in glustershd.log: >>> >>> [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] >>> [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: >>> remote operation failed. >>> [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, >>> {gfid=cb39a1e4-2a4c-4727-861d-3ed9e >>> f00681b}, {errno=2}, {error=No such file or directory}] >>> [2024-01-18 06:08:57.594051 +0000] W...

...boot with version > > 10.4, but doesn't with version 11. What could be wrong? > > > > I only see these errors on one of the "good" servers in glustershd.log: > > > > [2024-01-18 06:08:57.328480 +0000] W [MSGID: 114031] > > [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-workdata-client-0: > > remote operation failed. > > [{path=<gfid:cb39a1e4-2a4c-4727-861d-3ed9ef00681b>}, > > {gfid=cb39a1e4-2a4c-4727-861d-3ed9e > > f00681b}, {errno=2}, {error=No such file or directory}] > > [2024-01-18 06:08:57.594051 +0000] W [MSGID: 114031...

...week). Running a rebalance command should a lot of errors and failures in the status report of the rebalance. I also get quite a lot of this error in the logfile (settings is the small file I?m talking about) > > [2023-08-17 21:57:31.910494 +0000] W [MSGID: 114031] [client-rpc-fops_v2.c:2561:client4_0_lookup_cbk] 0-apptivegrid-client-0: remote operation failed. [{path=/62/03/8c/62038cd116e9a6857794aa14/settings}, {gfid=1d38410a-1c14-4346-a7e5-68856ed310e9}, {errno=2}, {error=No such file or directory}] > > and this > > [2023-08-17 21:57:31.902676 +0000] I [MSGID: 109018] [dht-common.c:1838:d...