Kingsley Tart
2022-May-26 12:55 UTC
[Gluster-users] transport endpoint not connected on just 2 files
Hi, I've got a strange issue where on all clients I've tested on (tested on 4) I have "transport endpoint is not connected" on two files in a directory, whereas other files can be read fine. Any ideas? On one of the servers (all same version): # gluster --version glusterfs 9.1 On one of the clients (same thing with all of them) - problem with files "gw3" and "gw11": [root at gw6 btl]# cd /mnt/runqueues/runners/ [root at gw6 runners]# ls -la ls: cannot access gw11: Transport endpoint is not connected ls: cannot access gw3: Transport endpoint is not connected total 8 drwxr-xr-x 2 root root 4096 May 26 09:48 . drwxr-xr-x 13 root root 4096 Apr 12 2021 .. -rw-r--r-- 1 root root 0 May 26 09:49 gw1 -rw-r--r-- 1 root root 0 May 26 09:49 gw10 -????????? ? ? ? ? ? gw11 -rw-r--r-- 1 root root 0 May 26 09:49 gw2 -????????? ? ? ? ? ? gw3 -rw-r--r-- 1 root root 0 May 26 09:49 gw4 -rw-r--r-- 1 root root 0 May 26 09:49 gw6 -rw-r--r-- 1 root root 0 May 26 09:49 gw7 [root at gw6 runners]# cat * cat: gw11: Transport endpoint is not connected cat: gw3: Transport endpoint is not connected [root at gw6 runners]# Querying on a server shows those two problematic files: # gluster volume heal gw-runqueues info Brick gluster9a:/data/brick/gw-runqueues /runners /runners/gw11 /runners/gw3 Status: Connected Number of entries: 3 Brick gluster9b:/data/brick/gw-runqueues /runners /runners/gw11 /runners/gw3 Status: Connected Number of entries: 3 Brick gluster9c:/data/brick/gw-runqueues Status: Connected Number of entries: 0 However several hours later there's no obvious change. The servers have hardly any load and the volume is tiny. From a client: # find /mnt/runqueues | wc -l 35 glfsheal-gw-runqueues.log from server gluster9a: https://pastebin.com/7mPszBBM glfsheal-gw-runqueues.log from server gluster9b: https://pastebin.com/rxXs5Tcv Any pointers would be much appreciated! Cheers, Kingsley.
Strahil Nikolov
2022-May-27 03:59 UTC
[Gluster-users] transport endpoint not connected on just 2 files
Check the file attributes on all bricks: getfattr -d -e hex -m. /data/brick/gw-runqueues/<path to file> Best Regards,Strahil Nikolov On Thu, May 26, 2022 at 16:05, Kingsley Tart<gluster at gluster.dogwind.com> wrote: Hi, I've got a strange issue where on all clients I've tested on (tested on 4) I have "transport endpoint is not connected" on two files in a directory, whereas other files can be read fine. Any ideas? On one of the servers (all same version): # gluster --version glusterfs 9.1 On one of the clients (same thing with all of them) - problem with files "gw3" and "gw11": [root at gw6 btl]# cd /mnt/runqueues/runners/ [root at gw6 runners]# ls -la ls: cannot access gw11: Transport endpoint is not connected ls: cannot access gw3: Transport endpoint is not connected total 8 drwxr-xr-x? 2 root root 4096 May 26 09:48 . drwxr-xr-x 13 root root 4096 Apr 12? 2021 .. -rw-r--r--? 1 root root? ? 0 May 26 09:49 gw1 -rw-r--r--? 1 root root? ? 0 May 26 09:49 gw10 -?????????? ? ?? ? ?? ? ? ?? ? ? ? ? ? ? gw11 -rw-r--r--? 1 root root? ? 0 May 26 09:49 gw2 -?????????? ? ?? ? ?? ? ? ?? ? ? ? ? ? ? gw3 -rw-r--r--? 1 root root? ? 0 May 26 09:49 gw4 -rw-r--r--? 1 root root? ? 0 May 26 09:49 gw6 -rw-r--r--? 1 root root? ? 0 May 26 09:49 gw7 [root at gw6 runners]# cat * cat: gw11: Transport endpoint is not connected cat: gw3: Transport endpoint is not connected [root at gw6 runners]# Querying on a server shows those two problematic files: # gluster volume heal gw-runqueues info Brick gluster9a:/data/brick/gw-runqueues /runners /runners/gw11 /runners/gw3 Status: Connected Number of entries: 3 Brick gluster9b:/data/brick/gw-runqueues /runners /runners/gw11 /runners/gw3 Status: Connected Number of entries: 3 Brick gluster9c:/data/brick/gw-runqueues Status: Connected Number of entries: 0 However several hours later there's no obvious change. The servers have hardly any load and the volume is tiny. From a client: # find /mnt/runqueues | wc -l 35 glfsheal-gw-runqueues.log from server gluster9a: https://pastebin.com/7mPszBBM glfsheal-gw-runqueues.log from server gluster9b: https://pastebin.com/rxXs5Tcv Any pointers would be much appreciated! Cheers, Kingsley. ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users at gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20220527/adcb00d9/attachment.html>