Hu Bert
2021-Mar-03 09:14 UTC
[Gluster-users] Upgrade 7.8 -> 8.4: transport endpoint not connected
Hi there,
i just did an upgrade on my test servers from version 7.8 to version
8.4 (debian buster, replicate 3 setup, 2 volumes), and noticed that
one of two mounts wasn't working afterwards.
procedure (same i used for 6.x -> 7.x):
- stop all gluster processes on the server
- apt update && apt upgrade
- check volume, services, mounts etc. afterwards
The volumes themselves are working, but one of the transport endpoints
is not connected.
Before the upgrade:
ls -lah /shared/
drwxr-xr-x 4 root root 75 Apr 1 2020 private
drwxr-xr-x 5 root root 46 Apr 16 2019 public
after the upgrade:
ls -lah /shared/
ls: cannot access '/shared/private': Transport endpoint is not connected
d????????? ? ? ? ? ? private
drwxr-xr-x 2 root root 4.0K Feb 11 2019 public
directory permissions and ownership screwed up? And even the date of
the 2nd dir (public) has changed.
Job for glusterfssharedstorage.service failed because the control
process exited with error code.
See "systemctl status glusterfssharedstorage.service" and
"journalctl
-xe" for details.
systemctl status glusterfssharedstorage.service
? glusterfssharedstorage.service - Mount glusterfs sharedstorage
Loaded: loaded (/lib/systemd/system/glusterfssharedstorage.service;
enabled; vendor preset: enabled)
Active: activating (start) since Wed 2021-03-03 09:55:16 CET; 1s ago
Cntrl PID: 2148 (mount-shared-st)
Tasks: 2 (limit: 4915)
Memory: 4.1M
CGroup: /system.slice/glusterfssharedstorage.service
??2148 /bin/bash /usr/libexec/glusterfs/mount-shared-storage.sh
??2213 sleep 10
Mar 03 09:55:16 dirac systemd[1]: Starting Mount glusterfs sharedstorage...
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: ERROR: Mount
point does not exist
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: Please specify a
mount point
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: Usage:
Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: man 8 /sbin/mount.glusterfs
Mar 03 09:55:26 dirac mount-shared-storage.sh[2148]: /shared/private
failed to mount
Mar 03 09:55:36 dirac mount-shared-storage.sh[2148]: /shared/public
has been mounted
Mar 03 09:55:36 dirac systemd[1]: glusterfssharedstorage.service:
Control process exited, code=exited, status=1/FAILURE
Well... does anyone have an idea what might have gone wrong?
While writing/thinking i was able to fix it:
- umount /shared/private -> directory looks normal again (no ???)
- systemctl start glusterfssharedstorage.service
- check if mount is done -> yes
ah ok, read it here...
https://docs.gluster.org/en/latest/Upgrade-Guide/generic-upgrade-procedure/
"Upgrade procedure for clients
Unmount all glusterfs mount points on the client"
My servers are mounting the volumes as clients as well. During the
last version upgrade i never had done the unmount before - is this v8
related? Just curious...
Best regards,
Hubert
Strahil Nikolov
2021-Mar-13 20:25 UTC
[Gluster-users] Upgrade 7.8 -> 8.4: transport endpoint not connected
Usually this is not supposed to happen, but I guess the FUSE client had trouble to reconnect to all bricks. Best Regards, Strahil Nikolov ? ?????, 3 ???? 2021 ?., 11:14:38 ?. ???????+2, Hu Bert <revirii at googlemail.com> ??????: Hi there, i just did an upgrade on my test servers from version 7.8 to version 8.4 (debian buster, replicate 3 setup, 2 volumes), and noticed that one of two mounts wasn't working afterwards. procedure (same i used for 6.x -> 7.x): - stop all gluster processes on the server - apt update && apt upgrade - check volume, services, mounts etc. afterwards The volumes themselves are working, but one of the transport endpoints is not connected. Before the upgrade: ls -lah /shared/ drwxr-xr-x? 4 root root? 75 Apr? 1? 2020 private drwxr-xr-x? 5 root root? 46 Apr 16? 2019 public after the upgrade: ls -lah /shared/ ls: cannot access '/shared/private': Transport endpoint is not connected d?????????? ? ?? ? ?? ? ? ?? ? ? ? ? ? ? private drwxr-xr-x? 2 root root 4.0K Feb 11? 2019 public directory permissions and ownership screwed up? And even the date of the 2nd dir (public) has changed. Job for glusterfssharedstorage.service failed because the control process exited with error code. See "systemctl status glusterfssharedstorage.service" and "journalctl -xe" for details. systemctl status glusterfssharedstorage.service ? glusterfssharedstorage.service - Mount glusterfs sharedstorage ? Loaded: loaded (/lib/systemd/system/glusterfssharedstorage.service; enabled; vendor preset: enabled) ? Active: activating (start) since Wed 2021-03-03 09:55:16 CET; 1s ago Cntrl PID: 2148 (mount-shared-st) ? Tasks: 2 (limit: 4915) ? Memory: 4.1M ? CGroup: /system.slice/glusterfssharedstorage.service ? ? ? ? ? ??2148 /bin/bash /usr/libexec/glusterfs/mount-shared-storage.sh ? ? ? ? ? ??2213 sleep 10 Mar 03 09:55:16 dirac systemd[1]: Starting Mount glusterfs sharedstorage... Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: ERROR: Mount point does not exist Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: Please specify a mount point Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: Usage: Mar 03 09:55:16 dirac mount-shared-storage.sh[2148]: man 8 /sbin/mount.glusterfs Mar 03 09:55:26 dirac mount-shared-storage.sh[2148]: /shared/private failed to mount Mar 03 09:55:36 dirac mount-shared-storage.sh[2148]: /shared/public has been mounted Mar 03 09:55:36 dirac systemd[1]: glusterfssharedstorage.service: Control process exited, code=exited, status=1/FAILURE Well... does anyone have an idea what might have gone wrong? While writing/thinking i was able to fix it: - umount /shared/private -> directory looks normal again (no ???) - systemctl start glusterfssharedstorage.service - check if mount is done -> yes ah ok, read it here... https://docs.gluster.org/en/latest/Upgrade-Guide/generic-upgrade-procedure/ "Upgrade procedure for clients Unmount all glusterfs mount points on the client" My servers are mounting the volumes as clients as well. During the last version upgrade i never had done the unmount before - is this v8 related? Just curious... Best regards, Hubert ________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users at gluster.org https://lists.gluster.org/mailman/listinfo/gluster-users