John Burt
2020-Dec-06 17:50 UTC
[Gluster-users] glusterd fails to start after upgrade to 8.3
I'm having trouble with glusterfs after updating to 8.3.
6-node + thin arbiter setup. Volume was stopped, and all nodes were
updated. After reboot, volume was unable to be restarted as glusterd was
not running.
Any suggestions on how to get glusterd back up and running?
[2020-12-06 17:37:13.725899] I [MSGID: 100030] [glusterfsd.c:2687:main]
0-/usr/sbin/glusterd: Started running version [{arg=/usr/sbin/glusterd},
{version=8.3}, {cmdlinestr=/usr/sbin/glusterd -p /var/run/glusterd.pid
--log-level INFO}]
[2020-12-06 17:37:13.728018] I [glusterfsd.c:2423:daemonize] 0-glusterfs:
Pid of current running process is 2568
[2020-12-06 17:37:13.764725] I [MSGID: 106478] [glusterd.c:1444:init]
0-management: Maximum allowed open file descriptors set to 65536
[2020-12-06 17:37:13.764843] I [MSGID: 106479] [glusterd.c:1521:init]
0-management: Using /var/lib/glusterd as working directory
[2020-12-06 17:37:13.764876] I [MSGID: 106479] [glusterd.c:1525:init]
0-management: Using /var/run/gluster as pid file working directory
[2020-12-06 17:37:13.771710] I [socket.c:984:__socket_server_bind]
0-socket.management: process started listening on port (24007)
[2020-12-06 17:37:13.775031] I [socket.c:928:__socket_server_bind]
0-socket.management: closing (AF_UNIX) reuse check socket 12
[2020-12-06 17:37:13.776133] I [MSGID: 106059] [glusterd.c:1912:init]
0-management: max-port override: 60999
[2020-12-06 17:37:17.861392] I [MSGID: 106513]
[glusterd-store.c:2130:glusterd_restore_op_version] 0-glusterd: retrieved
op-version: 70200
[2020-12-06 17:37:17.862195] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: tier-enabled
[2020-12-06 17:37:17.862376] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: brick-0
[2020-12-06 17:37:17.862413] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: brick-1
[2020-12-06 17:37:17.862446] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: brick-2
[2020-12-06 17:37:17.862478] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: brick-3
[2020-12-06 17:37:17.862509] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: brick-4
[2020-12-06 17:37:17.862540] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: brick-5
[2020-12-06 17:37:17.862572] W [MSGID: 106204]
[glusterd-store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown
key: ta-brick-0
[2020-12-06 17:37:18.145259] I [MSGID: 106544]
[glusterd.c:153:glusterd_uuid_init] 0-management: retrieved UUID:
1f7af201-eff5-4922-a1f6-a269e0d5fda8
[2020-12-06 17:37:18.146645] E [MSGID: 101032]
[store.c:460:gf_store_handle_retrieve] 0-: Path corresponding to
/var/lib/glusterd/vols/gvolume0/bricks/(null). [No such file or directory]
[2020-12-06 17:37:18.146773] E [MSGID: 106201]
[glusterd-store.c:3507:glusterd_store_retrieve_volumes] 0-management:
Unable to restore volume: gvolume0
[2020-12-06 17:37:18.146846] E [MSGID: 101019] [xlator.c:622:xlator_init]
0-management: Initialization of volume failed. review your volfile again.
[{name=management}]
[2020-12-06 17:37:18.146876] E [MSGID: 101066]
[graph.c:424:glusterfs_graph_init] 0-management: initializing translator
failed
[2020-12-06 17:37:18.146903] E [MSGID: 101176]
[graph.c:777:glusterfs_graph_activate] 0-graph: init failed
[2020-12-06 17:37:18.147140] W [MSGID: 100032]
[glusterfsd.c:1438:cleanup_and_exit] 0-: received signum (-1), shutting down
[2020-12-06 17:37:18.147264] W [mgmt-pmap.c:132:rpc_clnt_mgmt_pmap_signout]
0-glusterfs: failed to create XDR payload
Thanks,
--
John
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20201206/c92b2e1d/attachment.html>
Strahil Nikolov
2020-Dec-07 21:07 UTC
[Gluster-users] glusterd fails to start after upgrade to 8.3
> [2020-12-06 17:37:17.862195] W [MSGID: 106204] [glusterd- > store.c:3149:glusterd_store_update_volinfo] 0-management: Unknown > key: tier-enabledThis sounds interesting. According to https://docs.gluster.org/en/latest/Upgrade-Guide/upgrade-to-8/ "- Tiering support (tier xlator and changetimerecorder)" is deprecated.> [2020-12-06 17:37:18.146645] E [MSGID: 101032] > [store.c:460:gf_store_handle_retrieve] 0-: Path corresponding to > /var/lib/glusterd/vols/gvolume0/bricks/(null). [No such file or > directory]Is this path existing ?> [2020-12-06 17:37:18.146773] E [MSGID: 106201] [glusterd- > store.c:3507:glusterd_store_retrieve_volumes] 0-management: Unable to > restore volume: gvolume0 > [2020-12-06 17:37:18.146846] E [MSGID: 101019] > [xlator.c:622:xlator_init] 0-management: Initialization of volume > failed. review your volfile again. [{name=management}]What is your volfile's contents ? Most probably the fastest recover would be to roll back the upgrade (remove 8.3 and install 7.X that you have used prior the upgrade), yet rollback is officially not supported. Best Regards,Strahil Nikolov -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20201207/4576c815/attachment.html>