[Gluster-users] Bricks going offline

John Burt
2020-03-12T04:03:48Z

I'm struggling to keep bricks online, where they keep crashing almost
immediately after forcing the volume online. Any suggestions for
troubleshooting?

Here is an excerpt from the logs from one of the bricks going offline.

[2020-03-12 03:46:03.589778] W [socket.c:774:__socket_rwv]
0-tcp.gvolume0-server: readv on 192.168.1.204:49005 failed (No data
available)
[2020-03-12 03:46:03.589863] I [MSGID: 115036]
[server.c:501:server_rpc_notify] 0-gvolume0-server: disconnecting
connection from
CTX_ID:a9d855a5-5db9-466f-9b59-7486e5e95671-GRAPH_ID:0-PID:1398-HOST:odroid4-PC_NAME:gvolume0-client-5-RECON_NO:-1
[2020-03-12 03:46:03.590182] I [MSGID: 101055]
[client_t.c:436:gf_client_unref] 0-gvolume0-server: Shutting down
connection
CTX_ID:a9d855a5-5db9-466f-9b59-7486e5e95671-GRAPH_ID:0-PID:1398-HOST:odroid4-PC_NAME:gvolume0-client-5-RECON_NO:-1
[2020-03-12 03:46:04.713141] W [socket.c:774:__socket_rwv]
0-tcp.gvolume0-server: readv on 192.168.1.219:49016 failed (No data
available)
[2020-03-12 03:46:04.713208] I [MSGID: 115036]
[server.c:501:server_rpc_notify] 0-gvolume0-server: disconnecting
connection from
CTX_ID:589c452c-81fb-42b4-9e1a-67bea2930457-GRAPH_ID:0-PID:1314-HOST:odroid19-PC_NAME:gvolume0-client-5-RECON_NO:-1
[2020-03-12 03:46:04.713490] I [MSGID: 101055]
[client_t.c:436:gf_client_unref] 0-gvolume0-server: Shutting down
connection
CTX_ID:589c452c-81fb-42b4-9e1a-67bea2930457-GRAPH_ID:0-PID:1314-HOST:odroid19-PC_NAME:gvolume0-client-5-RECON_NO:-1
[2020-03-12 03:46:05.624023] W [MSGID: 100032]
[glusterfsd.c:1596:cleanup_and_exit] 0-: received signum (15), shutting down
[2020-03-12 03:47:18.126938] I [MSGID: 100030] [glusterfsd.c:2867:main]
0-/usr/sbin/glusterfsd: Started running /usr/sbin/glusterfsd version 7.2
(args: /usr/sbin/glusterfsd -s odroid20 --volfile-id
gvolume0.odroid20.mnt-brick2-gvolume0 -p
/var/run/gluster/vols/gvolume0/odroid20-mnt-brick2-gvolume0.pid -S
/var/run/gluster/59a03312d88125a3.socket --brick-name /mnt/brick2/gvolume0
-l /var/log/glusterfs/bricks/mnt-brick2-gvolume0.log --xlator-option
*-posix.glusterd-uuid=07fa2e6a-43ea-4483-a4a9-f2321ae5ff86 --process-name
brick --brick-port 49154 --xlator-option
gvolume0-server.transport.rdma.listen-port=49155 --xlator-option
gvolume0-server.listen-port=49154 --volfile-server-transport=socket,rdma)
[2020-03-12 03:47:18.128162] I [glusterfsd.c:2594:daemonize] 0-glusterfs:
Pid of current running process is 13222
[2020-03-12 03:47:18.136097] I [socket.c:957:__socket_server_bind]
0-socket.glusterfsd: closing (AF_UNIX) reuse check socket 9
[2020-03-12 03:47:18.138848] I [MSGID: 101190]
[event-epoll.c:682:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 0
[2020-03-12 03:47:18.138959] I [MSGID: 101190]
[event-epoll.c:682:event_dispatch_epoll_worker] 0-epoll: Started thread
with index 1
[2020-03-12 03:47:18.149567] I
[rpcsvc.c:2690:rpcsvc_set_outstanding_rpc_limit] 0-rpc-service: Configured
rpc.outstanding-rpc-limit with value 64
[2020-03-12 03:47:18.151945] W [MSGID: 103071]
[rdma.c:4472:__gf_rdma_ctx_create] 0-rpc-transport/rdma: rdma_cm event
channel creation failed [No such device]
[2020-03-12 03:47:18.151986] W [MSGID: 103055] [rdma.c:4782:init]
0-rdma.gvolume0-server: Failed to initialize IB Device
[2020-03-12 03:47:18.152013] W [rpc-transport.c:366:rpc_transport_load]
0-rpc-transport: 'rdma' initialization failed
[2020-03-12 03:47:18.152303] W [rpcsvc.c:1981:rpcsvc_create_listener]
0-rpc-service: cannot create listener, initing the transport failed
[2020-03-12 03:47:18.152328] E [MSGID: 115045] [server.c:1199:server_init]
0-gvolume0-server: creation of 1 listeners failed, continuing with
succeeded transport
[2020-03-12 03:47:18.156858] I [socket.c:957:__socket_server_bind]
0-socket.gvolume0-changelog: closing (AF_UNIX) reuse check socket 13
[2020-03-12 03:47:18.157403] I [trash.c:2450:init] 0-gvolume0-trash: no
option specified for 'eliminate', using NULL
Final graph:
+------------------------------------------------------------------------------+
  1: volume gvolume0-posix
  2:     type storage/posix
  3:     option glusterd-uuid 07fa2e6a-43ea-4483-a4a9-f2321ae5ff86
  4:     option directory /mnt/brick2/gvolume0
  5:     option volume-id 5ffb0119-6f15-4b7c-9856-4747ba31b358
  6:     option fips-mode-rchecksum on
  7:     option shared-brick-count 1
  8:     option reserve-size 0
  9: end-volume
 10:
 11: volume gvolume0-arbiter
 12:     type features/arbiter
 13:     subvolumes gvolume0-posix
 14: end-volume
 15:
 16: volume gvolume0-trash
 17:     type features/trash
 18:     option trash-dir .trashcan
 19:     option brick-path /mnt/brick2/gvolume0
 20:     option trash-internal-op off
 21:     subvolumes gvolume0-arbiter
 22: end-volume
 23:
 24: volume gvolume0-changelog
 25:     type features/changelog
 26:     option changelog-brick /mnt/brick2/gvolume0
 27:     option changelog-dir /mnt/brick2/gvolume0/.glusterfs/changelogs
 28:     option changelog-barrier-timeout 120
 29:     subvolumes gvolume0-trash
 30: end-volume
 31:
 32: volume gvolume0-bitrot-stub
 33:     type features/bitrot-stub
 34:     option export /mnt/brick2/gvolume0
 35:     option bitrot disable
 36:     subvolumes gvolume0-changelog
 37: end-volume
 38:
 39: volume gvolume0-access-control
 40:     type features/access-control
 41:     subvolumes gvolume0-bitrot-stub
 42: end-volume
 43:
 44: volume gvolume0-locks
 45:     type features/locks
 46:     option enforce-mandatory-lock off
 47:     subvolumes gvolume0-access-control
 48: end-volume
 49:
 50: volume gvolume0-worm
 51:     type features/worm
 52:     option worm off
 53:     option worm-file-level off
 54:     option worm-files-deletable on
 55:     subvolumes gvolume0-locks
 56: end-volume
 57:
 58: volume gvolume0-read-only
 59:     type features/read-only
 60:     option read-only off
 61:     subvolumes gvolume0-worm
 62: end-volume
 63:
 64: volume gvolume0-leases
 65:     type features/leases
 66:     option leases off
 67:     subvolumes gvolume0-read-only
 68: end-volume
 69:
 70: volume gvolume0-upcall
 71:     type features/upcall
 72:     option cache-invalidation off
 73:     subvolumes gvolume0-leases
 74: end-volume
 75:
 76: volume gvolume0-io-threads
 77:     type performance/io-threads
 78:     subvolumes gvolume0-upcall
 79: end-volume
 80:
 81: volume gvolume0-selinux
 82:     type features/selinux
 83:     option selinux on
 84:     subvolumes gvolume0-io-threads
 85: end-volume
 86:
 87: volume gvolume0-marker
 88:     type features/marker
 89:     option volume-uuid 5ffb0119-6f15-4b7c-9856-4747ba31b358
 90:     option timestamp-file /var/lib/glusterd/vols/gvolume0/marker.tstamp
 91:     option quota-version 0
 92:     option xtime off
 93:     option gsync-force-xtime off
 94:     option quota off
 95:     option inode-quota off
 96:     subvolumes gvolume0-selinux
 97: end-volume
 98:
 99: volume gvolume0-barrier
100:     type features/barrier
101:     option barrier disable
102:     option barrier-timeout 120
103:     subvolumes gvolume0-marker
104: end-volume
105:
106: volume gvolume0-index
107:     type features/index
108:     option index-base /mnt/brick2/gvolume0/.glusterfs/indices
109:     option xattrop-dirty-watchlist trusted.afr.dirty
110:     option xattrop-pending-watchlist trusted.afr.gvolume0-
111:     subvolumes gvolume0-barrier
112: end-volume
113:
114: volume gvolume0-quota
115:     type features/quota
116:     option volume-uuid gvolume0
117:     option server-quota off
118:     option deem-statfs off
119:     subvolumes gvolume0-index
120: end-volume
121:
122: volume /mnt/brick2/gvolume0
123:     type debug/io-stats
124:     option auth.addr./mnt/brick2/gvolume0.allow *
125:     option auth-path /mnt/brick2/gvolume0
126:     option auth.login.6b0eba26-2c74-4542-a792-7c07595e1c53.password
9938f115-0d9e-4d7c-bc2b-8627dac85e42
127:     option auth.login./mnt/brick2/gvolume0.allow
6b0eba26-2c74-4542-a792-7c07595e1c53
128:     option unique-id /mnt/brick2/gvolume0
129:     option log-level INFO
130:     option threads 16
131:     option latency-measurement off
132:     option count-fop-hits off
133:     option global-threading off
134:     subvolumes gvolume0-quota
135: end-volume
136:
137: volume gvolume0-server
138:     type protocol/server
139:     option transport.socket.listen-port 49154
140:     option rpc-auth.auth-glusterfs on
141:     option rpc-auth.auth-unix on
142:     option rpc-auth.auth-null on
143:     option rpc-auth-allow-insecure on
144:     option transport.rdma.listen-port 49155
145:     option transport-type rdma
146:     option auth.login./mnt/brick2/gvolume0.allow
6b0eba26-2c74-4542-a792-7c07595e1c53
147:     option auth.login.6b0eba26-2c74-4542-a792-7c07595e1c53.password
9938f115-0d9e-4d7c-bc2b-8627dac85e42
148:     option auth-path /mnt/brick2/gvolume0
149:     option auth.addr./mnt/brick2/gvolume0.allow *
150:     option transport.socket.keepalive 1
151:     option transport.socket.ssl-enabled off
152:     option transport.socket.keepalive-time 20
153:     option transport.socket.keepalive-interval 2
154:     option transport.socket.keepalive-count 9
155:     option transport.listen-backlog 1024
156:     subvolumes /mnt/brick2/gvolume0
157: end-volume
158:
+------------------------------------------------------------------------------+
[2020-03-12 03:47:19.622547] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.219"
[2020-03-12 03:47:19.622761] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.622837] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:9ba4a0b3-15ef-46da-a2c8-94e6425fb2fe-GRAPH_ID:0-PID:1543-HOST:odroid19-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.627203] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.203"
[2020-03-12 03:47:19.627482] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.627879] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:94ee8e47-2d8b-4661-94ff-ec339c928883-GRAPH_ID:0-PID:2859-HOST:odroid3-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.638411] I [rpcsvc.c:866:rpcsvc_handle_rpc_call]
0-rpc-service: spawned a request handler thread for queue 0
[2020-03-12 03:47:19.639320] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.217"
[2020-03-12 03:47:19.639372] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.639413] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:5ef50bb9-ca91-4834-bd0f-f1d1a4ebfe82-GRAPH_ID:0-PID:2860-HOST:odroid17-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.640263] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.211"
[2020-03-12 03:47:19.640314] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.640352] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:305eee4b-944a-4929-b9d1-0d983cbe5e5c-GRAPH_ID:0-PID:2953-HOST:odroid11-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.651496] I [rpcsvc.c:866:rpcsvc_handle_rpc_call]
0-rpc-service: spawned a request handler thread for queue 1
[2020-03-12 03:47:19.652290] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.205"
[2020-03-12 03:47:19.652348] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.652389] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:41955f0b-756a-4b23-abb5-5915065c7988-GRAPH_ID:0-PID:1841-HOST:odroid5-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.653788] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.215"
[2020-03-12 03:47:19.653955] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.654030] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:955893d7-2363-4b8c-9859-cd025f9c5a9d-GRAPH_ID:0-PID:2877-HOST:odroid15-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.655833] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.202"
[2020-03-12 03:47:19.655917] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.655975] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:5d06cc3b-20b3-4aa1-a9e4-9f2a8ad2ec60-GRAPH_ID:0-PID:2873-HOST:odroid2-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.656185] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.212"
[2020-03-12 03:47:19.656253] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.656308] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:9733b48d-6568-4ac5-a7c4-7ee6dcbe5eb9-GRAPH_ID:0-PID:1736-HOST:odroid12-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.657924] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.209"
[2020-03-12 03:47:19.658030] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.658090] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:d1b92e61-b6b9-4256-aec9-9e74928cfa6d-GRAPH_ID:0-PID:2882-HOST:odroid9-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.659290] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.214"
[2020-03-12 03:47:19.659499] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.659683] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:89dc1052-5b31-4e14-a9b8-4289887a5539-GRAPH_ID:0-PID:2748-HOST:odroid14-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.660733] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.206"
[2020-03-12 03:47:19.660798] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.660837] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:b60d4094-86c8-48e9-b227-9d0ff2145e08-GRAPH_ID:0-PID:2858-HOST:odroid6-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.669176] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.213"
[2020-03-12 03:47:19.669282] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.669499] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:1b1e0280-9b53-4244-ba2b-c37f934126b9-GRAPH_ID:0-PID:1687-HOST:odroid13-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.674585] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.218"
[2020-03-12 03:47:19.674671] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.674758] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:5fc6cfcf-6e58-48eb-894f-c16d4377af29-GRAPH_ID:0-PID:2888-HOST:odroid18-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.678824] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.201"
[2020-03-12 03:47:19.678881] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.678918] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:ccc64172-a27b-45d4-9e61-a982e0c6129a-GRAPH_ID:0-PID:3038-HOST:odroid1-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.679921] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.207"
[2020-03-12 03:47:19.680014] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.680088] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:b2aece5f-c214-4afe-9f35-debc378b148b-GRAPH_ID:0-PID:2880-HOST:odroid7-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.684127] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.208"
[2020-03-12 03:47:19.684190] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.684232] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:d9be2285-e369-41cb-9333-8d8d8d631fe6-GRAPH_ID:0-PID:2870-HOST:odroid8-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.696075] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.216"
[2020-03-12 03:47:19.696248] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.696321] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:885af054-aa6d-405a-b8e9-31a7a807fed7-GRAPH_ID:0-PID:2870-HOST:odroid16-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.698740] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.210"
[2020-03-12 03:47:19.698809] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.698851] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:61a16350-2549-45cb-ac0a-717ea3c12415-GRAPH_ID:0-PID:2297-HOST:odroid10-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:19.770633] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.220"
[2020-03-12 03:47:19.770679] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:19.770709] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:6b9faf45-b335-4054-a990-ca279994b816-GRAPH_ID:0-PID:13383-HOST:odroid20-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:20.237138] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.2"
[2020-03-12 03:47:20.237195] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:dd5899a9-a5eb-4efc-a283-5dbc56e11f60-GRAPH_ID:0-PID:3417-HOST:localhost.localdomain-PC_NAME:gvolume0-client-5-RECON_NO:-4
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:20.239653] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.2"
[2020-03-12 03:47:20.239704] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:dd5899a9-a5eb-4efc-a283-5dbc56e11f60-GRAPH_ID:2-PID:3417-HOST:localhost.localdomain-PC_NAME:gvolume0-client-5-RECON_NO:-2
(version: 7.2) with subvol /mnt/brick2/gvolume0
[2020-03-12 03:47:21.135088] I [addr.c:54:compare_addr_and_update]
0-/mnt/brick2/gvolume0: allowed = "*", received addr =
"192.168.1.204"
[2020-03-12 03:47:21.135135] I [login.c:110:gf_auth] 0-auth/login: allowed
user names: 6b0eba26-2c74-4542-a792-7c07595e1c53
[2020-03-12 03:47:21.135162] I [MSGID: 115029]
[server-handshake.c:549:server_setvolume] 0-gvolume0-server: accepted
client from
CTX_ID:0bb3b680-0594-45e2-894c-92761aa51ab7-GRAPH_ID:0-PID:1955-HOST:odroid4-PC_NAME:gvolume0-client-5-RECON_NO:-0
(version: 7.2) with subvol /mnt/brick2/gvolume0
pending frames:
frame : type(1) op(SETXATTR)
patchset: git://git.gluster.org/glusterfs.git
signal received: 6
time of crash:
2020-03-12 03:48:01
configuration details:
argp 1
backtrace 1
dlfcn 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 7.2


Thanks,
--

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20200311/959cd23d/attachment.html>