[Gluster-users] Volume mounts read-only

Pavel Szalbot
Mon, 24 Jul 2017 05:44:48 -0000

Hi, are you sure there are not two hosts with the same IP address?
Happened to me once and logs looked pretty similar (frequent
disconnects, all other clients seemed fine, gluster cluster reported
no issues).
-ps


On Fri, Jul 21, 2017 at 9:13 PM, Jamie Lawrence
<jlawrence at squaretrade.com> wrote:
> Hello Glusterites,
>
> I have a volume that will not mount read/write. V3.10.3 on Centos 7, this
is a replica-3 volume, mounted with the fuse client. This is in support of an
Ovirt installation, but I've isolated the problem to Gluster. `gluster peer
status` looks normal, as does a `gluster v status`. Server.allow-insecure is set
to on.
>
> Notable from the client volume logs are the 'connection refused'
messages[1], that seem to match the gluster server logs[2], but I'm not
exactly sure what that's telling me, or what I can do about it.  Two other
hosts currently have the volume mounted and can write just fine; Because of
operational reasons, I have not tested to see what would happen if I un/remount
it on the currently functioning hosts. I trialed it disabling iptables, no
change.
>
> I'm not sure what else to experiment with, and couldn't find
anything relevant in the googles.  Thus this email.
>
> Has anyone seen this before? TIA,
>
> -j
>
>
> [1] rhev-data-center-mnt-glusterSD-sc5-gluster-10g-1:ovirt__engine.log
> [2017-07-21 18:19:07.210232] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:19:08.280799] I [rpc-clnt.c:2001:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:19:08.286512] E [socket.c:2309:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused)
> [2017-07-21 18:19:09.260107] W [fuse-bridge.c:1067:fuse_setattr_cbk]
0-glusterfs-fuse: 1721: SETATTR() /__DIRECT_IO_TEST__ => -1 (Read-only file
system)
> [2017-07-21 18:19:10.286556] I [rpc-clnt.c:2001:rpc_clnt_reconfig]
0-ovirt_engine-client-1: changing port to 49155 (from 0)
> [2017-07-21 18:19:10.292217] E [socket.c:2309:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused)
> [2017-07-21 18:19:11.206829] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:19:11.212077] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-1: changing port to 49155 (from 0)
> [2017-07-21 18:19:11.216493] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:19:11.221265] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:19:12.292228] I [rpc-clnt.c:2001:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:19:12.297970] E [socket.c:2309:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused)
> [2017-07-21 18:19:14.297828] I [rpc-clnt.c:2001:rpc_clnt_reconfig]
0-ovirt_engine-client-1: changing port to 49155 (from 0)
> [2017-07-21 18:19:14.303567] E [socket.c:2309:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused)
> [2017-07-21 18:19:15.217792] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:19:15.223013] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-1: changing port to 49155 (from 0)
> [2017-07-21 18:19:15.227406] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:19:15.232193] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused); disconnecting socket
>
>
> [2] glustershd.log:
> [2017-07-21 18:31:36.890242] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:31:36.895707] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:31:39.896196] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-1: changing port to 49155 (from 0)
> [2017-07-21 18:31:39.901850] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:31:40.901569] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:31:40.907006] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:31:43.907508] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-1: changing port to 49155 (from 0)
> [2017-07-21 18:31:43.913141] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-1: connection to 172.16.0.152:49155 failed (Connection
refused); disconnecting socket
> [2017-07-21 18:31:44.912807] I [rpc-clnt.c:2000:rpc_clnt_reconfig]
0-ovirt_engine-client-0: changing port to 49220 (from 0)
> [2017-07-21 18:31:44.918243] E [socket.c:2316:socket_connect_finish]
0-ovirt_engine-client-0: connection to 172.16.0.151:49220 failed (Connection
refused); disconnecting socket
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users