Hi Atul,
In Short: it is due to client side quorum behavior
Detailed info:
I see that there are 3 nodes in the cluster ie master1, master2, compute01
However the volume is being hosted only on master1 and master2.
Also, see that you have enabled server side quorum, and client side quorum from
vol info
cluster.quorum-type: auto =================>client side quorum options
cluster.server-quorum-type: server =================>server side quorum
options
cluster.server-quorum-ratio: 51% =================>server side quorum
options
Given that you are using compute01 (which is more of a dummy node in this case),
hence even though one node is down, still server side quorum is maintained
Client side quorum means, that the nodes are reachable by the client(IO path).
If set to "auto", this option allows writes to the file only if number
of bricks that are up >= ceil (of the total number of bricks that constitute
that replica/2). If the number of replicas is even, then there is a further
check: If the number of up bricks is exactly equal to n/2, then the first brick
must be one of the bricks that is up. If it is more than n/2 then it is not
necessary that the first brick is one of the up bricks.
In x2 case, the first brick of a replica pair must be always up for data to be
written from client.
Hence when you bring down node1 you get readonly, but when you bring down node2
you can still write to the volume.
----- Original Message -----
From: "Atul Yadav" <atulyadavtech at gmail.com>
To: "Atin Mukherjee" <amukherj at redhat.com>,
"gluster-users" <gluster-users at gluster.org>
Sent: Monday, 14 November, 2016 8:04:24 PM
Subject: [Gluster-users] Glusterfs readonly Issue
Dear Team,
In the event of the failure of master1, master 2 glusterfs home directory will
become read only fs.
If we manually shutdown the master 2, then there is no impact on the file system
and all io operation will complete with out any problem.
can you please provide some guidance to isolate the problem.
# gluster peer status
Number of Peers: 2
Hostname: master1-ib.dbt.au
Uuid: a5608d66-a3c6-450e-a239-108668083ff2
State: Peer in Cluster (Connected)
Hostname: compute01-ib.dbt.au
Uuid: d2c47fc2-f673-4790-b368-d214a58c59f4
State: Peer in Cluster (Connected)
# gluster vol info home
Volume Name: home
Type: Replicate
Volume ID: 2403ddf9-c2e0-4930-bc94-734772ef099f
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp,rdma
Bricks:
Brick1: master1-ib.dbt.au:/glusterfs/home/brick1
Brick2: master2-ib.dbt.au:/glusterfs/home/brick2
Options Reconfigured:
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
network.remote-dio: enable
cluster.quorum-type: auto
nfs.disable: on
performance.readdir-ahead: on
cluster.server-quorum-type: server
config.transport: tcp,rdma
network.ping-timeout: 10
cluster.server-quorum-ratio: 51%
cluster.enable-shared-storage: disable
# gluster vol heal home info
Brick master1-ib.dbt.au:/glusterfs/home/brick1
Status: Connected
Number of entries: 0
Brick master2-ib.dbt.au:/glusterfs/home/brick2
Status: Connected
Number of entries: 0
# gluster vol heal home info heal-failed
Gathering list of heal failed entries on volume home has been unsuccessful on
bricks that are down. Please check if all brick processes are running[root at
master2
Thank You
Atul Yadav
_______________________________________________
Gluster-users mailing list
Gluster-users at gluster.org
http://www.gluster.org/mailman/listinfo/gluster-users