Joshua J. Kugler
2016-Mar-23 01:33 UTC
[Gluster-users] Peer probe succeeded, but "not in 'Peer in Cluster' state"
On Tuesday, March 22, 2016 18:27:46 Atin Mukherjee wrote:> This is the problem, peer handshaking hasn't finished yet. To get to > know the reason I'd need to get the glusterd log file from 192.168.122.10.Here's the log from the other machine (.10).> As a workaround can you do the following? > > 1. From node 1 open /var/lib/glusterd/peers/<uuid file>, modify state=3 > 2. Repeat step 1 for node 2 as well if state is different > 3. restart both the glusterd instances.Hmm, just realized this: This is "Box A" [root at vmserver-a peers]# ls -l total 4 -rw------- 1 root root 74 Mar 22 17:15 fe722085-ac0f-4449-a43f-2dc9dd1fd8fb [root at vmserver-a peers]# cat fe722085-ac0f-4449-a43f-2dc9dd1fd8fb uuid=fe722085-ac0f-4449-a43f-2dc9dd1fd8fb state=3 hostname1=192.168.122.1 This is "Box B" (from where I'm running the gluster create command: [root at vmserver-b peers]# ls -l total 4 -rw------- 1 root root 75 Mar 22 17:15 d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 [root at vmserver-b peers]# cat d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 uuid=d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 state=3 hostname1=192.168.122.10 Why is the gateway in the peer list? To my knowledge, that's not getting added anywhere. This is weird. Also odd: [root at vmserver-a peers]# gluster peer status Number of Peers: 1 Hostname: 192.168.122.1 Uuid: fe722085-ac0f-4449-a43f-2dc9dd1fd8fb State: Peer in Cluster (Disconnected) [root at vmserver-b peers]# gluster peer status Number of Peers: 1 Hostname: 192.168.122.10 Uuid: d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 State: Peer in Cluster (Connected) So, somehow, on vmserver-a is getting 192.168.122.1 in its peer list instead of 192.168.122.11 Very strange. j> ~Atin > > >> Also send the glusterd log of the node where the commands have failed. > > > > The two logs are attached. The peer status says connected. The log file > > says "FAILED : Host 192.168.122.10 is not in 'Peer in Cluster' state" > > I'm confused. :) > > > > Thanks for your help on this! > > > > j > > > > > > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users at gluster.org > > http://www.gluster.org/mailman/listinfo/gluster-users-- Joshua J. Kugler - Fairbanks, Alaska Azariah Enterprises - Programming and Website Design joshua at azariah.com - Jabber: pedahzur at gmail.com PGP Key: http://pgp.mit.edu/ ID 0x73B13B6A -------------- next part -------------- A non-text attachment was scrubbed... Name: etc-glusterfs-glusterd.vol.log Type: text/x-log Size: 40319 bytes Desc: not available URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160322/fd6400c5/attachment.bin>
Joshua J. Kugler
2016-Mar-23 02:10 UTC
[Gluster-users] Peer probe succeeded, but "not in 'Peer in Cluster' state"
Hmm...I'm wondering if my networking is messed up some how. But why would that cause host b to see host a as the gateway and not as the proper IP? j On Tuesday, March 22, 2016 17:33:03 Joshua J. Kugler wrote:> On Tuesday, March 22, 2016 18:27:46 Atin Mukherjee wrote: > > This is the problem, peer handshaking hasn't finished yet. To get to > > know the reason I'd need to get the glusterd log file from > > 192.168.122.10. > > Here's the log from the other machine (.10). > > > As a workaround can you do the following? > > > > 1. From node 1 open /var/lib/glusterd/peers/<uuid file>, modify state=3 > > 2. Repeat step 1 for node 2 as well if state is different > > 3. restart both the glusterd instances. > > Hmm, just realized this: > > This is "Box A" > [root at vmserver-a peers]# ls -l > total 4 > -rw------- 1 root root 74 Mar 22 17:15 fe722085-ac0f-4449-a43f-2dc9dd1fd8fb > [root at vmserver-a peers]# cat fe722085-ac0f-4449-a43f-2dc9dd1fd8fb > uuid=fe722085-ac0f-4449-a43f-2dc9dd1fd8fb > state=3 > hostname1=192.168.122.1 > > This is "Box B" (from where I'm running the gluster create command: > [root at vmserver-b peers]# ls -l > total 4 > -rw------- 1 root root 75 Mar 22 17:15 d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 > [root at vmserver-b peers]# cat d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 > uuid=d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 > state=3 > hostname1=192.168.122.10 > > Why is the gateway in the peer list? To my knowledge, that's not getting > added anywhere. This is weird. > > Also odd: > [root at vmserver-a peers]# gluster peer status > Number of Peers: 1 > > Hostname: 192.168.122.1 > Uuid: fe722085-ac0f-4449-a43f-2dc9dd1fd8fb > State: Peer in Cluster (Disconnected) > > [root at vmserver-b peers]# gluster peer status > Number of Peers: 1 > > Hostname: 192.168.122.10 > Uuid: d8e1d7a0-077a-4a50-93f6-d3922e3b96b9 > State: Peer in Cluster (Connected) > > So, somehow, on vmserver-a is getting 192.168.122.1 in its peer list instead > of 192.168.122.11 > > Very strange. > > j > > > ~Atin > > > > >> Also send the glusterd log of the node where the commands have failed. > > > > > > The two logs are attached. The peer status says connected. The log file > > > says "FAILED : Host 192.168.122.10 is not in 'Peer in Cluster' state" > > > I'm confused. :) > > > > > > Thanks for your help on this! > > > > > > j > > > > > > > > > > > > _______________________________________________ > > > Gluster-users mailing list > > > Gluster-users at gluster.org > > > http://www.gluster.org/mailman/listinfo/gluster-users-- Joshua J. Kugler - Fairbanks, Alaska Azariah Enterprises - Programming and Website Design joshua at azariah.com - Jabber: pedahzur at gmail.com PGP Key: http://pgp.mit.edu/ ID 0x73B13B6A