Gluster users - Jul 2013 - 3.4.beta4 - Peer Connected but in State "Accepted Peer Request"

I had a node that was attached but not functioning properly. In this case the
gluster daemon itself wouldn't launch giving this at startup:
http://pastebin.com/rnsMNGBT

I went into /var/lib/glusterd and removed everything under peers/volumes - thus
allowing the daemon to start cleanly.

After doing a gluster volume sync from a known good peer - everything appears
normal except I cannot add any new volumes using that peer:

volume create: perftest: failed: Host cs75-i is not in 'Peer in Cluster'
state

Peer status reads:

Hostname: cs75-i
Uuid: 339280d7-b4d3-414b-be84-e8e4638f3751
State: Accepted peer request (Connected)

restarting the daemon again on cs75-i yields:

Hostname: cs75-i
Uuid: 339280d7-b4d3-414b-be84-e8e4638f3751
State: Peer in Cluster (Connected)

To which I then receive:

volume create: perftest: success: please start the volume to access data

The questions is, what is the difference between these two states, and why does
a simple restart on the daemon correct the issue?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130712/8f42cf68/attachment.html>

On 13-Jul-2013 12:43 AM, "Ryan Aydelott" <ryade at mcs.anl.gov>
wrote:
>
> I had a node that was attached but not functioning properly. In this case
the gluster daemon itself wouldn't launch giving this at startup:
http://pastebin.com/rnsMNGBT
>
> I went into /var/lib/glusterd and removed everything under peers/volumes
- thus allowing the daemon to start cleanly.
>
> After doing a gluster volume sync from a known good peer - everything
appears normal except I cannot add any new volumes using that
peer:
>
> volume create: perftest: failed: Host cs75-i is not in 'Peer in
Cluster'
state
>
> Peer status reads:
>
> Hostname: cs75-i
> Uuid: 339280d7-b4d3-414b-be84-e8e4638f3751
> State: Accepted peer request (Connected)
>
> restarting the daemon again on cs75-i yields:
>
> Hostname: cs75-i
> Uuid: 339280d7-b4d3-414b-be84-e8e4638f3751
> State: Peer in Cluster (Connected)
>
> To which I then receive:
>
> volume create: perftest: success: please start the volume to access data
>
> The questions is, what is the difference between these two states, and
why does a simple restart on the daemon correct the
issue?
>
During a peer probe, the peers exchange data between each other several
times, before successfully finishing the probe. The peers move through
several states when these exchanges happen. The 'Peer in Cluster' state
is
the final state reached on a successful probe. The other states are
intermediary states, movement among which depend on these data exchanges.
If one of these data exchanges doesn't happen, then the peer gets stuck in
the state.
Some of the exchanges also happen when a peer starts up, so a restart can
sometimes solve these kind of issues.

If you could get the glusterd logs, for the peer we could try to find out
why the peer got stuck in the first place.

- Kaushal

_______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130713/936365e1/attachment.html>