Gluster users - Feb 2018 - Ip based peer probe volume create error

Hi,

i attached logs.

node3_cmd_history.log is empty because i did not run any command on node3.

At logs i saw node 1 peer status contains ip addresses as peer. But other nodes
node1 visible at it?s name .

node 1 peer status is 

root at pri:/var/log/glusterfs# gluster peer status
Number of Peers: 2

Hostname: 51.15.90.60
Uuid: eed0d6c6-90ef-4705-b3f9-0b028d769df3
State: Peer in Cluster (Connected)

Hostname: 163.172.151.120
Uuid: 42bc6ec5-5826-4a40-b2d3-8a34e6caabb7
State: Peer in Cluster (Connected)

node 2 

root at sec:/var/log/glusterfs# gluster peer status
Number of Peers: 2

Hostname: pri.ostechnix.lan
Uuid: 700fb496-4270-4e2a-878e-1a93ea0a5d0c
State: Peer in Cluster (Connected)
Other names:
51.15.77.14

Hostname: 163.172.151.120
Uuid: 42bc6ec5-5826-4a40-b2d3-8a34e6caabb7
State: Peer in Cluster (Connected)

node 3


root at third:/var/log/glusterfs# gluster peer status
Number of Peers: 2

Hostname: pri.ostechnix.lan
Uuid: 700fb496-4270-4e2a-878e-1a93ea0a5d0c
State: Peer in Cluster (Connected)
Other names:
51.15.77.14

Hostname: 51.15.90.10
Uuid: eed0d6c6-90ef-4705-b3f9-0b028d769df3
State: Peer in Cluster (Connected)






> On 7 Feb 2018, at 18:14, Gaurav Yadav <gyadav at redhat.com> wrote:
> 
> Hi,
> 
> Could you please send me the glusterd.logs and cmd-history logs from all
the nodes.
> 
> 
> Thanks
> Gaurav
> 
> On Wed, Feb 7, 2018 at 5:13 PM, Ercan Aydo?an <ercan.aydogan at
gmail.com <mailto:ercan.aydogan at gmail.com>> wrote:
> Hello,
> 
> i have dedicated 3.11.3 version glusterfs 3 nodes. i can create volumes if
peer probe with  hostname.
> 
> But ip based probe it?s not working.
> 
> What?s the correct /etc/hosts and hostname values when ip based peer probe.
> 
> Do i need still need peer FQDN names on /etc/hosts .
> 
> i need advice how can fix this issue.
> 
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org <mailto:Gluster-users at gluster.org>
> http://lists.gluster.org/mailman/listinfo/gluster-users
<http://lists.gluster.org/mailman/listinfo/gluster-users>
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: node2_glusterd.log
Type: application/octet-stream
Size: 12319 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: node2_cmd_history.log
Type: application/octet-stream
Size: 65 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0001.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: node1_cmd_history.log
Type: application/octet-stream
Size: 391 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0002.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0003.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: node1_glusterd.log
Type: application/octet-stream
Size: 26281 bytes
Desc: not available
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0003.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180207/835684d4/attachment-0004.html>

>From glusterd.logs, It looks like address resolution issue in
"glusterd_brickinfo_new_from_brick" functon call.

After seeing command history, I could see that you have 3 nodes, and
firstly you are peer probing 51.15.90.60  and  163.172.151.120 from
51.15.77.14
So here itself you have 3 node cluster, after all this you are going on
node 2 and again peer probing 51.15.77.14.
 Ideally it should work, with above steps, but due to some address
resolution volume create is getting failed, and from glusterd logs its not
clear.

For a workaround could you please try avoiding peer probing again from node
2 and give a try.

Meanwhile for further debugging could you please send us  glusterd debug
logs.





On Wed, Feb 7, 2018 at 9:18 PM, Ercan Aydo?an <ercan.aydogan at gmail.com>
wrote:
> Hi,
>
> i attached logs.
>
> node3_cmd_history.log is empty because i did not run any command on node3.
>
> At logs i saw node 1 peer status contains ip addresses as peer. But other
> nodes node1 visible at it?s name .
>
> node 1 peer status is
>
> root at pri:/var/log/glusterfs# gluster peer status
> Number of Peers: 2
>
> Hostname: 51.15.90.60
> Uuid: eed0d6c6-90ef-4705-b3f9-0b028d769df3
> State: Peer in Cluster (Connected)
>
> Hostname: 163.172.151.120
> Uuid: 42bc6ec5-5826-4a40-b2d3-8a34e6caabb7
> State: Peer in Cluster (Connected)
>
> node 2
>
> root at sec:/var/log/glusterfs# gluster peer status
> Number of Peers: 2
>
> Hostname: pri.ostechnix.lan
> Uuid: 700fb496-4270-4e2a-878e-1a93ea0a5d0c
> State: Peer in Cluster (Connected)
> Other names:
> 51.15.77.14
>
> Hostname: 163.172.151.120
> Uuid: 42bc6ec5-5826-4a40-b2d3-8a34e6caabb7
> State: Peer in Cluster (Connected)
>
> node 3
>
>
> root at third:/var/log/glusterfs# gluster peer status
> Number of Peers: 2
>
> Hostname: pri.ostechnix.lan
> Uuid: 700fb496-4270-4e2a-878e-1a93ea0a5d0c
> State: Peer in Cluster (Connected)
> Other names:
> 51.15.77.14
>
> Hostname: 51.15.90.10
> Uuid: eed0d6c6-90ef-4705-b3f9-0b028d769df3
> State: Peer in Cluster (Connected)
>
>
>
>
>
>
>
>
>
>
>
> On 7 Feb 2018, at 18:14, Gaurav Yadav <gyadav at redhat.com> wrote:
>
> Hi,
>
> Could you please send me the glusterd.logs and cmd-history logs from all
> the nodes.
>
>
> Thanks
> Gaurav
>
> On Wed, Feb 7, 2018 at 5:13 PM, Ercan Aydo?an <ercan.aydogan at
gmail.com>
> wrote:
>
>> Hello,
>>
>> i have dedicated 3.11.3 version glusterfs 3 nodes. i can create volumes
>> if peer probe with  hostname.
>>
>> But ip based probe it?s not working.
>>
>> What?s the correct /etc/hosts and hostname values when ip based peer
>> probe.
>>
>> Do i need still need peer FQDN names on /etc/hosts .
>>
>> i need advice how can fix this issue.
>>
>>
>> _______________________________________________
>> Gluster-users mailing list
>> Gluster-users at gluster.org
>> http://lists.gluster.org/mailman/listinfo/gluster-users
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.gluster.org/pipermail/gluster-users/attachments/20180208/07b40d72/attachment.html>

On 8/02/2018 4:45 AM, Gaurav Yadav wrote:
> After seeing command history, I could see that you have 3 nodes, and 
> firstly you are peer probing 51.15.90.60? and 163.172.151.120 from? 
> 51.15.77.14
> So here itself you have 3 node cluster, after all this you are going 
> on node 2 and again peer probing 51.15.77.14.
> ?Ideally it should work, with above steps, but due to some address 
> resolution volume create is getting failed, and from glusterd logs its 
> not clear.
Given 163.172.151.120 is on a different subnet I'd guess routing issues. 
Can it be pinged from both the other nodes?

-- 
Lindsay