Gluster users - Mar 2015 - Peers not connecting after changing IP address

On my setup: on the host from which I peer probed, it's all hostnames.
On the other hosts, it's all IPs.

Can you check if it's the case on your setup too?

Thanks,
JF


On 10/03/15 16:29, Alex Crow wrote:
> Hi,
> 
> They only have the hostname:
> 
> uuid=22b88f85-0554-419f-a279-980fceaeaf49
> state=3
> hostname1=zalma
> 
> And pinging these hostnames give the correct IP. Still no connection
> though.
> 
> Thanks,
> 
> Alex
> 
> On 10/03/15 15:04, JF Le Fill?tre wrote:
>> Hello,
>>
>> Check the files in the peer directory:
>>
>> /var/lib/glusterd/peers
>>
>> They contain the IP addresses of the peers.
>>
>> I haven't done it but I assume that if you update those files on
all
>> servers you should be back online.
>>
>> Thanks,
>> JF
>>
>>
>> On 10/03/15 16:00, Alex Crow wrote:
>>> Hi,
>>>
>>> I've had a 4 node Dis/Rep cluster up and running for a while,
but
>>> recently moved two of the nodes (the replicas of the other 2) to a
>>> nearby datacentre. The IP addresses of the moved two therefore
changed,
>>> but I updated the /etc/hosts file on all four hosts to reflect the
>>> change (and the peers were all probed by name, not IP).
>>>
>>> However at each site the other two peers show as disconnected, even
>>> though the servers can all successfully talk to each other. Is
there
>>> some way I can kick this back into life?
>>>
>>> Regards,
>>>
>>> Alex
>>>
> 
-- 

 Jean-Fran?ois Le Fill?tre
 -------------------------------
 HPC Systems Administrator
 LCSB - University of Luxembourg
 -------------------------------
 PGP KeyID 0x134657C6

Hi JF,

They are all hostnames, no IPs anywhere. The odd thing is that now, the 
remote site says everything is up, whereas the local servers only show 
each other as connected. It's a bit odd.

Cheers

Alex

On 10/03/15 15:40, JF Le Fill?tre wrote:
> On my setup: on the host from which I peer probed, it's all hostnames.
> On the other hosts, it's all IPs.
>
> Can you check if it's the case on your setup too?
>
> Thanks,
> JF
>
>
> On 10/03/15 16:29, Alex Crow wrote:
>> Hi,
>>
>> They only have the hostname:
>>
>> uuid=22b88f85-0554-419f-a279-980fceaeaf49
>> state=3
>> hostname1=zalma
>>
>> And pinging these hostnames give the correct IP. Still no connection
>> though.
>>
>> Thanks,
>>
>> Alex
>>
>> On 10/03/15 15:04, JF Le Fill?tre wrote:
>>> Hello,
>>>
>>> Check the files in the peer directory:
>>>
>>> /var/lib/glusterd/peers
>>>
>>> They contain the IP addresses of the peers.
>>>
>>> I haven't done it but I assume that if you update those files
on all
>>> servers you should be back online.
>>>
>>> Thanks,
>>> JF
>>>
>>>
>>> On 10/03/15 16:00, Alex Crow wrote:
>>>> Hi,
>>>>
>>>> I've had a 4 node Dis/Rep cluster up and running for a
while, but
>>>> recently moved two of the nodes (the replicas of the other 2)
to a
>>>> nearby datacentre. The IP addresses of the moved two therefore
changed,
>>>> but I updated the /etc/hosts file on all four hosts to reflect
the
>>>> change (and the peers were all probed by name, not IP).
>>>>
>>>> However at each site the other two peers show as disconnected,
even
>>>> though the servers can all successfully talk to each other. Is
there
>>>> some way I can kick this back into life?
>>>>
>>>> Regards,
>>>>
>>>> Alex
>>>>
-- 
This message is intended only for the addressee and may contain
confidential information. Unless you are that person, you may not
disclose its contents or use it in any way and are requested to delete
the message along with any attachments and notify us immediately.
"Transact" is operated by Integrated Financial Arrangements plc. 29
Clement's Lane, London EC4N 7AE. Tel: (020) 7608 4900 Fax: (020) 7608
5300. (Registered office: as above; Registered in England and Wales
under number: 3727592). Authorised and regulated by the Financial
Conduct Authority (entered on the Financial Services Register; no. 190856).

Hi all,

You can take this post as a warning to wear your glasses and 
double-check routing.

I had a few ip rule/ip route entries to direct traffic in and out of 
specific interfaces. The "office" range was under 172.18.x and the new
datacenter was under 192.168.x. I'd not spotted that I'd changed the 
last two octets on my ip rule/route settings only on the "datacenter" 
range, so the "office" one had the first two octets from its original 
range and the last two from the DC range.

I have had this problem before, and I wonder if it's just my own brain 
or does anyone get confused between eg.:

172.16.24.110
192.168.24.10

It's like my visual system can't parse the difference, especially when 
there are a few of them in a list together!

Apologies for wasting time and thanks to JF.

Cheers

Alex

On 10/03/15 15:40, JF Le Fill?tre wrote:
> On my setup: on the host from which I peer probed, it's all hostnames.
> On the other hosts, it's all IPs.
>
> Can you check if it's the case on your setup too?
>
> Thanks,
> JF
>
>
> On 10/03/15 16:29, Alex Crow wrote:
>> Hi,
>>
>> They only have the hostname:
>>
>> uuid=22b88f85-0554-419f-a279-980fceaeaf49
>> state=3
>> hostname1=zalma
>>
>> And pinging these hostnames give the correct IP. Still no connection
>> though.
>>
>> Thanks,
>>
>> Alex
>>
>> On 10/03/15 15:04, JF Le Fill?tre wrote:
>>> Hello,
>>>
>>> Check the files in the peer directory:
>>>
>>> /var/lib/glusterd/peers
>>>
>>> They contain the IP addresses of the peers.
>>>
>>> I haven't done it but I assume that if you update those files
on all
>>> servers you should be back online.
>>>
>>> Thanks,
>>> JF
>>>
>>>
>>> On 10/03/15 16:00, Alex Crow wrote:
>>>> Hi,
>>>>
>>>> I've had a 4 node Dis/Rep cluster up and running for a
while, but
>>>> recently moved two of the nodes (the replicas of the other 2)
to a
>>>> nearby datacentre. The IP addresses of the moved two therefore
changed,
>>>> but I updated the /etc/hosts file on all four hosts to reflect
the
>>>> change (and the peers were all probed by name, not IP).
>>>>
>>>> However at each site the other two peers show as disconnected,
even
>>>> though the servers can all successfully talk to each other. Is
there
>>>> some way I can kick this back into life?
>>>>
>>>> Regards,
>>>>
>>>> Alex
>>>>