tinc - Feb 2017 - LocalDiscovery flip flopping and network design tips

On Tue, Feb 14, 2017 at 1:46 PM, Guus Sliepen <guus at tinc-vpn.org>
wrote:
> On Tue, Feb 14, 2017 at 11:21:34AM -0500, James Hartig wrote:
>
>> Those 2 boxes are in the same subnet and have addresses of 10.240.0.4
and
>> 10.240.0.5, respectively, on their eth0 interface. Port 655 on tcp and
udp
>> is open to the world. The tinc_test_2 box has a ConnectTo of
tinc_test_1.
>> When tinc_test_2 is started, it prints out:
>>   UDP address of tinc_test_1 set to 104.154.59.151 port 655
>>   UDP address of tinc_test_1 set to 10.240.0.4 port 655
>>   UDP address of tinc_test_1 set to 104.154.59.151 port 655
>>   UDP address of tinc_test_1 set to 10.240.0.4 port 655
>> repeatedly for a minute or so before finally settling on 10.240.0.4.
>>
>> Is there a reason it's flip flopping? Is that expected? Am I doing
>> something wrong?
>
> No, you are not doing anything wrong. Although I've not seen this kind
> of flip-flopping behavior myself, it is possible this behaviour occurs,
> although it should only happen in the first 10 seconds or so. When
> LocalDiscovery is enabled, tinc tries to send probe packets to both the
> address it learned from its TCP connections and to the local network.
> When receiving a valid packet, it notes the source address of that
> packet. If it is different from the source address of the previous valid
> UDP packet, a log message is printed about it.
When you say "and to the local network" what IP does it try to send to
on the local network? The subnet address? Is this configurable with
the LocalDiscoveryAddress?

For instance if I have a host:
/etc/tinc/test/tinc_test_5
Subnet = 10.80.0.6/32
Address = 104.154.59.152

Will it try to contact 10.80.0.6 over the local network, or if I
specify a LocalDiscoveryAddress on tinc_test_5 of 10.240.0.6, will it
use 10.240.0.6 instead? Apologizes for my confusion. Since I'm setting
Subnet to be the internal IP, is there any difference in that versus
LocalDiscoveryAddress?
>> Additionally, we have multiple Google Compute regions with their own
>> subnets and external DCs with their own subnets and we'd like to
install
>> tinc on all servers but keep inner-Google traffic to the internal IPs
and
>> not over external IPs since it's an order of magnitude cheaper. My
first
>> thinking is a hub and spoke model. We have 2 boxes in each subnet that
have
>> port 655 open to the world, and all the other servers have 655 open to
>> internal ips only. With LocalDiscovery (as well as IndirectData = yes
on
>> "non-public" servers) this works work pretty well, as far as
I can tell.
>> But it wouldn't solve the inner-Google traffic between subnets
since Google
>> Subnet0 would talk over public to Google Subnet1. What's the best
way of
>> doing something like this? I was thinking maybe 2 instances of tinc on
the
>> "public" boxes, but Google servers only have a single
interface, eth0, that
>> has the internal IP, so I couldn't listen on the external and
internal IPs
>> separately.
>
> You can use BindToAddress to have tinc bind to a specific IP address. So
> you can have two tinc daemons, one binding to the internal IP address,
> and another to the external IP address, even if they are residing on the
> same network interface. Would that help?
I can't use BindToAddress as far as I know because there isn't an
interface on the box with the external IP. The servers only have 1
interface, eth0, that has a single IP, eg 10.240.0.4, and Google
forwards packets to their public IP, eg 104.154.59.151, to the
internal IP, eg 10.240.0.4, so the server itself doesn't actually know
its public IP.

I think if my understanding above of the local network stuff is
correct, and since it's unicast in 1.1, then it should work fine.
Google Subnet0 and Google Subnet1 can talk to each other's private
IPs, so theoretically discovery should work and it should use their
private IPs.
> --
> Met vriendelijke groet / with kind regards,
>      Guus Sliepen <guus at tinc-vpn.org>
>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc

On 14 February 2017 at 18:59, James Hartig <james at levenlabs.com>
wrote:
> When you say "and to the local network" what IP does it try to
send to
> on the local network? The subnet address?
No. The Subnet option deals with routing *inside* the VPN, not the
underlying "real" network.

In tinc 1.1, the address that local discovery probes are sent to is
the local address of the recipient node, as determined by the socket
local address of its metaconnection. That's the address shown next to
"local" in the dump edges output. In your case the local address is
advertised correctly - there is no problem there.

Hang on a second. I've just re-read your original message and I
believe you are confused about what the "Subnet" option does. Again,
it deals with addresses *inside* the VPN. In the configuration you
posted you seem to be using 10.240.0.4 and 10.240.0.5 as internal
addresses, but then your other statements (and especially your dump
edges output) seem to indicate that 10.240.0.4 and 10.240.0.5 are
*also* the physical addresses of the machines on their physical
network interfaces.

That's not going to work: as soon as tinc manages to establish the
VPN, 10.240.0.4 and 10.240.0.5 become routable on *both* the virtual
and physical interfaces, resulting in conflicts, and it all goes
downhill from there. That would completely explain the weird phenomena
you're reporting. If you make sure to use different IP subnets for VPN
addresses and physical addresses, your problems should go away.

On 14 February 2017 at 20:36, Etienne Dechamps <etienne at edechamps.fr>
wrote:
> On 14 February 2017 at 18:59, James Hartig <james at levenlabs.com>
wrote:
>> When you say "and to the local network" what IP does it try
to send to
>> on the local network? The subnet address?
>
> No. The Subnet option deals with routing *inside* the VPN, not the
> underlying "real" network.
>
> In tinc 1.1, the address that local discovery probes are sent to is
> the local address of the recipient node, as determined by the socket
> local address of its metaconnection. That's the address shown next to
> "local" in the dump edges output. In your case the local address
is
> advertised correctly - there is no problem there.