tinc - Aug 2017 - using both ConnectTo and AutoConnect to avoid network partitions

Hi Guus

Thanks for clarifying. Some follow up questions:

- How do we patch 1.1pre14 with this fix? Or will there be a 1.1pre15 to
upgrade to?
- What is the workaround until we patch with this fix? Using a combination
of AutoConnect and ConnectTo?
- When we use ConnectTo, is it mandatory to have a cert file in the hosts/*
dir with an IP to ConnectTo ?


   -nirmal

On Tue, Aug 22, 2017 at 12:10 PM, Guus Sliepen <guus at tinc-vpn.org>
wrote:
> On Mon, Aug 21, 2017 at 05:37:06PM -0700, Nirmal Thacker wrote:
>
> > Today our Tinc network saw a network partition when we took one tinc
node
> > down.
> >
> > We knew there was a network partition since the graph showed a split.
> This
> > graph is not very helpful but its what I have at the moment:
> >
> > http://i.imgur.com/XP2PSWc.png
>
> The graph is very clear.
>
> > Some questions:
> > - should we have a combination of both ConnectTo and AutoConnect to
avoid
> > such a network split?
>
> No, it's a bug in AutoConnect. I've just pushed a fix to the 1.1
branch
> that will try to continue to connect to unreachable nodes, even if a
> node already has 3 or more connections.
>
> > - Say we have 3 ConnectTo variables and then AutoConnect=yes, would
there
> > ever be more than 3 connections ? (I read somewhere that AutoConnect
will
> > make upto 3 connections only)
>
> There can always be more than 3 connections, even when AutoConnect is
> enabled.
>
> When starting, tinc will try to make outgoing connections to all nodes
> listed in ConnectTo statements. This can be more than 3 nodes. After
> that, the AutoConnect algorithm kicks in.
>
> The AutoConnect algorithm tries to regulate the number of established
> connections, either by creating more outgoing connections, or by
> closing connections that it made itself. It will never close incoming
> connections, and it also won't close outgoing connections to a node
that
> isn't already connected to at least one other node.
>
> Ideally, after a while connections get rearranged such that no node has
> more than 3 connections. But this can take a while, or it might never
> happen; for example if you have 5 nodes behind NAT, and one public node,
> then the public node will always have 5 connections.
>
> --
> Met vriendelijke groet / with kind regards,
>      Guus Sliepen <guus at tinc-vpn.org>
>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20170822/bf08c61c/attachment.html>

On Tue, Aug 22, 2017 at 03:19:18PM -0700, Nirmal Thacker wrote:
> - How do we patch 1.1pre14 with this fix? Or will there be a 1.1pre15 to
> upgrade to?
There will be an 1.1pre15, but if you want you can apply the following
commit:

https://tinc-vpn.org/git/browse?p=tinc;a=commitdiff;h=92fdabc439bdb5e16f64a4bf2ed1deda54f7c544
> - What is the workaround until we patch with this fix? Using a combination
> of AutoConnect and ConnectTo?
Yes.
> - When we use ConnectTo, is it mandatory to have a cert file in the hosts/*
> dir with an IP to ConnectTo ?
Yes. Tinc always needs the public key of a peer and an Address in order
to be able to connect to it.

-- 
Met vriendelijke groet / with kind regards,
     Guus Sliepen <guus at tinc-vpn.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20170823/53fa99b7/attachment.sig>

Thanks Guus

I have one more question.

- We see several log messages that we dont currently understand - Can you
comment on what they mean and if they are concerning? I've obfuscated
IP's
and node names so please ignore those. Our tinc daemon command is: tincd -n
<vpn name>

-- Received short packet
-- Got REQ_KEY from node003 while we already started a SPTPS session!
-- Invalid packet seqno: 7951 != 1 from node003 (22.22.22.22 port 655)
-- Failed to verify SIG record from node003 (22.22.22.22 port 655)
-- message repeated 3 times: [ Received short packet]
-- Metadata socket read error for node004 (33.33.33.33 port 655):
Connection reset by peer
-- Failed to decrypt and verify packet from node005 (44.44.44.44 port 655)

   -nirmal

On Tue, Aug 22, 2017 at 11:08 PM, Guus Sliepen <guus at tinc-vpn.org>
wrote:
> On Tue, Aug 22, 2017 at 03:19:18PM -0700, Nirmal Thacker wrote:
>
> > - How do we patch 1.1pre14 with this fix? Or will there be a 1.1pre15
to
> > upgrade to?
>
> There will be an 1.1pre15, but if you want you can apply the following
> commit:
>
> https://tinc-vpn.org/git/browse?p=tinc;a=commitdiff;h>
92fdabc439bdb5e16f64a4bf2ed1deda54f7c544
>
> > - What is the workaround until we patch with this fix? Using a
> combination
> > of AutoConnect and ConnectTo?
>
> Yes.
>
> > - When we use ConnectTo, is it mandatory to have a cert file in the
> hosts/*
> > dir with an IP to ConnectTo ?
>
> Yes. Tinc always needs the public key of a peer and an Address in order
> to be able to connect to it.
>
> --
> Met vriendelijke groet / with kind regards,
>      Guus Sliepen <guus at tinc-vpn.org>
>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20170823/d679b907/attachment.html>

Hi Guus

Following your suggestion we reconfigured our tinc network as follows.
Here is a new graph and below is our updated configuration:
http://imgur.com/a/n6ksh

- 2 Tinc nodes (yellow labels) have a public external IP and port 655 open.
They both have ConnectTo's to each other and AutoConnect = yes
- The remainder tinc nodes (blue labels) have their tinc.conf set up as
follows:
      ConnectTo = yellow1
      ConnectTo = yellow2
      AutoConnect = yes
- Blue labeled nodes also have their port 655 open, but no node in the
network has a ConnectTo to any blue labeled node
- we are still using tinc1.1pre14
- The configuration was loaded by ensuring:
    - each node has the keys and Address for their ConnectTo targets
    - tinc was reloaded using the command: sudo tinc -n <vpn_name> reload

The main motivation to do this: To avoid the network split bug we hit, that
was addressed earlier in this email and to do this by ensuring deliberate
and redundant connections to yellow1 and yellow2

We are concerned that:
- We still dont see edges in the graph that show connections between every
blue labeled node to both the yellow labeled nodes

Any reason why we dont see these edges?

Is there something missing in our configuration?

Thanks


   -nirmal

On Tue, Aug 22, 2017 at 11:08 PM, Guus Sliepen <guus at tinc-vpn.org>
wrote:
> On Tue, Aug 22, 2017 at 03:19:18PM -0700, Nirmal Thacker wrote:
>
> > - How do we patch 1.1pre14 with this fix? Or will there be a 1.1pre15
to
> > upgrade to?
>
> There will be an 1.1pre15, but if you want you can apply the following
> commit:
>
> https://tinc-vpn.org/git/browse?p=tinc;a=commitdiff;h>
92fdabc439bdb5e16f64a4bf2ed1deda54f7c544
>
> > - What is the workaround until we patch with this fix? Using a
> combination
> > of AutoConnect and ConnectTo?
>
> Yes.
>
> > - When we use ConnectTo, is it mandatory to have a cert file in the
> hosts/*
> > dir with an IP to ConnectTo ?
>
> Yes. Tinc always needs the public key of a peer and an Address in order
> to be able to connect to it.
>
> --
> Met vriendelijke groet / with kind regards,
>      Guus Sliepen <guus at tinc-vpn.org>
>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20170831/365f43ae/attachment.html>