Hi all! I still have never managed to fully wrap my head around how UDP data tunnels can be established between nodes. Everytime I think I understand it, I see something that confuses me again Just now I am seeing the following: I have nodes A, B + C A has everybody's keys and host configuration files. B and C only have A's key, and host config with A's public IP address. B and C DO NOT have each others keys. Likewise B and C both have a ConnectTo = A configuration directive. Only A is directly reachable with TCP and UDP on port 655, so there's no point to adding other ConnectTo Lines as neither B nor C are going to be directly reachable. I send some ICMP ping packets from B to C and intially I see as expected, the encapsulated tinc packets flow from B -> A and A -> C and back along that path/ But then, suddenly, I am seeing flow of UDP from B to C on port 655. Now, I'm not asking about the NAT hole punching here, but rather; How is this possible if B and C do not have each others keys? I thought I understood this before in that somehow the key data is shared over the meta connection, but then I read that no, each host much have the key of the other to establish the direct connection. But I am looking at tcpdump right now in the terminal and seeing the UDP tunnel packets flowing from B to C. I am really trying to understand how I can make this situation more persistent, but it seems so very random. Even in a case where I would make node B publicly reachable and add the keys everywhere, without an Explicit ConnectTo = B directive on node C, I still see packets routed via A. I would really like to know if there's some way to more reliably ensure that the UDP tunnel would be established from B to C and avoid a (transcontinental) route via A! Thank you if you can shed any light on this! k/
On Mon, May 14, 2018 at 4:44 AM, Keith Whyte <keith at rhizomatica.org> wrote:> But then, suddenly, I am seeing flow of UDP from B to C on port 655. > Now, I'm not asking about the NAT hole punching here, but rather; > > How is this possible if B and C do not have each others keys?> I thought > I understood this before in that somehow the key data is shared over the > meta connection,I believe the above is correct. In your example, A will provide B's key to C, and ... A will also provide C's key to B. B and C will then attempt to establish a direct connection. Whether or not B and C will succeed depends upon whether or not they have public IP addresses, what type of firewalls exist, whether or not B and C are on the same private LAN, etc.> but then I read that no, each host much have the key of > the other to establish the direct connection. But I am looking at > tcpdump right now in the terminal and seeing the UDP tunnel packets > flowing from B to C.Where do you read the above?> I am really trying to understand how I can make this situation more > persistent, but it seems so very random. > > Even in a case where I would make node B publicly reachable and add the > keys everywhere, without an Explicit ConnectTo = B directive on node C, > I still see packets routed via A. > > I would really like to know if there's some way to more reliably ensure > that the UDP tunnel would be established from B to C and avoid a > (transcontinental) route via A!I believe, by default, Tinc will always try to establish direct data connections between all nodes. Perhaps such connections are only attempted when there is data to send. That is, if two nodes never communicate with each other, they will not try to directly connect to each other. On-demand direct connections would explain why you observed the first few pings are routed via A rather than directly. Cheers, Parke
Here are a few facts that should make things clearer. Regarding keys: - The key used for the metaconnections (routing protocol over TCP) - i.e. the one you configure in your host files - is NOT the same as the key used for UDP data tunnels. - The key for data tunnels is negotiated over the metaconnections, by sending REQ_KEY and ANS_KEY messages over the metagraph (i.e. the graph of metaconnections). So, in your example, B will send a REQ_KEY message to A, which will forward it to C, which will respond with an ANS_KEY message, also forwarded through A. - These "data keys" are generated on-the-fly and are ephemeral, with an expiry time set by the KeyExpire configuration option (1 hour by default). When the expiry time is up, the key is renegotiated using the same procedure as above. (Note that in the 1.1 SPTPS protocol, the way the keys are exchanged is somewhat different, but the overall principle is the same.) Regarding forwarding of UDP data packets: - When tinc doesn't know if a node is reachable directly over UDP (or if the hole punching hasn't been done yet), it will always fall back to a "safe route", i.e. forwarding via an intermediate node (or even going over the TCP metaconnection if the node is already a direct neighbour). This is to avoid packet loss and to allow packets to flow as early as possible, even if the optimal route is not fully set up yet. - While tinc is doing the above, in parallel, it will send UDP "probe" packets to the destination node. These probes serve three purposes at the same time: (1) they enable tinc to determine if the destination node is reachable over UDP; (2) if it is, they enable tinc to determine the PMTU; and (3) they de facto act as NAT hole punchers (because both nodes will be sending probes to each over at the same time using the same ports). - If the UDP probes do come back, indicating that the node is directly reachable over UDP, tinc will stop sending actual data packets through the "safe route" and will start sending UDP packets directly to the destination node. (Though depending on packet size, tinc might wait for PMTU estimation to be complete first.) This happens transparently to the user; as you witnessed, the packets just suddenly start flowing over UDP automagically. (This whole process is quite clever and very well done IMHO - Guus gets the credit for this fantastic job.) - As long as the UDP packets keep flowing, tinc will continue to send packets over this direct UDP tunnel. - If the UDP packets stop flowing, tinc will fall back to the "safe route" again and this whole process restarts from scratch. This happens if the UDP tunnel suddenly breaks (because of a change in network conditions), but it can also happen simply because the UDP tunnel became idle for some time. Note that this is a somewhat simplified overview, and the details can differ slightly depending on whether you're using the 1.0 protocol or the 1.1 SPTPS protocol. On 14 May 2018 at 12:44, Keith Whyte <keith at rhizomatica.org> wrote:> Hi all! > > I still have never managed to fully wrap my head around how UDP data > tunnels can be established between nodes. > > Everytime I think I understand it, I see something that confuses me again > > > Just now I am seeing the following: > > I have nodes A, B + C > > A has everybody's keys and host configuration files. > > B and C only have A's key, and host config with A's public IP address. > > B and C DO NOT have each others keys. Likewise B and C both have a > ConnectTo = A configuration directive. > > Only A is directly reachable with TCP and UDP on port 655, so there's no > point to adding other ConnectTo Lines as neither B nor C are going to be > directly reachable. > > I send some ICMP ping packets from B to C and intially I see as > expected, the encapsulated tinc packets flow from B -> A and A -> C and > back along that path/ > > But then, suddenly, I am seeing flow of UDP from B to C on port 655. > Now, I'm not asking about the NAT hole punching here, but rather; > > How is this possible if B and C do not have each others keys? I thought > I understood this before in that somehow the key data is shared over the > meta connection, but then I read that no, each host much have the key of > the other to establish the direct connection. But I am looking at > tcpdump right now in the terminal and seeing the UDP tunnel packets > flowing from B to C. > > I am really trying to understand how I can make this situation more > persistent, but it seems so very random. > > Even in a case where I would make node B publicly reachable and add the > keys everywhere, without an Explicit ConnectTo = B directive on node C, > I still see packets routed via A. > > I would really like to know if there's some way to more reliably ensure > that the UDP tunnel would be established from B to C and avoid a > (transcontinental) route via A! > > Thank you if you can shed any light on this! > > k/ > > > > > > > > > _______________________________________________ > tinc mailing list > tinc at tinc-vpn.org > https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://www.tinc-vpn.org/pipermail/tinc/attachments/20180514/e91d749a/attachment.html>
Hi all, many thanks for the replies! On 14/05/18 19:05, Parke wrote:> On Mon, May 14, 2018 at 4:44 AM, Keith Whyte <keith at rhizomatica.org> wrote: >> but then I read that no, each host much have the key of >> the other to establish the direct connection. But I am looking at >> tcpdump right now in the terminal and seeing the UDP tunnel packets >> flowing from B to C. > Where do you read the above?I think it was here in the mailing list, and also in the documentation, but never mind. It's clear now that this applies to meta connections. On 14/05/18 20:39, Etienne Dechamps wrote:> > - While tinc is doing the above, in parallel, it will send UDP "probe" > packets to the destination node. These probes serve three purposes at > the same time: (1) they enable tinc to determine if the destination > node is reachable over UDP; (2) if it is, they enable tinc to > determine the PMTU; and (3) they de facto act as NAT hole punchers > (because both nodes will be sending probes to each over at the same > time using the same ports).Ah! I'm noticing that configuring static ports on the NAT in front of a tinc node with a public IP seems to make a difference to the number of other tinc nodes that can then establish direct data with it. Also, I have two tinc nodes behind a domestic gateway and one of them is doing direct data to a tinc node that is behind a NAT with no explicit port forwarding, the other is relaying. This is just the way it is with NAT hole punching, I suppose. (there are more than A,B+C in my real world setup)> (This whole process is quite clever and very well done IMHO - Guus > gets the credit for this fantastic job.) >Yes indeed!