Hello, I'm running a tinc network including dozens of nodes in switch mode. Some are running stable branch 1.0, while a small set of nodes are running 1.1 with ed25519 support. I discovered some routing issue between two nodes: (names are hidden) A (1.1): ConnectTo = B ConnectTo = C IndirectData = yes Mode = Switch B (1.0): Mode = Switch C (1.1 but only with RSA key): Mode = Switch Connect = D IndirectData = yes D (1.1): Mode = Switch ConnectTo = B And here are the `tinc dump edges` output, non-releated edges are filtered. B to A at xxx port xxx local xxx port xxx options 400000d weight 51 B to C at xxx port xxx local xxx port xxx options c weight 706 B to D at xxx port xxx local xxx port xxx options 400000c weight 984 A to B at xxx port xxx local xxx port xxx options d weight 51 A to C at xxx port xxx local xxx port xxx options d weight 29 C to B at xxx port xxx local xxx port xxx options c weight 706 C to A at xxx port xxx local xxx port xxx options 400000d weight 29 C to D at xxx port xxx local xxx port xxx options 400000c weight 182 D to B at xxx port xxx local xxx port xxx options c weight 984 D to C at xxx port xxx local xxx port xxx options c weight 182 Both A and D are configured with both Ed25519 and RSA keys. Node B and C are configured with only RSA key. tinc dump nodes on node A | grep D: D id xxxx at xxx port 655 cipher 0 digest 0 maclength 0 compression 0 options 400000c status 0078 nexthop B via A distance 2 pmtu 1518 (min 0 max 1518) tinc dump nodes on node D | grep A: A id xxx at xxx port 8964 cipher 0 digest 0 maclength 0 compression 0 options 400000d status 0078 nexthop B via C distance 2 pmtu 1518 (min 0 max 1518) And then, A ping test from A $ ping D approx 400ms Network traffic from both A and D should A send data via B and then D. and D replies data via B and then A. Of course ping from D -> A is approx 400ms as ping measures round-trip time. And next, I did a weird thing, delete the ed25519 key on D, making D insufficient for sptps, and restart tincd on D. The dump edges is similiar. And the ping from A -> D reduced to 200ms immediately. Network traffic from both A and D shows A send data via B and then D, and D replies data via C and then A. Question 1: Anybody can explain the `dump nodes` on D: nexthop B via C Why is it like this? I don't believe the forwarding chain will involve both of B, and C. I think either B or C is enough. Question 2:>From `dump nodes` on A, why tinc forwards data from A to D through Brather than C. Both of the routes have 2 hops, but forwarding via C will have lower weight. In tinc's weight, lower weight usually means better connection. Question 3: Why A -> D and D -> A forwards via different sets of nodes when only RSA keys are used ? As I know the graph in tinc looks directed but each edge has the same weight with its reverse edge (at least in my experiment). Question 4: Why enabling sptps or removing it will result in different routes? Thanks
On Tue, Apr 21, 2015 at 10:25:23PM +0800, Qijiang Fan wrote:> I'm running a tinc network including dozens of nodes in switch mode. > Some are running stable branch 1.0, while a small set of nodes are > running 1.1 with ed25519 support.[...]> Both A and D are configured with both Ed25519 and RSA keys. > Node B and C are configured with only RSA key.[...]> And then, > A ping test from A > $ ping D > approx 400ms[...]> And next, I did a weird thing, delete the ed25519 key on D, making D > insufficient for sptps, and restart tincd on D. > And the ping from A -> D reduced to 200ms immediately.The reason is that if both nodes A and D have Ed25519 keys, they will use the new protocol between them. However, if they cannot reach each other directly via UDP, packets have to be forwarded via B and/or C. Since they do not understand the new protocol, messages between A and D have to be shoehorned in the legacy protocol in an inefficient way. When you removed the Ed25519 key, nodes A and D used the legacy protocol between them, allowing for more efficient forwarding of traffic.> Question 1: > Anybody can explain the `dump nodes` on D: > nexthop B via C > Why is it like this? I don't believe the forwarding chain will involve > both of B, and C. I think either B or C is enough."nexthop" and "via" are somewhat different things. For VPN traffic, packets will be sent to the "via" node. For metadata, traffic will go via the "nexthop" in the metaconnection graph.> Question 2: > From `dump nodes` on A, why tinc forwards data from A to D through B > rather than C. Both of the routes have 2 hops, but forwarding via C > will have lower weight. In tinc's weight, lower weight usually means > better connection.Here the problem is likely because you set IndirectData = yes in tinc.conf. This will actually prevent traffic going directly to nodes that you don't have a ConnectTo for. So remove that option and try again.> Question 3: > Why A -> D and D -> A forwards via different sets of nodes when only > RSA keys are used ? > As I know the graph in tinc looks directed but each edge has the same > weight with its reverse edge (at least in my experiment). > > Question 4: > Why enabling sptps or removing it will result in different routes?Again, since you have IndirectData = yes, tinc will make suboptimal decisions. -- Met vriendelijke groet / with kind regards, Guus Sliepen <guus at tinc-vpn.org> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: Digital signature URL: <http://www.tinc-vpn.org/pipermail/tinc/attachments/20150424/d6a73318/attachment.sig>