tinc - Apr 2015 - Help needed with Tinc Setup on remote hosts and lots of ISPs / Failover Problems between ISPs

Hello Guus, 

thank you very much for your suggestions, I could not dive into it further
because I was traveling, but now I have time to reconfigure the network.

At first I really like the idea of having 3 Daemons on the headquarter, one for
each ISP. The firewall should forward the port 655 from each ISP's public IP
Address to my internal server and to the ports 655, 656, 657 respectively, which
I guess you had in mind when you wrote:
> If you want/need tighter control, you might have to run three tinc
> daemons at the headquarters, one for each ISP you have there.
My question now is, for every tinc daemon I need a tun or tap device, so how
should the routing be done correctly? I have the VPN Network 10.69.0.0/11.

Right now I have one tinc daemon and one tun0 device. I route the complete
10.96.0.0/11 to tun0. How do I have to proceed if I want this 10.96.0.0/11 be
available from all 3 tinc-deamons (which from the internet-side will have every
one it's own public IP with a different ISP)?

The idea would be that I:

* do not have to care if a line goes down, remote offices just reconnect to one
of the other lines
* in the event of a severe degradation of a line I just stop the corresponding
daemon, all remote offices which had used this internet line just reconnect to
one of the others
* do not really care to which ISP every remote office connects

But I am not sure about the routing on the VPN server where the 3 daemons should
reside ...

Thank you,
best
Ray

On Thu, Apr 16, 2015 at 10:09:05PM +0200, Raimund Sacherer wrote:
> At first I really like the idea of having 3 Daemons on the headquarter, one
for each ISP. The firewall should forward the port 655 from each ISP's
public IP Address to my internal server and to the ports 655, 656, 657
respectively
[...]
> My question now is, for every tinc daemon I need a tun or tap device, so
how should the routing be done correctly? I have the VPN Network 10.69.0.0/11.
> 
> Right now I have one tinc daemon and one tun0 device. I route the complete
10.96.0.0/11 to tun0. How do I have to proceed if I want this 10.96.0.0/11 be
available from all 3 tinc-deamons (which from the internet-side will have every
one it's own public IP with a different ISP)?
> 
> The idea would be that I:
> 
> * do not have to care if a line goes down, remote offices just reconnect to
one of the other lines
> * in the event of a severe degradation of a line I just stop the
corresponding daemon, all remote offices which had used this internet line just
reconnect to one of the others
> * do not really care to which ISP every remote office connects
In this case, I think it's best if you do not add any routes in the
tinc-up script. So there, you just configure the address of the
interface, but don't supply a netmask:

#!/bin/sh
ifconfig $INTERFACE 10.96.x.y

Then, assuming you run tinc in router Mode (the default), you should
create a script named "subnet-up" in the same directory as tinc-up,
and
put this in it:

#!/bin/sh
ip addr add $SUBNET dev $INTERFACE

And a "subnet-down" script:

#!/bin/sh
ip addr del $SUBNET dev $INTERFACE

This will cause tinc to automatically add and remove routing entries,
depending on which nodes are reachable. You will get multiple routing
entries for the same subnet but to different interfaces. One of them
will be chosen by the kernel based on the order of addition, but you
don't care about it so it's fine. If tinc detects that a node goes down,
the subnet-down script will remove the offending route, and the kernel
will then use another one. And if you manually stop a tincd its routes
will be removed as well.

-- 
Met vriendelijke groet / with kind regards,
     Guus Sliepen <guus at tinc-vpn.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20150424/a58128b4/attachment.sig>

Comments below

On Fri, 24 Apr 2015, Guus Sliepen wrote:
> On Thu, Apr 16, 2015 at 10:09:05PM +0200, Raimund Sacherer wrote:
> 
> > At first I really like the idea of having 3 Daemons on the
headquarter, one for each ISP. The firewall should forward the port 655 from
each ISP's public IP Address to my internal server and to the ports 655,
656, 657 respectively
> [...]
> > My question now is, for every tinc daemon I need a tun or tap device,
so how should the routing be done correctly? I have the VPN Network
10.69.0.0/11.
> > 
> > Right now I have one tinc daemon and one tun0 device. I route the
complete 10.96.0.0/11 to tun0. How do I have to proceed if I want this
10.96.0.0/11 be available from all 3 tinc-deamons (which from the internet-side
will have every one it's own public IP with a different ISP)?
> > 
> > The idea would be that I:
> > 
> > * do not have to care if a line goes down, remote offices just
reconnect to one of the other lines
> > * in the event of a severe degradation of a line I just stop the
corresponding daemon, all remote offices which had used this internet line just
reconnect to one of the others
> > * do not really care to which ISP every remote office connects
> 
> In this case, I think it's best if you do not add any routes in the
> tinc-up script. So there, you just configure the address of the
> interface, but don't supply a netmask:
> 
> #!/bin/sh
> ifconfig $INTERFACE 10.96.x.y
Won't a netmask of 255.255.255.255 be better than not specifying any? 
Otherwise it falls back to old classful adressing and would assume 
10.0.0.0/8 - which is clearly always wrong.

# ifconfig eth2 10.96.3.4
# route -n |grep eth2
10.0.0.0        0.0.0.0         255.0.0.0       U     0   0   0 eth2

Or are tun devices different in this regard?
> Then, assuming you run tinc in router Mode (the default), you should
> create a script named "subnet-up" in the same directory as
tinc-up, and
> put this in it:
> 
> #!/bin/sh
> ip addr add $SUBNET dev $INTERFACE
> 
> And a "subnet-down" script:
> 
> #!/bin/sh
> ip addr del $SUBNET dev $INTERFACE
Don't you mean "ip route add/del" here?
> This will cause tinc to automatically add and remove routing entries,
> depending on which nodes are reachable. You will get multiple routing
> entries for the same subnet but to different interfaces. One of them
> will be chosen by the kernel based on the order of addition, but you
> don't care about it so it's fine. If tinc detects that a node goes
down,
> the subnet-down script will remove the offending route, and the kernel
> will then use another one. And if you manually stop a tincd its routes
> will be removed as well.
> 
> 
c'ya
sven-haegar

-- 
Three may keep a secret, if two of them are dead.
- Ben F.

Hello Guus, 
> This will cause tinc to automatically add and remove routing entries,
> depending on which nodes are reachable. You will get multiple routing
> entries for the same subnet but to different interfaces. One of them
> will be chosen by the kernel based on the order of addition, but you
> don't care about it so it's fine. If tinc detects that a node goes
down,
> the subnet-down script will remove the offending route, and the kernel
> will then use another one. And if you manually stop a tincd its routes
> will be removed as well.
Let's say I do care to a point which line should be taken, I just should be
able to add metrics (weight) to the routes and the kernel should choose the
route with the least metric, right?

Best
Ray