tinc - Mar 2018 - Help on a Nat To Nat soluction - tinc servers won't ping remote clients

I have a three tinc server setup, similar to "4.3 How Connections
Work" using the configuration mostly
likehttp://ostolc.org/site-to-site-vpn-with-tinc.html

The clients (Ubuntus, Debians and Windows 10s) can all ping (and SSH) to each
other remotely.
As far as that is concerned it's working great - thanks so much for some
great software.

However, on each of the Tinc servers (A and C) neither of them can ping other
remote clients.
Of course, A and C can ping each other.
If I use tcpdump -nni tun0 icmpI can see the echo packets leave the server, and
on a remote client see the request received and the reply sent.
However the server never gets the reply.It seems that on each server there is no
internal routing between enp1s0 and tun0 for IPs that are not server IPs
I guess I can live with such a limitation, but would still like to know why!!
Here's Server A config. Of course it's symmetrical so the other two will
be similar.
B is a DigitalOcean Droplet
TINC.CONFName = AAddressFamily = ipv4ConnectTo = BDevice =
/dev/net/tunLocalDiscovery = yes

TINC-UPip link set $INTERFACE upip addr add 192.168.20.3/24 dev $INTERFACEroute
add -net 192.168.14.0/24 gw 192.168.20.3
route add -net 192.168.6.0/24  gw 192.168.4.99
HOST AAddress = A.dyndns.org
Port = 655
##Subnet on the virtual private network that is local for this host.Subnet =
192.168.4.0/24Subnet = 192.168.6.0/24Subnet = 192.168.20.3/32
# The public key generated by `tincd -n example -K' is stored here-----BEGIN
RSA PUBLIC KEY----------END RSA PUBLIC KEY-----

ROUTE TABLE on AKernel IP routing tableDestination     Gateway         Genmask 
       Flags Metric Ref    Use Ifacedefault         192.168.4.1         
0.0.0.0           UG    100    0        0 enp1s0link-local      *               
           255.255.0.0     U     1000   0        0 enp1s0192.168.4.0     *     
                 255.255.255.0   U     100    0        0 enp1s0192.168.6.0   
 192.168.4.99    255.255.255.0   UG    0      0        0 enp1s0192.168.14.0   
192.168.20.3   255.255.255.0   UG    0      0        0 tun0192.168.20.0    *   
                  255.255.255.0   U     0      0        0 tun0

The Net, 192.168.20.0 is one for TINC itself, where 192.168.20.3 is
A, 192.168.20.2 is B and 192.168.20.1 is C
And I explicitly static route to it. (Doing it the way shown in other examples
has same issue)Net 192.168.14.0 is the C local network
Net 192.168.4.0 is the A local network (Net 192.168.6.0 is via another router
with WAN IP of 192.168.4.99
IP of A is 192.168.4.30, IP of C is 192.168.14.20
Only thing wrong is, for exampleOn A, ping 192.168.14.60 does not work
On C, ping 192.168.4.26 does not work
But on clients 192.168.14.60 and 192.168.4.26 can ping each other.

All firewalls are off, and iptables flushed

Very puzzling!!
John

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20180331/6fbf9101/attachment.html>

On Sat, Mar 31, 2018 at 12:00:57PM +0000, John Radley (yahoo) wrote:
> I have a three tinc server setup, similar to "4.3 How Connections
> Work" using the configuration mostly like
> http://ostolc.org/site-to-site-vpn-with-tinc.html
> 
> The clients (Ubuntus, Debians and Windows 10s) can all ping (and SSH)
> to each other remotely. As far as that is concerned it's working great
> - thanks so much for some great software.
> 
> However, on each of the Tinc servers (A and C) neither of them can
> ping other remote clients. Of course, A and C can ping each other. If
> I use tcpdump -nni tun0 icmpI can see the echo packets leave the
> server, and on a remote client see the request received and the reply
> sent. However the server never gets the reply. It seems that on each
> server there is no internal routing between enp1s0 and tun0 for IPs
> that are not server IPs. I guess I can live with such a limitation,
> but would still like to know why!!
Tinc itself doesn't take of that routing outside of the VPN itself, so
it is up to you to configure it correctly.
> TINC-UP
> ip link set $INTERFACE up
> ip addr add 192.168.20.3/24 dev $INTERFACE
> route add -net 192.168.14.0/24 gw 192.168.20.3
> route add -net 192.168.6.0/24  gw 192.168.4.99
First, if you are already using "ip" to assign an address, then
instead of the "route" command, use the "ip route" command
to configure extra routes, like so:

    ip route add 192.168.14.0/24 via 192.168.20.3
    ip route add 192.168.6.0/24 via 192.168.4.99

Note that the first route command is equivalent to:

    ip route add 192.168.14.0/24 dev $INTERFACE
> ROUTE TABLE on A
> Kernel IP routing table
> Destination     Gateway         Genmask         Flags  Metric Ref Use Iface
> default         192.168.4.1     0.0.0.0         UG     100    0   0  
enp1s0
> link-local      *               255.255.0.0     U      1000   0   0  
enp1s0
> 192.168.4.0     *               255.255.255.0   U      100    0   0  
enp1s0
> 192.168.6.0     192.168.4.99    255.255.255.0   UG     0      0   0  
enp1s0
> 192.168.14.0    192.168.20.3    255.255.255.0   UG     0      0   0   tun0
> 192.168.20.0    *               255.255.255.0   U      0      0   0   tun0
[...]
> Net 192.168.4.0 is the A local network
> IP of A is 192.168.4.30, IP of C is 192.168.14.20
[...]
> Only thing wrong is, for example on A, ping 192.168.14.60 does not work
> On C, ping 192.168.4.26 does not work
The problem is most likely with the hosts 192.168.14.60 and
192.168.4.26. What does their routing table look like? If 192.168.4.26
just has:

    Destination     Gateway         Genmask         Flags  Metric Ref Use Iface
    default         192.168.4.1     0.0.0.0         UG     100    0   0   enp1s0
    link-local      *               255.255.0.0     U      1000   0   0   enp1s0
    192.168.4.0     *               255.255.255.0   U      100    0   0   enp1s0

Then packets for 192.168.20.* or 192.168.14.* will go the the default
gateway 192.168.4.1, and will not go to your host running tinc. A ping
packet from C might reach host 192.168.14.26, but that host will send
the return packet in the wrong direction.

To fix this, you must either add a route that looks like this to each
host on A:

    192.168.14.0    192.168.4.30    255.255.255.0   UG     0      0   0   enp1s0

Or you have to tell the router (192.168.4.1) that packets for
192.168.14.0/24 should be forwarded to 192.168.4.30. And you have to do
something similar on the other sites.
> But on clients 192.168.14.60 and 192.168.4.26 can ping each other.
Ok, that is weird... if those can ping each other, they should both be
able to ping A and C as well.

-- 
Met vriendelijke groet / with kind regards,
     Guus Sliepen <guus at tinc-vpn.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20180401/3aab2442/attachment-0001.sig>

Hello List,

I have this setup:

hosta  <--> hostb  <-->  hostc

Hosta and hostc are not directly connected via tinc. But both are 
conncted via hostb (I called my network tincnet). This works fine I can 
ssh from hosta to hostc and vice versa without any problems.

hostc is in a whitelisted iprange at some service provider.

I need hosta to talk to a certain ip (lets call it ipaddressx) via hostc.

I added the iptables mangle rule to mark all traffic to ipaddressx at 
port 700.

-A OUTPUT  -p 6 -m tcp -d ipaddressx/255.255.255.255 --dport 700 -j MARK 
--set-mark 0x1

I added:
     ip route add default via iphostc dev tincnet table hostc
     ip rule add from 0.0.0.0/0 fwmark 1 table hostc

Now when I try this:

traceroute -T -n ipaddressx -p 700

The route goes via the ip of hostb and not via the ip of hostc as I 
would have expected.
If I remove the iptables rule the route goes directly via the ip of 
hosta. So the mangle rule and ip rule lines are okay I think.
Of course I also checked this via telnet ipaddressx 700 and watched via 
tcpdump what happened on hostb and hostc.

A weird thing is when I try the add route with any ip in the tincnet 
subnet the route gets added even if that ip is not in use and all 
traffic still goes via the ip of hostb.
ie: ip route add default via any_ip_in_the_tincnet_subnet dev tincnet 
table hostc

Does any one know what is happening here?

Is it tincd at hostb that intercepts the traffic actually meant for 
hostc and thinks it's meant for hostb and rewrites stuff automaticaly?  
Or am I missing something in the ip route / ip rules part?

I am using tinc a lot but so far it was between tinc nodes that are also 
directly connected. and never had this problem before.

If I just use iptables on hosta and hostc with nat en prerouting it 
works fine. I just tell iptables on hosta that all traffic to ipaddressx 
has to be dnatted to hostc and at hostc I just dnat this to the 
destination ip.

But I really would like to understand how to do this via mangle/fwmark 
and ip route  / ip rule way.

hosta is centos 7 tinc 1.0.31
hostb is centos 5 tinc 1.0.25
hostc is centos 5 tinc 1.0.13

I hope someone can help me on my way.

Thx

Hans de Groot

Can you post details of your tinc configuration? Especially the Mode (is it
"switch" or "router"?) and the DeviceType (is it
"tun" or "tap"?).

If operating in "router" Mode, does hostc have a "Subnet =
ipaddressx" in
its host configuration file? That would be required for things to work you
expect them to. (Also, does hostb have a Subnet that encompasses
ipaddressx? That would explain why packets are being misrouted, not just
dropped.)

On 10 April 2018 at 14:36, Hans de Groot <hansg at dandy.nl> wrote:
> Hello List,
>
> I have this setup:
>
> hosta  <--> hostb  <-->  hostc
>
> Hosta and hostc are not directly connected via tinc. But both are conncted
> via hostb (I called my network tincnet). This works fine I can ssh from
> hosta to hostc and vice versa without any problems.
>
> hostc is in a whitelisted iprange at some service provider.
>
> I need hosta to talk to a certain ip (lets call it ipaddressx) via hostc.
>
> I added the iptables mangle rule to mark all traffic to ipaddressx at port
> 700.
>
> -A OUTPUT  -p 6 -m tcp -d ipaddressx/255.255.255.255 --dport 700 -j MARK
> --set-mark 0x1
>
> I added:
>     ip route add default via iphostc dev tincnet table hostc
>     ip rule add from 0.0.0.0/0 fwmark 1 table hostc
>
> Now when I try this:
>
> traceroute -T -n ipaddressx -p 700
>
> The route goes via the ip of hostb and not via the ip of hostc as I would
> have expected.
> If I remove the iptables rule the route goes directly via the ip of hosta.
> So the mangle rule and ip rule lines are okay I think.
> Of course I also checked this via telnet ipaddressx 700 and watched via
> tcpdump what happened on hostb and hostc.
>
> A weird thing is when I try the add route with any ip in the tincnet
> subnet the route gets added even if that ip is not in use and all traffic
> still goes via the ip of hostb.
> ie: ip route add default via any_ip_in_the_tincnet_subnet dev tincnet
> table hostc
>
> Does any one know what is happening here?
>
> Is it tincd at hostb that intercepts the traffic actually meant for hostc
> and thinks it's meant for hostb and rewrites stuff automaticaly?  Or am
I
> missing something in the ip route / ip rules part?
>
> I am using tinc a lot but so far it was between tinc nodes that are also
> directly connected. and never had this problem before.
>
> If I just use iptables on hosta and hostc with nat en prerouting it works
> fine. I just tell iptables on hosta that all traffic to ipaddressx has to
> be dnatted to hostc and at hostc I just dnat this to the destination ip.
>
> But I really would like to understand how to do this via mangle/fwmark and
> ip route  / ip rule way.
>
> hosta is centos 7 tinc 1.0.31
> hostb is centos 5 tinc 1.0.25
> hostc is centos 5 tinc 1.0.13
>
> I hope someone can help me on my way.
>
> Thx
>
> Hans de Groot
>
>
>
>
>
>
>
>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20180410/e90567ff/attachment.html>

On Tue, Apr 10, 2018 at 03:36:08PM +0200, Hans de Groot wrote:
> hosta  <--> hostb  <-->  hostc
> 
> Hosta and hostc are not directly connected via tinc. But both are conncted
> via hostb (I called my network tincnet). This works fine I can ssh from
> hosta to hostc and vice versa without any problems.
> 
> hostc is in a whitelisted iprange at some service provider.
> 
> I need hosta to talk to a certain ip (lets call it ipaddressx) via hostc.
> 
> I added the iptables mangle rule to mark all traffic to ipaddressx at port
> 700.
> 
> -A OUTPUT  -p 6 -m tcp -d ipaddressx/255.255.255.255 --dport 700 -j MARK
> --set-mark 0x1
> 
> I added:
>     ip route add default via iphostc dev tincnet table hostc
>     ip rule add from 0.0.0.0/0 fwmark 1 table hostc
If you are running tinc in router mode (which is the default), then the
"via iphostc" option does not have any effect. The packets will go to
dev tincnet, but there is nothing in the header of IP packets that
contains the address of the gateway.

Also, if you are using router mode, then you must inform tinc about
which peer to send packets with destination address ipaddressx to. So
you can add "Subnet = ipaddressx/" to hostc's hosts/hostc. But
tinc only
routes on address, not on ports.
> Now when I try this:
> 
> traceroute -T -n ipaddressx -p 700
> 
> The route goes via the ip of hostb and not via the ip of hostc as I would
> have expected.
There are two possibilities for this: one is that tinc thinks the
packets with destination address ipaddressx should go to hostb (because
of what is in the Subnet statements), the other is that hosta and hostc
cannot directly communicate with each other, and traffic is routed via
hostb, and you have Forwarding = kernel in your tinc.conf. The latter
will force the packets to be sent to the tun interface on hostb, and if
you don't have any rules on hostb to send packets for ipaddressx port
700 to hostc, they will not be forwarded the way you want.
> A weird thing is when I try the add route with any ip in the tincnet subnet
> the route gets added even if that ip is not in use and all traffic still
> goes via the ip of hostb.
> ie: ip route add default via any_ip_in_the_tincnet_subnet dev tincnet table
> hostc
Again, in router mode, the "via ..." option has no effect at all.
> Is it tincd at hostb that intercepts the traffic actually meant for hostc
> and thinks it's meant for hostb and rewrites stuff automaticaly?  Or am
I
> missing something in the ip route / ip rules part?
Tinc itself does not rewrite anything.
> But I really would like to understand how to do this via mangle/fwmark and
> ip route  / ip rule way.
It would help if you could show us your tinc.conf from hostb, and all
hosts/* files, so we can check how you configured tinc exactly.

-- 
Met vriendelijke groet / with kind regards,
     Guus Sliepen <guus at tinc-vpn.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20180410/1b4d8d26/attachment.sig>

Hi,

I had set this 2 hop vpn use tinc.

The thing is make sure you can reach the ip of host c from host a, seems it
works as you can ssh.

And the nat you have turn on in hostc and ip forward enabled in kernel of
hostb and hostc.

If your ipadreesx dest will be a http,  better use a proxy server,  set in
hostc. Not using routing or nat thing. It works for me and faster with
cache.



On Tue, 10 Apr 2018 21:56 Hans de Groot, <hansg at dandy.nl> wrote:
> Hello List,
>
> I have this setup:
>
> hosta  <--> hostb  <-->  hostc
>
> Hosta and hostc are not directly connected via tinc. But both are
> conncted via hostb (I called my network tincnet). This works fine I can
> ssh from hosta to hostc and vice versa without any problems.
>
> hostc is in a whitelisted iprange at some service provider.
>
> I need hosta to talk to a certain ip (lets call it ipaddressx) via hostc.
>
> I added the iptables mangle rule to mark all traffic to ipaddressx at
> port 700.
>
> -A OUTPUT  -p 6 -m tcp -d ipaddressx/255.255.255.255 --dport 700 -j MARK
> --set-mark 0x1
>
> I added:
>      ip route add default via iphostc dev tincnet table hostc
>      ip rule add from 0.0.0.0/0 fwmark 1 table hostc
>
> Now when I try this:
>
> traceroute -T -n ipaddressx -p 700
>
> The route goes via the ip of hostb and not via the ip of hostc as I
> would have expected.
> If I remove the iptables rule the route goes directly via the ip of
> hosta. So the mangle rule and ip rule lines are okay I think.
> Of course I also checked this via telnet ipaddressx 700 and watched via
> tcpdump what happened on hostb and hostc.
>
> A weird thing is when I try the add route with any ip in the tincnet
> subnet the route gets added even if that ip is not in use and all
> traffic still goes via the ip of hostb.
> ie: ip route add default via any_ip_in_the_tincnet_subnet dev tincnet
> table hostc
>
> Does any one know what is happening here?
>
> Is it tincd at hostb that intercepts the traffic actually meant for
> hostc and thinks it's meant for hostb and rewrites stuff automaticaly?
> Or am I missing something in the ip route / ip rules part?
>
> I am using tinc a lot but so far it was between tinc nodes that are also
> directly connected. and never had this problem before.
>
> If I just use iptables on hosta and hostc with nat en prerouting it
> works fine. I just tell iptables on hosta that all traffic to ipaddressx
> has to be dnatted to hostc and at hostc I just dnat this to the
> destination ip.
>
> But I really would like to understand how to do this via mangle/fwmark
> and ip route  / ip rule way.
>
> hosta is centos 7 tinc 1.0.31
> hostb is centos 5 tinc 1.0.25
> hostc is centos 5 tinc 1.0.13
>
> I hope someone can help me on my way.
>
> Thx
>
> Hans de Groot
>
>
>
>
>
>
>
>
> _______________________________________________
> tinc mailing list
> tinc at tinc-vpn.org
> https://www.tinc-vpn.org/cgi-bin/mailman/listinfo/tinc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://www.tinc-vpn.org/pipermail/tinc/attachments/20180411/8b92549e/attachment-0001.html>