thr3ads.net - LARTC - routing TCP to another box preserving ORIGINAL client IPs [Mar 2007]

If this information is useful, please help other people find it:
Share via:

Alec Matusis

2007-Mar-08 04:07 UTC

routing TCP to another box preserving ORIGINAL client IPs

My TCP clients connect to box A. I need to forward those connections to a
server on box B, such that the original client IPs are visible to the server
on B.

Each box has two Ethernet ports. One port on each box is connected to WAN,
and they are cross-connected in a LAN via remaining ports:

        -------------------           -------------------
WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
        -------------------           -------------------


Is there a way to do this with iproute2 and iptables tools ONLY? Can you
provide an example? Nothing in Google after more than a week of searching.
An additional requirement is to reduce the load on box A as much as possible
(I guess the server on B would still have to reply to the client via A, not
using B''s own WAN interface however..)

Martin A. Brown

2007-Mar-08 04:23 UTC

head link

Re: routing TCP to another box preserving ORIGINAL client IPs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings Alec,

 : My TCP clients connect to box A. I need to forward those 
 : connections to a server on box B, such that the original client 
 : IPs are visible to the server on B.
 : 
 : Each box has two Ethernet ports. One port on each box is 
 : connected to WAN, and they are cross-connected in a LAN via 
 : remaining ports:
 : 
 :         -------------------           -------------------
 : WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
 :         -------------------           -------------------
 : 
 : 
 : Is there a way to do this with iproute2 and iptables tools ONLY? 
 : Can you provide an example? Nothing in Google after more than a 
 : week of searching. An additional requirement is to reduce the 
 : load on box A as much as possible (I guess the server on B would 
 : still have to reply to the client via A, not using B''s own WAN 
 : interface however..)

You need to provide us a bit more information to help you figure out 
the right way to solve this problem.  Why is DNAT not sufficient?  
Given your description, you should simply be able to:

  iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1

If it were that simple, though, you shouldn''t have spent a week 
looking for the answer.

Do you have a TCP service on Box A which is providing services to 
the client across the WAN?  If so, then you are looking for 
something called transparent proxying in the Linux networking world.  
You will want to examine the tproxy patches to iptables [0].

If you go with the transparent proxying method, it''s helpful to 
remember:

  * the client thinks it''s connected to Box A
  * Box A knows its connected to the client
  * Box A uses client''s source address to initiate traffic to Box B
  * Box B thinks it''s connected to client

In either case, you are correct about routing.  Box B must send its 
traffic back to Box A to forward back across the LAN.

Good luck,

- -Martin

 [0] http://www.balabit.com/products/oss/tproxy/
     http://www.balabit.com/downloads/tproxy/linux-2.4/

- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
5Q7zrssGkA02Pq+298i9tEA=O3sf
-----END PGP SIGNATURE-----

Alec Matusis

2007-Mar-08 04:46 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

Hello Martin,

I tried implementing DNAT as you indicated:

iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1

After that, I can see SYN packets arriving on BOX_B_ETH1, having the
original client''s IP. Only half of the connection gets established
after
this:
I cannot see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor on
BOX_A_ETH0). I think the reason is that since box A never sends a SYN packet
itself, it never classifies the connection as ESTABLISHED, so all further
traffic gets rejected. It''s still a mystery to me what happens to SYN
packets from be in this scenario however.

It turns out that I have to supplement DNAT with SNAT for this to work
correctly.
On box A:
iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
--to-destination $BOX_B_ETH1
iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
$SERVER_PORT -j SNAT --to-source $BOX_A_ETH1

in this case, the clients can connect, however the server on B sees only IP
of BOX_A_ETH1, not the original client IPs.

Regarding tproxy:
I am currently running the TCP server on box A. I would like to move it to
box B to reduce the load on A. Other services on A are bound to the same IP
address as the server that I need to move, so simply moving that IP address
to BOX_B_ETH0 is impossible.
Box A has 2.6.11 kernel. Does tproxy install over netfilter? I am sort of
scared to tamper with netfilter installation on A, since it is currently
running live services.

It it possible to implement this scenario:
  * the client thinks it''s connected to Box A
  * Box A knows its connected to the client
  * Box A uses client''s source address to initiate traffic to Box B
  * Box B thinks it''s connected to client
with advanced routing and iptables only?

Thanks

Alec. 



-----Original Message-----
From: Martin A. Brown [mailto:martin@linux-ip.net] 
Sent: Wednesday, March 07, 2007 8:24 PM
To: Alec Matusis
Cc: lartc@mailman.ds9a.nl
Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings Alec,

 : My TCP clients connect to box A. I need to forward those 
 : connections to a server on box B, such that the original client 
 : IPs are visible to the server on B.
 : 
 : Each box has two Ethernet ports. One port on each box is 
 : connected to WAN, and they are cross-connected in a LAN via 
 : remaining ports:
 : 
 :         -------------------           -------------------
 : WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
 :         -------------------           -------------------
 : 
 : 
 : Is there a way to do this with iproute2 and iptables tools ONLY? 
 : Can you provide an example? Nothing in Google after more than a 
 : week of searching. An additional requirement is to reduce the 
 : load on box A as much as possible (I guess the server on B would 
 : still have to reply to the client via A, not using B''s own WAN 
 : interface however..)

You need to provide us a bit more information to help you figure out 
the right way to solve this problem.  Why is DNAT not sufficient?  
Given your description, you should simply be able to:

  iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1

If it were that simple, though, you shouldn''t have spent a week 
looking for the answer.

Do you have a TCP service on Box A which is providing services to 
the client across the WAN?  If so, then you are looking for 
something called transparent proxying in the Linux networking world.  
You will want to examine the tproxy patches to iptables [0].

If you go with the transparent proxying method, it''s helpful to 
remember:

  * the client thinks it''s connected to Box A
  * Box A knows its connected to the client
  * Box A uses client''s source address to initiate traffic to Box B
  * Box B thinks it''s connected to client

In either case, you are correct about routing.  Box B must send its 
traffic back to Box A to forward back across the LAN.

Good luck,

- -Martin

 [0] http://www.balabit.com/products/oss/tproxy/
     http://www.balabit.com/downloads/tproxy/linux-2.4/

- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
5Q7zrssGkA02Pq+298i9tEA=O3sf
-----END PGP SIGNATURE-----

Alec Matusis

2007-Mar-09 00:05 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

I have not considered bridge and ebtables yet. I assume I''d have to
implement this in the "routing" box A. Is ebtables CPU-intensive? The
goal
of setting up this forwarding is to reduce the load on the router box A.

Also, is it very surprising that there is no way to build a transparent
proxy/router with standard iproute2 and iptables tools ONLY? It would seem
that transparent forwarding of TCP connections to another machine is a very
common task.


From: Robb Bossley [mailto:robb.bossley@gmail.com] 
Sent: Thursday, March 08, 2007 9:13 AM
To: Alec Matusis
Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

Have you considered putting a bridge in and using ebtables?  That might be
what you are looking for.


On 3/7/07, Alec Matusis < alecm at chatango.com> wrote:
Hello Martin,

I tried implementing DNAT as you indicated: 

iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1

After that, I can see SYN packets arriving on BOX_B_ETH1, having the
original client''s IP. Only half of the connection gets established
after
this:
I cannot see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor on
BOX_A_ETH0). I think the reason is that since box A never sends a SYN packet
itself, it never classifies the connection as ESTABLISHED, so all further 
traffic gets rejected. It''s still a mystery to me what happens to SYN
packets from be in this scenario however.

It turns out that I have to supplement DNAT with SNAT for this to work
correctly.
On box A: 
iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
--to-destination $BOX_B_ETH1
iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
$SERVER_PORT -j SNAT --to-source $BOX_A_ETH1 

in this case, the clients can connect, however the server on B sees only IP
of BOX_A_ETH1, not the original client IPs.

Regarding tproxy:
I am currently running the TCP server on box A. I would like to move it to 
box B to reduce the load on A. Other services on A are bound to the same IP
address as the server that I need to move, so simply moving that IP address
to BOX_B_ETH0 is impossible.
Box A has 2.6.11 kernel. Does tproxy install over netfilter? I am sort of 
scared to tamper with netfilter installation on A, since it is currently
running live services.

It it possible to implement this scenario:
  * the client thinks it''s connected to Box A
  * Box A knows its connected to the client 
  * Box A uses client''s source address to initiate traffic to Box B
  * Box B thinks it''s connected to client
with advanced routing and iptables only?

Thanks

Alec.



-----Original Message----- 
From: Martin A. Brown [mailto:martin@linux-ip.net]
Sent: Wednesday, March 07, 2007 8:24 PM
To: Alec Matusis
Cc: lartc@mailman.ds9a.nl 
Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Greetings Alec,

: My TCP clients connect to box A. I need to forward those 
: connections to a server on box B, such that the original client
: IPs are visible to the server on B.
:
: Each box has two Ethernet ports. One port on each box is
: connected to WAN, and they are cross-connected in a LAN via 
: remaining ports:
:
:         -------------------           -------------------
: WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
:         -------------------           ------------------- 
:
:
: Is there a way to do this with iproute2 and iptables tools ONLY?
: Can you provide an example? Nothing in Google after more than a
: week of searching. An additional requirement is to reduce the 
: load on box A as much as possible (I guess the server on B would
: still have to reply to the client via A, not using B''s own WAN
: interface however..)

You need to provide us a bit more information to help you figure out 
the right way to solve this problem.  Why is DNAT not sufficient?
Given your description, you should simply be able to:

  iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1

If it were that simple, though, you shouldn''t have spent a week 
looking for the answer.

Do you have a TCP service on Box A which is providing services to
the client across the WAN?  If so, then you are looking for
something called transparent proxying in the Linux networking world. 
You will want to examine the tproxy patches to iptables [0].

If you go with the transparent proxying method, it''s helpful to
remember:

  * the client thinks it''s connected to Box A
  * Box A knows its connected to the client 
  * Box A uses client''s source address to initiate traffic to Box B
  * Box B thinks it''s connected to client

In either case, you are correct about routing.  Box B must send its
traffic back to Box A to forward back across the LAN. 

Good luck,

- -Martin

[0] http://www.balabit.com/products/oss/tproxy/
     http://www.balabit.com/downloads/tproxy/linux-2.4/ 

- --
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 ( http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
5Q7zrssGkA02Pq+298i9tEA=O3sf
-----END PGP SIGNATURE-----

_______________________________________________ 
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

ArcosCom Linux User

2007-Mar-09 00:48 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

Are you sure?
Take a look into:
http://tldp.org/HOWTO/TransparentProxy-6.html

Regards

El Vie, 9 de Marzo de 2007, 1:05, Alec Matusis escribió:> I have not considered bridge and ebtables yet. I assume I''d have
to
> implement this in the "routing" box A. Is ebtables CPU-intensive?
The goal
> of setting up this forwarding is to reduce the load on the router box A.
>
> Also, is it very surprising that there is no way to build a transparent
> proxy/router with standard iproute2 and iptables tools ONLY? It would seem
> that transparent forwarding of TCP connections to another machine is a
> very
> common task.
>
>
> From: Robb Bossley [mailto:robb.bossley@gmail.com]
> Sent: Thursday, March 08, 2007 9:13 AM
> To: Alec Matusis
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> Have you considered putting a bridge in and using ebtables?  That might be
> what you are looking for.
>
>
> On 3/7/07, Alec Matusis < alecm at chatango.com> wrote:
> Hello Martin,
>
> I tried implementing DNAT as you indicated:
>
> iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> After that, I can see SYN packets arriving on BOX_B_ETH1, having the
> original client''s IP. Only half of the connection gets established
after
> this:
> I cannot see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor
> on
> BOX_A_ETH0). I think the reason is that since box A never sends a SYN
> packet
> itself, it never classifies the connection as ESTABLISHED, so all further
> traffic gets rejected. It''s still a mystery to me what happens to
SYN
> packets from be in this scenario however.
>
> It turns out that I have to supplement DNAT with SNAT for this to work
> correctly.
> On box A:
> iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
> --to-destination $BOX_B_ETH1
> iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
> $SERVER_PORT -j SNAT --to-source $BOX_A_ETH1
>
> in this case, the clients can connect, however the server on B sees only
> IP
> of BOX_A_ETH1, not the original client IPs.
>
> Regarding tproxy:
> I am currently running the TCP server on box A. I would like to move it to
> box B to reduce the load on A. Other services on A are bound to the same
> IP
> address as the server that I need to move, so simply moving that IP
> address
> to BOX_B_ETH0 is impossible.
> Box A has 2.6.11 kernel. Does tproxy install over netfilter? I am sort of
> scared to tamper with netfilter installation on A, since it is currently
> running live services.
>
> It it possible to implement this scenario:
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
> with advanced routing and iptables only?
>
> Thanks
>
> Alec.
>
>
>
> -----Original Message-----
> From: Martin A. Brown [mailto:martin@linux-ip.net]
> Sent: Wednesday, March 07, 2007 8:24 PM
> To: Alec Matusis
> Cc: lartc@mailman.ds9a.nl
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Greetings Alec,
>
> : My TCP clients connect to box A. I need to forward those
> : connections to a server on box B, such that the original client
> : IPs are visible to the server on B.
> :
> : Each box has two Ethernet ports. One port on each box is
> : connected to WAN, and they are cross-connected in a LAN via
> : remaining ports:
> :
> :         -------------------           -------------------
> : WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
> :         -------------------           -------------------
> :
> :
> : Is there a way to do this with iproute2 and iptables tools ONLY?
> : Can you provide an example? Nothing in Google after more than a
> : week of searching. An additional requirement is to reduce the
> : load on box A as much as possible (I guess the server on B would
> : still have to reply to the client via A, not using B''s own WAN
> : interface however..)
>
> You need to provide us a bit more information to help you figure out
> the right way to solve this problem.  Why is DNAT not sufficient?
> Given your description, you should simply be able to:
>
>   iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> If it were that simple, though, you shouldn''t have spent a week
> looking for the answer.
>
> Do you have a TCP service on Box A which is providing services to
> the client across the WAN?  If so, then you are looking for
> something called transparent proxying in the Linux networking world.
> You will want to examine the tproxy patches to iptables [0].
>
> If you go with the transparent proxying method, it''s helpful to
> remember:
>
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
>
> In either case, you are correct about routing.  Box B must send its
> traffic back to Box A to forward back across the LAN.
>
> Good luck,
>
> - -Martin
>
> [0] http://www.balabit.com/products/oss/tproxy/
>      http://www.balabit.com/downloads/tproxy/linux-2.4/
>
> - --
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 ( http://linux-ip.net/sw/pine-gpg-filter/)
>
> iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
> 5Q7zrssGkA02Pq+298i9tEA> =O3sf
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>

Alec Matusis

2007-Mar-09 03:13 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

I tried method 2 multiple times for 2 weeks, and it does not work. Perhaps
someone can tell me what the problem is? Maybe I am missing some kernel
option? How do I check that?

Router box A (wan ip x.x.x.83  on eth0) :

# iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       tcp  --  0.0.0.0/0            x.x.x.83        tcp dpt:5224 MARK
set 0x3 

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination 

#iptables -t filter -L -n
Chain INPUT (policy ACCEPT)
...

# ip ru l
0:      from all lookup local 
32765:  from all fwmark 0x3 lookup 2 
32766:  from all lookup main 
32767:  from all lookup default

# ip ro l t 2
default via 10.18.1.1 dev eth1

# cat /proc/sys/net/ipv4/ip_forward
1

checking a few things from A:
# telnet 10.18.1.1 5224
Trying 10.18.1.1...
Connected to 10.18.1.1 (10.18.1.1).
Escape character is ''^]''.

--success. It means that box B is ready to accept connections on 5224 via
LAN (it''s INPUT chain''s policy in filter is ACCEPT for
simplicity).
>From an outside clientOutside client /> telnet x.x.x.83 5224
Trying x.x.x.83 ...
telnet: connect to address x.x.x.83: Connection refused

This suggests that the problem lies within box A.




-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl]
On Behalf Of ArcosCom Linux User
Sent: Thursday, March 08, 2007 4:49 PM
To: lartc@mailman.ds9a.nl
Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

Are you sure?
Take a look into:
http://tldp.org/HOWTO/TransparentProxy-6.html

Regards

El Vie, 9 de Marzo de 2007, 1:05, Alec Matusis escribió:> I have not considered bridge and ebtables yet. I assume I''d have
to
> implement this in the "routing" box A. Is ebtables CPU-intensive?
The goal
> of setting up this forwarding is to reduce the load on the router box A.
>
> Also, is it very surprising that there is no way to build a transparent
> proxy/router with standard iproute2 and iptables tools ONLY? It would seem
> that transparent forwarding of TCP connections to another machine is a
> very
> common task.
>
>
> From: Robb Bossley [mailto:robb.bossley@gmail.com]
> Sent: Thursday, March 08, 2007 9:13 AM
> To: Alec Matusis
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> Have you considered putting a bridge in and using ebtables?  That might be
> what you are looking for.
>
>
> On 3/7/07, Alec Matusis < alecm at chatango.com> wrote:
> Hello Martin,
>
> I tried implementing DNAT as you indicated:
>
> iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> After that, I can see SYN packets arriving on BOX_B_ETH1, having the
> original client''s IP. Only half of the connection gets established
after
> this:
> I cannot see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor
> on
> BOX_A_ETH0). I think the reason is that since box A never sends a SYN
> packet
> itself, it never classifies the connection as ESTABLISHED, so all further
> traffic gets rejected. It''s still a mystery to me what happens to
SYN
> packets from be in this scenario however.
>
> It turns out that I have to supplement DNAT with SNAT for this to work
> correctly.
> On box A:
> iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
> --to-destination $BOX_B_ETH1
> iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
> $SERVER_PORT -j SNAT --to-source $BOX_A_ETH1
>
> in this case, the clients can connect, however the server on B sees only
> IP
> of BOX_A_ETH1, not the original client IPs.
>
> Regarding tproxy:
> I am currently running the TCP server on box A. I would like to move it to
> box B to reduce the load on A. Other services on A are bound to the same
> IP
> address as the server that I need to move, so simply moving that IP
> address
> to BOX_B_ETH0 is impossible.
> Box A has 2.6.11 kernel. Does tproxy install over netfilter? I am sort of
> scared to tamper with netfilter installation on A, since it is currently
> running live services.
>
> It it possible to implement this scenario:
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
> with advanced routing and iptables only?
>
> Thanks
>
> Alec.
>
>
>
> -----Original Message-----
> From: Martin A. Brown [mailto:martin@linux-ip.net]
> Sent: Wednesday, March 07, 2007 8:24 PM
> To: Alec Matusis
> Cc: lartc@mailman.ds9a.nl
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Greetings Alec,
>
> : My TCP clients connect to box A. I need to forward those
> : connections to a server on box B, such that the original client
> : IPs are visible to the server on B.
> :
> : Each box has two Ethernet ports. One port on each box is
> : connected to WAN, and they are cross-connected in a LAN via
> : remaining ports:
> :
> :         -------------------           -------------------
> : WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
> :         -------------------           -------------------
> :
> :
> : Is there a way to do this with iproute2 and iptables tools ONLY?
> : Can you provide an example? Nothing in Google after more than a
> : week of searching. An additional requirement is to reduce the
> : load on box A as much as possible (I guess the server on B would
> : still have to reply to the client via A, not using B''s own WAN
> : interface however..)
>
> You need to provide us a bit more information to help you figure out
> the right way to solve this problem.  Why is DNAT not sufficient?
> Given your description, you should simply be able to:
>
>   iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> If it were that simple, though, you shouldn''t have spent a week
> looking for the answer.
>
> Do you have a TCP service on Box A which is providing services to
> the client across the WAN?  If so, then you are looking for
> something called transparent proxying in the Linux networking world.
> You will want to examine the tproxy patches to iptables [0].
>
> If you go with the transparent proxying method, it''s helpful to
> remember:
>
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
>
> In either case, you are correct about routing.  Box B must send its
> traffic back to Box A to forward back across the LAN.
>
> Good luck,
>
> - -Martin
>
> [0] http://www.balabit.com/products/oss/tproxy/
>      http://www.balabit.com/downloads/tproxy/linux-2.4/
>
> - --
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 ( http://linux-ip.net/sw/pine-gpg-filter/)
>
> iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
> 5Q7zrssGkA02Pq+298i9tEA> =O3sf
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Rangi Biddle

2007-Mar-09 04:31 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

Hi Alex,

Perhaps another test to see if it is really working is to connect to the
public address (x.x.x.83) from box A.  The other thing perhaps you might
want to check is if there is a firewall running in between that is blocking
5224.

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl]
On Behalf Of Alec Matusis
Sent: Friday, March 09, 2007 4:13 PM
To: linux@arcoscom.com; lartc@mailman.ds9a.nl
Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

I tried method 2 multiple times for 2 weeks, and it does not work. Perhaps
someone can tell me what the problem is? Maybe I am missing some kernel
option? How do I check that?

Router box A (wan ip x.x.x.83  on eth0) :

# iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       tcp  --  0.0.0.0/0            x.x.x.83        tcp dpt:5224 MARK
set 0x3 

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination 

#iptables -t filter -L -n
Chain INPUT (policy ACCEPT)
...

# ip ru l
0:      from all lookup local 
32765:  from all fwmark 0x3 lookup 2 
32766:  from all lookup main 
32767:  from all lookup default

# ip ro l t 2
default via 10.18.1.1 dev eth1

# cat /proc/sys/net/ipv4/ip_forward
1

checking a few things from A:
# telnet 10.18.1.1 5224
Trying 10.18.1.1...
Connected to 10.18.1.1 (10.18.1.1).
Escape character is ''^]''.

--success. It means that box B is ready to accept connections on 5224 via
LAN (it''s INPUT chain''s policy in filter is ACCEPT for
simplicity).
>From an outside clientOutside client /> telnet x.x.x.83 5224
Trying x.x.x.83 ...
telnet: connect to address x.x.x.83: Connection refused

This suggests that the problem lies within box A.




-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl]
On Behalf Of ArcosCom Linux User
Sent: Thursday, March 08, 2007 4:49 PM
To: lartc@mailman.ds9a.nl
Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

Are you sure?
Take a look into:
http://tldp.org/HOWTO/TransparentProxy-6.html

Regards

El Vie, 9 de Marzo de 2007, 1:05, Alec Matusis escribió:> I have not considered bridge and ebtables yet. I assume I''d have
to
> implement this in the "routing" box A. Is ebtables CPU-intensive?
The goal
> of setting up this forwarding is to reduce the load on the router box A.
>
> Also, is it very surprising that there is no way to build a transparent
> proxy/router with standard iproute2 and iptables tools ONLY? It would seem
> that transparent forwarding of TCP connections to another machine is a
> very
> common task.
>
>
> From: Robb Bossley [mailto:robb.bossley@gmail.com]
> Sent: Thursday, March 08, 2007 9:13 AM
> To: Alec Matusis
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> Have you considered putting a bridge in and using ebtables?  That might be
> what you are looking for.
>
>
> On 3/7/07, Alec Matusis < alecm at chatango.com> wrote:
> Hello Martin,
>
> I tried implementing DNAT as you indicated:
>
> iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> After that, I can see SYN packets arriving on BOX_B_ETH1, having the
> original client''s IP. Only half of the connection gets established
after
> this:
> I cannot see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor
> on
> BOX_A_ETH0). I think the reason is that since box A never sends a SYN
> packet
> itself, it never classifies the connection as ESTABLISHED, so all further
> traffic gets rejected. It''s still a mystery to me what happens to
SYN
> packets from be in this scenario however.
>
> It turns out that I have to supplement DNAT with SNAT for this to work
> correctly.
> On box A:
> iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
> --to-destination $BOX_B_ETH1
> iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
> $SERVER_PORT -j SNAT --to-source $BOX_A_ETH1
>
> in this case, the clients can connect, however the server on B sees only
> IP
> of BOX_A_ETH1, not the original client IPs.
>
> Regarding tproxy:
> I am currently running the TCP server on box A. I would like to move it to
> box B to reduce the load on A. Other services on A are bound to the same
> IP
> address as the server that I need to move, so simply moving that IP
> address
> to BOX_B_ETH0 is impossible.
> Box A has 2.6.11 kernel. Does tproxy install over netfilter? I am sort of
> scared to tamper with netfilter installation on A, since it is currently
> running live services.
>
> It it possible to implement this scenario:
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
> with advanced routing and iptables only?
>
> Thanks
>
> Alec.
>
>
>
> -----Original Message-----
> From: Martin A. Brown [mailto:martin@linux-ip.net]
> Sent: Wednesday, March 07, 2007 8:24 PM
> To: Alec Matusis
> Cc: lartc@mailman.ds9a.nl
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Greetings Alec,
>
> : My TCP clients connect to box A. I need to forward those
> : connections to a server on box B, such that the original client
> : IPs are visible to the server on B.
> :
> : Each box has two Ethernet ports. One port on each box is
> : connected to WAN, and they are cross-connected in a LAN via
> : remaining ports:
> :
> :         -------------------           -------------------
> : WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
> :         -------------------           -------------------
> :
> :
> : Is there a way to do this with iproute2 and iptables tools ONLY?
> : Can you provide an example? Nothing in Google after more than a
> : week of searching. An additional requirement is to reduce the
> : load on box A as much as possible (I guess the server on B would
> : still have to reply to the client via A, not using B''s own WAN
> : interface however..)
>
> You need to provide us a bit more information to help you figure out
> the right way to solve this problem.  Why is DNAT not sufficient?
> Given your description, you should simply be able to:
>
>   iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> If it were that simple, though, you shouldn''t have spent a week
> looking for the answer.
>
> Do you have a TCP service on Box A which is providing services to
> the client across the WAN?  If so, then you are looking for
> something called transparent proxying in the Linux networking world.
> You will want to examine the tproxy patches to iptables [0].
>
> If you go with the transparent proxying method, it''s helpful to
> remember:
>
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
>
> In either case, you are correct about routing.  Box B must send its
> traffic back to Box A to forward back across the LAN.
>
> Good luck,
>
> - -Martin
>
> [0] http://www.balabit.com/products/oss/tproxy/
>      http://www.balabit.com/downloads/tproxy/linux-2.4/
>
> - --
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 ( http://linux-ip.net/sw/pine-gpg-filter/)
>
> iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
> 5Q7zrssGkA02Pq+298i9tEA> =O3sf
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Alec Matusis

2007-Mar-09 04:38 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

Hi Ranghi:

The connection to the public IP x.x.x.83 of box A from box A itself fails as
well, just like from a remote client.
On A, in table filter, both INPUT and FORWARD policies are ACCEPT.
I start suspecting that some options (e.g. IP advanced router) are not
compiled into the kernel (2.6.11), but I do not know how to check that?

-----Original Message-----
From: Rangi Biddle [mailto:rangi@ngen.net.nz] 
Sent: Thursday, March 08, 2007 8:32 PM
To: ''Alec Matusis''; lartc@mailman.ds9a.nl
Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

Hi Alex,

Perhaps another test to see if it is really working is to connect to the
public address (x.x.x.83) from box A.  The other thing perhaps you might
want to check is if there is a firewall running in between that is blocking
5224.

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl]
On Behalf Of Alec Matusis
Sent: Friday, March 09, 2007 4:13 PM
To: linux@arcoscom.com; lartc@mailman.ds9a.nl
Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

I tried method 2 multiple times for 2 weeks, and it does not work. Perhaps
someone can tell me what the problem is? Maybe I am missing some kernel
option? How do I check that?

Router box A (wan ip x.x.x.83  on eth0) :

# iptables -t mangle -L -n
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
MARK       tcp  --  0.0.0.0/0            x.x.x.83        tcp dpt:5224 MARK
set 0x3 

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination 

#iptables -t filter -L -n
Chain INPUT (policy ACCEPT)
...

# ip ru l
0:      from all lookup local 
32765:  from all fwmark 0x3 lookup 2 
32766:  from all lookup main 
32767:  from all lookup default

# ip ro l t 2
default via 10.18.1.1 dev eth1

# cat /proc/sys/net/ipv4/ip_forward
1

checking a few things from A:
# telnet 10.18.1.1 5224
Trying 10.18.1.1...
Connected to 10.18.1.1 (10.18.1.1).
Escape character is ''^]''.

--success. It means that box B is ready to accept connections on 5224 via
LAN (it''s INPUT chain''s policy in filter is ACCEPT for
simplicity).
>From an outside clientOutside client /> telnet x.x.x.83 5224
Trying x.x.x.83 ...
telnet: connect to address x.x.x.83: Connection refused

This suggests that the problem lies within box A.




-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl]
On Behalf Of ArcosCom Linux User
Sent: Thursday, March 08, 2007 4:49 PM
To: lartc@mailman.ds9a.nl
Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL client
IPs

Are you sure?
Take a look into:
http://tldp.org/HOWTO/TransparentProxy-6.html

Regards

El Vie, 9 de Marzo de 2007, 1:05, Alec Matusis escribió:> I have not considered bridge and ebtables yet. I assume I''d have
to
> implement this in the "routing" box A. Is ebtables CPU-intensive?
The goal
> of setting up this forwarding is to reduce the load on the router box A.
>
> Also, is it very surprising that there is no way to build a transparent
> proxy/router with standard iproute2 and iptables tools ONLY? It would seem
> that transparent forwarding of TCP connections to another machine is a
> very
> common task.
>
>
> From: Robb Bossley [mailto:robb.bossley@gmail.com]
> Sent: Thursday, March 08, 2007 9:13 AM
> To: Alec Matusis
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> Have you considered putting a bridge in and using ebtables?  That might be
> what you are looking for.
>
>
> On 3/7/07, Alec Matusis < alecm at chatango.com> wrote:
> Hello Martin,
>
> I tried implementing DNAT as you indicated:
>
> iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> After that, I can see SYN packets arriving on BOX_B_ETH1, having the
> original client''s IP. Only half of the connection gets established
after
> this:
> I cannot see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor
> on
> BOX_A_ETH0). I think the reason is that since box A never sends a SYN
> packet
> itself, it never classifies the connection as ESTABLISHED, so all further
> traffic gets rejected. It''s still a mystery to me what happens to
SYN
> packets from be in this scenario however.
>
> It turns out that I have to supplement DNAT with SNAT for this to work
> correctly.
> On box A:
> iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
> --to-destination $BOX_B_ETH1
> iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
> $SERVER_PORT -j SNAT --to-source $BOX_A_ETH1
>
> in this case, the clients can connect, however the server on B sees only
> IP
> of BOX_A_ETH1, not the original client IPs.
>
> Regarding tproxy:
> I am currently running the TCP server on box A. I would like to move it to
> box B to reduce the load on A. Other services on A are bound to the same
> IP
> address as the server that I need to move, so simply moving that IP
> address
> to BOX_B_ETH0 is impossible.
> Box A has 2.6.11 kernel. Does tproxy install over netfilter? I am sort of
> scared to tamper with netfilter installation on A, since it is currently
> running live services.
>
> It it possible to implement this scenario:
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
> with advanced routing and iptables only?
>
> Thanks
>
> Alec.
>
>
>
> -----Original Message-----
> From: Martin A. Brown [mailto:martin@linux-ip.net]
> Sent: Wednesday, March 07, 2007 8:24 PM
> To: Alec Matusis
> Cc: lartc@mailman.ds9a.nl
> Subject: Re: [LARTC] routing TCP to another box preserving ORIGINAL client
> IPs
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Greetings Alec,
>
> : My TCP clients connect to box A. I need to forward those
> : connections to a server on box B, such that the original client
> : IPs are visible to the server on B.
> :
> : Each box has two Ethernet ports. One port on each box is
> : connected to WAN, and they are cross-connected in a LAN via
> : remaining ports:
> :
> :         -------------------           -------------------
> : WAN -- |eth0   Box A   eth1|---LAN---|eth1   Box B   eth0| -- WAN
> :         -------------------           -------------------
> :
> :
> : Is there a way to do this with iproute2 and iptables tools ONLY?
> : Can you provide an example? Nothing in Google after more than a
> : week of searching. An additional requirement is to reduce the
> : load on box A as much as possible (I guess the server on B would
> : still have to reply to the client via A, not using B''s own WAN
> : interface however..)
>
> You need to provide us a bit more information to help you figure out
> the right way to solve this problem.  Why is DNAT not sufficient?
> Given your description, you should simply be able to:
>
>   iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>
> If it were that simple, though, you shouldn''t have spent a week
> looking for the answer.
>
> Do you have a TCP service on Box A which is providing services to
> the client across the WAN?  If so, then you are looking for
> something called transparent proxying in the Linux networking world.
> You will want to examine the tproxy patches to iptables [0].
>
> If you go with the transparent proxying method, it''s helpful to
> remember:
>
>   * the client thinks it''s connected to Box A
>   * Box A knows its connected to the client
>   * Box A uses client''s source address to initiate traffic to Box
B
>   * Box B thinks it''s connected to client
>
> In either case, you are correct about routing.  Box B must send its
> traffic back to Box A to forward back across the LAN.
>
> Good luck,
>
> - -Martin
>
> [0] http://www.balabit.com/products/oss/tproxy/
>      http://www.balabit.com/downloads/tproxy/linux-2.4/
>
> - --
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 ( http://linux-ip.net/sw/pine-gpg-filter/)
>
> iD8DBQFF74/JHEoZD1iZ+YcRAlRaAJ4wf2fIc3oBJnGstjUBdpKWn1wOsQCbB2Ee
> 5Q7zrssGkA02Pq+298i9tEA> =O3sf
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Martin A. Brown

2007-Mar-09 04:39 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alec,

 : I tried implementing DNAT as you indicated:
 : 
 : iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
 : 
 : After that, I can see SYN packets arriving on BOX_B_ETH1, having 
 : the original client''s IP.

OK, that means DNAT + routing on your BOX_A is working correctly.

 : Only half of the connection gets established after this: I cannot 
 : see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor 
 : on BOX_A_ETH0).

This is where your problem lies now.  You need to find out why the 
SYN (which you said was transmitted to BOX_B_ETH1) is not getting 
accepted by this IP stack.

  * reverse path filtering (net.ipv4.conf.*.rp_filter)
  * packet filtering rules on BOX_B?

 : I think the reason is that since box A never sends a SYN packet 
 : itself, it never classifies the connection as ESTABLISHED, so all 
 : further traffic gets rejected. It''s still a mystery to me what 
 : happens to SYN packets from be in this scenario however.

BOX_A will never have a socket in ESTABLISHED state.  BOX_A will 
have a state entry in the /proc/net/ip_conntrack table.  Examine 
/proc/net/ip_conntrack after sending a SYN to BOX_B.

 : It turns out that I have to supplement DNAT with SNAT for this to work
 : correctly.
 : On box A:
 : iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
 : --to-destination $BOX_B_ETH1
 : iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
 : $SERVER_PORT -j SNAT --to-source $BOX_A_ETH1

If this works, then I think you problem may be reverse path 
filtering.

 : in this case, the clients can connect, however the server on B 
 : sees only IP of BOX_A_ETH1, not the original client IPs.

[ tproxy recommendation snipped ]

- -Martin

- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFF8OURHEoZD1iZ+YcRAoenAJ9XCZyMf4K7TVCTs28bzIGeu3EEewCg07Cw
Spk8a+T/th+ESyPN4hSTjYs=k+5E
-----END PGP SIGNATURE-----

Alec Matusis

2007-Mar-09 08:16 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

Thanks Martin.

My rp_filter was disabled on server box B:
# cat /proc/sys/net/ipv4/conf/all/rp_filter             
0

I got it to work however, along the lines you were suggesting like this:
On Box A:
iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1

on box B, I finally found ACK packets. I do not know why I could not see
them before. It turns out that box B was trying to route ACKs via its WAN
interface eth0 directly to the client, in some sort of asymmetric router
fashion. This did not work.

So on B, I routed ACKs back to A:

#ip rule add from $BOX_B_ETH1 lookup 3
#ip route add default via $BOX_A_ETH1 table 3

Now it works, with DNAT on A and without anything else. 
I have 1 small follow-up question:

1) when I add custom rules like 
#ip rule add from $BOX_B_ETH1 lookup 3
it does not take effect for at least 1 minute, perhaps 2-3. 
Why is that? 
This is confusing, since it makes one think that the rule does not work,
while in reality it just has not taken effect.

Thanks a lot for your help,
Alec.
> -----Original Message-----
> From: Martin A. Brown [mailto:martin@linux-ip.net]
> Sent: Thursday, March 08, 2007 8:40 PM
> To: Alec Matusis
> Cc: lartc@mailman.ds9a.nl
> Subject: RE: [LARTC] routing TCP to another box preserving ORIGINAL
> client IPs
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Alec,
> 
>  : I tried implementing DNAT as you indicated:
>  :
>  : iptables -t nat -i eth0 [...] -j DNAT --to-destination $BOX_B_ETH1
>  :
>  : After that, I can see SYN packets arriving on BOX_B_ETH1, having
>  : the original client''s IP.
> 
> OK, that means DNAT + routing on your BOX_A is working correctly.
> 
>  : Only half of the connection gets established after this: I cannot
>  : see ACK packets from box B anywhere (neither on BOX_B_ETH0, nor
>  : on BOX_A_ETH0).
> 
> This is where your problem lies now.  You need to find out why the
> SYN (which you said was transmitted to BOX_B_ETH1) is not getting
> accepted by this IP stack.
> 
>   * reverse path filtering (net.ipv4.conf.*.rp_filter)
>   * packet filtering rules on BOX_B?
> 
>  : I think the reason is that since box A never sends a SYN packet
>  : itself, it never classifies the connection as ESTABLISHED, so all
>  : further traffic gets rejected. It''s still a mystery to me what
>  : happens to SYN packets from be in this scenario however.
> 
> BOX_A will never have a socket in ESTABLISHED state.  BOX_A will
> have a state entry in the /proc/net/ip_conntrack table.  Examine
> /proc/net/ip_conntrack after sending a SYN to BOX_B.
> 
>  : It turns out that I have to supplement DNAT with SNAT for this to
> work
>  : correctly.
>  : On box A:
>  : iptables -t nat -i eth0 -p tcp -m tcp --dport $SERVER_PORT -j DNAT
>  : --to-destination $BOX_B_ETH1
>  : iptables -t nat -A POSTROUTING -d $BOX_B_ETH1 -p tcp -m tcp --dport
>  : $SERVER_PORT -j SNAT --to-source $BOX_A_ETH1
> 
> If this works, then I think you problem may be reverse path
> filtering.
> 
>  : in this case, the clients can connect, however the server on B
>  : sees only IP of BOX_A_ETH1, not the original client IPs.
> 
> [ tproxy recommendation snipped ]
> 
> - -Martin
> 
> - --
> Martin A. Brown
> http://linux-ip.net/
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
> Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)
> 
> iD8DBQFF8OURHEoZD1iZ+YcRAoenAJ9XCZyMf4K7TVCTs28bzIGeu3EEewCg07Cw
> Spk8a+T/th+ESyPN4hSTjYs> =k+5E
> -----END PGP SIGNATURE-----

Martin A. Brown

2007-Mar-09 15:43 UTC

head link

RE: routing TCP to another box preserving ORIGINAL client IPs

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

 : 1) when I add custom rules like 
 : #ip rule add from $BOX_B_ETH1 lookup 3
 : it does not take effect for at least 1 minute, perhaps 2-3. 
 :
 : Why is that? 

It''s quite simple.  There is a routing cache.  Recently used routes 
are stored here for faster access.  If you would like to empty the 
routing cache, then you must make your changes to the routing tables 
and then empty the routing cache:

  ip route flush cache

Be very careful not to omit the word "cache".  You will have a nice 
little surprise if you forget the word "cache".

 : This is confusing, since it makes one think that the rule does 
 : not work, while in reality it just has not taken effect.

If you''d like to view the route cache:

  ip route show cache

Do you want to see a particular entry?

  ip route show cache 72.14.203.104

- -Martin

- -- 
Martin A. Brown
http://linux-ip.net/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: pgf-0.72 (http://linux-ip.net/sw/pine-gpg-filter/)

iD8DBQFF8YC/HEoZD1iZ+YcRAkfFAJ9zYzVRCVMTGE619avs4hZVe+yi2gCgtGRi
iCnX/HpQS3PiGIvlaJi7nlo=S+EQ
-----END PGP SIGNATURE-----

Reasonably Related Threads

Search for more reasonably related threads

LARTC - Mar 2007 - routing TCP to another box preserving ORIGINAL client IPs

routing TCP to another box preserving ORIGINAL client IPs

Re: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

RE: routing TCP to another box preserving ORIGINAL client IPs

Reasonably Related Threads