I am trying to set up a network that can accept tcp connections over two
different isp connection. The ultimate goal is to facilitate the
migration from one ISP to another ISP. I decided to delve into the
world of Linux Advanced Routing, so I poured over all of the
documentation on the internet. I have it set up in what I believe to be
the case where it should work, but I can''t understand why it
doesn''t.
Here is the drawing:
------------------------ --------------------------
| WAN Router 1 | | WAN Router 2 |
------------------------ --------------------------
\ /
\ /
\ /
---------------------
| WAN Switch |
----------------------
/ \
/ \
----------------------- ------------------------
| Firewall 1 | | Firewall 2 |
----------------------- ------------------------
\ /
\ /
\ /
-----------------------
| DMZ Switch |
-----------------------
|
|
*---*---*---*---*---*---*
| Linux Router |
*---*---*---*---*---*---*
|
|
------------------------
| Linux Server |
------------------------
Here is what I have going. The router has a single interface that I
have given an IP address on the network space assigned by each ISP. It
is performing NAT using two DNAT iptables rules for the Linux Server
that is sitting behind it. The linux router takes $WAN1_IP ->
10.0.2.205 and takes $WAN2_IP -> 10.0.2.206. I then set up a second
routing table so that I can have two default gateways depending on which
IP I want to take out. Here are the exact commands I have issued:
iptables -t nat -A PREROUTING -j DNAT -dst $WAN1_IP to 10.0.2.205
iptables -t nat -A PREROUTING -j DNAT -dst $WAN2_IP to 10.0.2.205
ip route show table main | grep -Ev ^default \> | while read ROUTE ; do
> ip route add table 4 $ROUTE
> done
ip route add default via $FIREWALL2_IP
ip route add default via $FIREWALL1_IP table 4
ip rule add from $WAN1_SUBNET table 4
After completing this setup, I am able to traceroute -s $WAN1_IP and
see it go out the WAN1 side and traceroute -s $WAN2_IP and see it go out
WAN2. I am also able to use the ip route get command and everything
works as I expect it to work. My first failure occurred when I tried to
create a tcp connection from the internet. It works coming in on
$WAN2_IP and fails using $WAN1_IP. The problem is that when I watch a
dump of the outgoing packets while trying to bring up a tcp connection
to $WAN1_IP, I see the packets coming in and then I see the reply going
out to the MAC address of $FIREWALL2 instead of the MAC address of
$FIREWALL1 as it should and thus the connection fails. Any ideas?
Thanks in advance.
Bryan
Bryan Goodgion
Cognitive Solutions, Inc.
813.286.4000 ext 103
Hi Bryan,
[ please post any follow up messages in plain text. HTML messages
don''t
archive well, contain lots of strangely escaped characters, and....
well, you probably know all of this...so keep it in plain text! ]
I presume you have a single service on your internal network ("Linux
Server") you wish to publish. It''s not clear to me as I read your
description exactly what the IP addressing of the various machines is.
: I am trying to set up a network that can accept tcp connections over
: two different isp connection.
And, I trust you have read the following?
http://linux-ip.net/html/adv-multi-internet.html#adv-multi-internet-inbound
http://lists.netfilter.org/pipermail/netfilter/2001-May/011697.html
: The ultimate goal is to facilitate the
: migration from one ISP to another ISP. I decided to delve into the
: world of Linux Advanced Routing, so I poured over all of the
: documentation on the internet.
What did you pour all over the documentation on the Internet? ;) Did it
get in your pores?
: I have it set up in what I believe to be the case where it should
: work, but I can''t understand why it doesn''t. Here is the
drawing:
<diagram snipped>
: Here is what I have going. The router has a single interface that I
: have given an IP address on the network space assigned by each ISP.
: It is performing NAT using two DNAT iptables rules for the Linux Server
: that is sitting behind it. The linux router takes $WAN1_IP ->
: 10.0.2.205 and takes $WAN2_IP -> 10.0.2.206. I then set up a second
: routing table so that I can have two default gateways depending on
: which IP I want to take out. Here are the exact commands I have
: issued:
OK....that''s what you should be doing, but that''s not what you
appear
to be doing here:
: iptables -t nat -A PREROUTING -j DNAT -dst $WAN1_IP to 10.0.2.205
: iptables -t nat -A PREROUTING -j DNAT -dst $WAN2_IP to 10.0.2.205
Here, the target IP is the same....10.0.2.205, for both links. Try this
instead:
iptables -t nat -A PREROUTING -j DNAT -dst $WAN2_IP to 10.0.2.206
(keep reading...)
: ip route show table main | grep -Ev ^default \
: > | while read ROUTE ; do
: > ip route add table 4 $ROUTE
: > done
:
: ip route add default via $FIREWALL2_IP
: ip route add default via $FIREWALL1_IP table 4
Yep! And everything should look dandy in table 4:
# ip route show table 4
: ip rule add from $WAN1_SUBNET table 4
Now, you''ll also want to add an entry to make sure that the packets
associated with connections to 10.0.2.205 go via table 4:
# ip rule add from 10.0.2.205 table 4
: After completing this setup, I am able to traceroute -s $WAN1_IP and
: see it go out the WAN1 side and traceroute -s $WAN2_IP and see it go
: out WAN2.
These are locally generated packets, and the "ip rule add from
$WAN1_SUBNET" will ensure you get the correct route.
: I am also able to use the ip route get command and everything works as
: I expect it to work. My first failure occurred when I tried to create
: a tcp connection from the internet. It works coming in on $WAN2_IP and
: fails using $WAN1_IP.
This confirms my diagnosis.
: The problem is that when I watch a dump of the outgoing packets while
: trying to bring up a tcp connection to $WAN1_IP, I see the packets
: coming in and then I see the reply going out to the MAC address of
: $FIREWALL2 instead of the MAC address of $FIREWALL1 as it should and
: thus the connection fails. Any ideas? Thanks in advance.
And one other thing! Make sure you have turned off reverse path
filtering:
# echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
and the other interfaces.
Good luck,
-Martin
--
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Thanks for the help Martin. I was under the impression that DNAT
altered the packets on the PREROUTING chain going both ways.
:What did you pour all over the documentation on the Internet? ;)
Did it
:get in your pores?
I actually studied it so intently that I was sweating. :)
:And one other thing! Make sure you have turned off reverse path
:filtering
This wasn''t on, though I am not exactly sure what it does. Needless to
say, everything is working well now. Thanks again.
-Bryan
-----Original Message-----
From: Martin A. Brown [mailto:mabrown-lartc@securepipe.com]
Sent: Wednesday, March 26, 2003 8:56 PM
To: Bryan Goodgion
Cc: lartc@mailman.ds9a.nl; Dave Salvador
Subject: Re: [LARTC] Simple Mulitple Link Scenario
Hi Bryan,
[ please post any follow up messages in plain text. HTML messages
don''t
archive well, contain lots of strangely escaped characters, and....
well, you probably know all of this...so keep it in plain text! ]
I presume you have a single service on your internal network ("Linux
Server") you wish to publish. It''s not clear to me as I read your
description exactly what the IP addressing of the various machines is.
: I am trying to set up a network that can accept tcp connections over
: two different isp connection.
And, I trust you have read the following?
http://linux-ip.net/html/adv-multi-internet.html#adv-multi-internet-inbo
und
http://lists.netfilter.org/pipermail/netfilter/2001-May/011697.html
: The ultimate goal is to facilitate the
: migration from one ISP to another ISP. I decided to delve into the
: world of Linux Advanced Routing, so I poured over all of the
: documentation on the internet.
What did you pour all over the documentation on the Internet? ;) Did
it
get in your pores?
: I have it set up in what I believe to be the case where it should
: work, but I can''t understand why it doesn''t. Here is the
drawing:
<diagram snipped>
: Here is what I have going. The router has a single interface that I
: have given an IP address on the network space assigned by each ISP.
: It is performing NAT using two DNAT iptables rules for the Linux
Server
: that is sitting behind it. The linux router takes $WAN1_IP ->
: 10.0.2.205 and takes $WAN2_IP -> 10.0.2.206. I then set up a second
: routing table so that I can have two default gateways depending on
: which IP I want to take out. Here are the exact commands I have
: issued:
OK....that''s what you should be doing, but that''s not what you
appear
to be doing here:
: iptables -t nat -A PREROUTING -j DNAT -dst $WAN1_IP to 10.0.2.205
: iptables -t nat -A PREROUTING -j DNAT -dst $WAN2_IP to 10.0.2.205
Here, the target IP is the same....10.0.2.205, for both links. Try this
instead:
iptables -t nat -A PREROUTING -j DNAT -dst $WAN2_IP to 10.0.2.206
(keep reading...)
: ip route show table main | grep -Ev ^default \
: > | while read ROUTE ; do
: > ip route add table 4 $ROUTE
: > done
:
: ip route add default via $FIREWALL2_IP
: ip route add default via $FIREWALL1_IP table 4
Yep! And everything should look dandy in table 4:
# ip route show table 4
: ip rule add from $WAN1_SUBNET table 4
Now, you''ll also want to add an entry to make sure that the packets
associated with connections to 10.0.2.205 go via table 4:
# ip rule add from 10.0.2.205 table 4
: After completing this setup, I am able to traceroute -s $WAN1_IP and
: see it go out the WAN1 side and traceroute -s $WAN2_IP and see it go
: out WAN2.
These are locally generated packets, and the "ip rule add from
$WAN1_SUBNET" will ensure you get the correct route.
: I am also able to use the ip route get command and everything works
as
: I expect it to work. My first failure occurred when I tried to
create
: a tcp connection from the internet. It works coming in on $WAN2_IP
and
: fails using $WAN1_IP.
This confirms my diagnosis.
: The problem is that when I watch a dump of the outgoing packets while
: trying to bring up a tcp connection to $WAN1_IP, I see the packets
: coming in and then I see the reply going out to the MAC address of
: $FIREWALL2 instead of the MAC address of $FIREWALL1 as it should and
: thus the connection fails. Any ideas? Thanks in advance.
And one other thing! Make sure you have turned off reverse path
filtering:
# echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter
and the other interfaces.
Good luck,
-Martin
--
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Bryan, : Thanks for the help Martin. I was under the impression that DNAT : altered the packets on the PREROUTING chain going both ways. Yes, connection tracking is a tricky little beast to understand.... : :What did you pour all over the documentation on the Internet? ;) : : Did it get in your pores? : I actually studied it so intently that I was sweating. :) Heh! : :And one other thing! Make sure you have turned off reverse path : :filtering : This wasn''t on, though I am not exactly sure what it does. Needless to : say, everything is working well now. Thanks again. Great! Glad to hear it. Here''s a tidbit on rp_filter (reverse path filtering): http://ipsysctl-tutorial.frozentux.net/ipsysctl-tutorial.html#AEN616 -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/