Hi.
I have a networking problem that is driving me nuts at the moment. I
have a multi homed network: Cable + DSL.
The problem I have is that although I am 99% sure that I have the
routing table rules set up correctly, for some reason
masqueraded/NATed traffic doesn''t go out of the correct interface.
i.e. I am getting traffic leaving eth2 with the source IP header set
to eth3 and vice versa.
There are 3 network interfaces:
eth0 (internal)
eth2 (DSL)
eth3 (Cable)
(eth1 is unused at present)
Here is my iptables setup (/etc/sysconfig/iptables):
################################
# Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
*nat
:PREROUTING ACCEPT [0:0]
# Port forwarding to an internal machine
-A PREROUTING -i eth2 -d 217.79.103.2 -p tcp -m tcp --dport 18001 -j
DNAT --to-destination 192.168.0.10:18001
-A PREROUTING -i eth3 -d 62.252.21.17 -p tcp -m tcp --dport 18001 -j
DNAT --to-destination 192.168.0.10:18001
# SSH Port Forwarding
-A PREROUTING -i eth2 -d 217.79.103.3 -p tcp -m tcp --dport 22 -j DNAT
--to-destination 192.168.0.10:22
:POSTROUTING ACCEPT [0:0]
# IP Masquerading Traffic From eth2 and eth3
-A POSTROUTING -o eth2 -j MASQUERADE
-A POSTROUTING -o eth3 -j MASQUERADE
:OUTPUT ACCEPT [0:0]
COMMIT
# Completed on Sat Dec 27 10:47:54 2003
# Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
-A FORWARD -i eth0 -o eth2 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
--state NEW,ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eth0 -o eth3 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state
--state NEW,ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eth2 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
--state ESTABLISHED,RELATED -j ACCEPT
-A FORWARD -i eth3 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state
--state ESTABLISHED,RELATED -j ACCEPT
:OUTPUT ACCEPT [0:0]
COMMIT
# Completed on Sat Dec 27 10:47:54 2003
###################################
Additionally, here is the script I use to set up the multi homed
routing:
####################################
# Add ip rules for routing
ip rule add from 217.79.103.0/29 table Griffin
ip rule add from 62.252.21.17 table NTL
# Add routing rules for specific interfaces to insure connectivity
ip route add to default via 217.79.103.1 dev eth2 table Griffin
ip route add to default via 62.252.21.254 dev eth3 table NTL
ip route add to 217.79.103.0/29 dev eth2 table Griffin
ip route add to 62.252.21.0/24 dev eth3 table NTL
# Default route is multi homed
ip route add to default \
nexthop via 217.79.103.1 dev eth2 weight 1 \
nexthop via 62.252.21.254 dev eth3 weight 1
# Commit routing changes
ip route flush cache
#############################
However, looking at tcpdump output from eth2:
11:19:27.153771 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
217.81.134.183.57626: R 0:0(0) ack 2502579442 win 0 (DF)
11:19:30.212427 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
217.81.134.183.57626: R 0:0(0) ack 1 win 0 (DF)
11:20:23.928900 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 >
217.81.134.183.58367: R 0:0(0) ack 2551899092 win 0 (DF)
This is wrong because cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com is
62.252.21.17, which is the IP address of eth3.
Similarly, tcpdump from eth3 says things like:
11:18:32.787404 217.79.103.2.adsl.griffin.net.uk.18001 >
p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 4066315873 win 0 (DF)
11:18:35.683228 217.79.103.2.adsl.griffin.net.uk.18001 >
p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
11:18:41.744790 217.79.103.2.adsl.griffin.net.uk.18001 >
p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF)
This is again wrong, because 217.79.103.2.adsl.griffin.net.uk is the
IP address of eth2.
I am pretty sure the IP rules I set up should work. They assign all
packets with source IP of a particular interface to a routing table
that is routed out via the correct gateway. However, some packets
(from what I have been able to tell, only the masqueraded packets,
but the test was not exhaustive) get sent out of the wrong interface.
Can anybody see a problem with this setup?
TIA.
Gordan
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Dear Bobic, I am sure you havent read Lartc Document clearly. Find inside the document, "iproute2" Those are clue for setting up local area network to connect using two or more connections to ISP. Regards, Rio Martin. On Wednesday 31 December 2003 23:49, Gordan Bobic wrote:> Hi. > I have a networking problem that is driving me nuts at the moment. I > have a multi homed network: Cable + DSL. > The problem I have is that although I am 99% sure that I have the > routing table rules set up correctly, for some reason > masqueraded/NATed traffic doesn''t go out of the correct interface. > i.e. I am getting traffic leaving eth2 with the source IP header set > to eth3 and vice versa. > There are 3 network interfaces: > eth0 (internal) > eth2 (DSL) > eth3 (Cable) > (eth1 is unused at present) > > Here is my iptables setup (/etc/sysconfig/iptables): > ################################ > # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003 > *nat > > :PREROUTING ACCEPT [0:0] > > # Port forwarding to an internal machine > -A PREROUTING -i eth2 -d 217.79.103.2 -p tcp -m tcp --dport 18001 -j > DNAT --to-destination 192.168.0.10:18001 > -A PREROUTING -i eth3 -d 62.252.21.17 -p tcp -m tcp --dport 18001 -j > DNAT --to-destination 192.168.0.10:18001 > # SSH Port Forwarding > -A PREROUTING -i eth2 -d 217.79.103.3 -p tcp -m tcp --dport 22 -j DNAT > --to-destination 192.168.0.10:22 > > :POSTROUTING ACCEPT [0:0] > > # IP Masquerading Traffic From eth2 and eth3 > -A POSTROUTING -o eth2 -j MASQUERADE > -A POSTROUTING -o eth3 -j MASQUERADE > > :OUTPUT ACCEPT [0:0] > > COMMIT > # Completed on Sat Dec 27 10:47:54 2003 > # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003 > *filter > > :INPUT ACCEPT [0:0] > :FORWARD ACCEPT [0:0] > > -A FORWARD -i eth0 -o eth2 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state > --state NEW,ESTABLISHED,RELATED -j ACCEPT > -A FORWARD -i eth0 -o eth3 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state > --state NEW,ESTABLISHED,RELATED -j ACCEPT > -A FORWARD -i eth2 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state > --state ESTABLISHED,RELATED -j ACCEPT > -A FORWARD -i eth3 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state > --state ESTABLISHED,RELATED -j ACCEPT > > :OUTPUT ACCEPT [0:0] > > COMMIT > # Completed on Sat Dec 27 10:47:54 2003 > ################################### > > Additionally, here is the script I use to set up the multi homed > routing: > > #################################### > # Add ip rules for routing > ip rule add from 217.79.103.0/29 table Griffin > ip rule add from 62.252.21.17 table NTL > > # Add routing rules for specific interfaces to insure connectivity > ip route add to default via 217.79.103.1 dev eth2 table Griffin > ip route add to default via 62.252.21.254 dev eth3 table NTL > > ip route add to 217.79.103.0/29 dev eth2 table Griffin > ip route add to 62.252.21.0/24 dev eth3 table NTL > > # Default route is multi homed > ip route add to default \ > nexthop via 217.79.103.1 dev eth2 weight 1 \ > nexthop via 62.252.21.254 dev eth3 weight 1 > > # Commit routing changes > ip route flush cache > ############################# > > However, looking at tcpdump output from eth2: > 11:19:27.153771 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 > > 217.81.134.183.57626: R 0:0(0) ack 2502579442 win 0 (DF) > 11:19:30.212427 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 > > 217.81.134.183.57626: R 0:0(0) ack 1 win 0 (DF) > 11:20:23.928900 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 > > 217.81.134.183.58367: R 0:0(0) ack 2551899092 win 0 (DF) > > This is wrong because cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com is > 62.252.21.17, which is the IP address of eth3. > > Similarly, tcpdump from eth3 says things like: > 11:18:32.787404 217.79.103.2.adsl.griffin.net.uk.18001 > > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 4066315873 win 0 (DF) > 11:18:35.683228 217.79.103.2.adsl.griffin.net.uk.18001 > > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF) > 11:18:41.744790 217.79.103.2.adsl.griffin.net.uk.18001 > > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF) > > This is again wrong, because 217.79.103.2.adsl.griffin.net.uk is the > IP address of eth2. > > I am pretty sure the IP rules I set up should work. They assign all > packets with source IP of a particular interface to a routing table > that is routed out via the correct gateway. However, some packets > (from what I have been able to tell, only the masqueraded packets, > but the test was not exhaustive) get sent out of the wrong interface. > > Can anybody see a problem with this setup? > > TIA. > > Gordan > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Ooops .. Sorry, i havent read the entire email sent to the list by Bobic. My mistake. Bobic having the problem similar to what i got with one of my server running kernel-2.4.20. All the interface i have are under the same brand (Realtek), eth0 would be for clients, eth1 for DSLCable, eth2 for Wireless 2.4Ghz. Weirdly, several of my clients set up correctly to use both eth1 and eth2, but there are many clients having the wrong route packets just as Bobic. This problem can be solved if i change to use SNAT instead of MASQUERADE. Try it Bobic. This Masquerade problem didnt appeared under my Linux 2.4.21 Regards, Rio Martin. On Monday 05 January 2004 09:04, Rio Martin wrote:> Dear Bobic, > I am sure you havent read Lartc Document clearly. > Find inside the document, "iproute2" > Those are clue for setting up local area network to connect using two or > more connections to ISP. > > Regards, > Rio Martin. > > On Wednesday 31 December 2003 23:49, Gordan Bobic wrote: > > Hi. > > I have a networking problem that is driving me nuts at the moment. I > > have a multi homed network: Cable + DSL. > > The problem I have is that although I am 99% sure that I have the > > routing table rules set up correctly, for some reason > > masqueraded/NATed traffic doesn''t go out of the correct interface. > > i.e. I am getting traffic leaving eth2 with the source IP header set > > to eth3 and vice versa. > > There are 3 network interfaces: > > eth0 (internal) > > eth2 (DSL) > > eth3 (Cable) > > (eth1 is unused at present) > > > > Here is my iptables setup (/etc/sysconfig/iptables): > > ################################ > > # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003 > > *nat > > > > :PREROUTING ACCEPT [0:0] > > > > # Port forwarding to an internal machine > > -A PREROUTING -i eth2 -d 217.79.103.2 -p tcp -m tcp --dport 18001 -j > > DNAT --to-destination 192.168.0.10:18001 > > -A PREROUTING -i eth3 -d 62.252.21.17 -p tcp -m tcp --dport 18001 -j > > DNAT --to-destination 192.168.0.10:18001 > > # SSH Port Forwarding > > -A PREROUTING -i eth2 -d 217.79.103.3 -p tcp -m tcp --dport 22 -j DNAT > > --to-destination 192.168.0.10:22 > > > > :POSTROUTING ACCEPT [0:0] > > > > # IP Masquerading Traffic From eth2 and eth3 > > -A POSTROUTING -o eth2 -j MASQUERADE > > -A POSTROUTING -o eth3 -j MASQUERADE > > > > :OUTPUT ACCEPT [0:0] > > > > COMMIT > > # Completed on Sat Dec 27 10:47:54 2003 > > # Generated by iptables-save v1.2.7a on Sat Dec 27 10:47:54 2003 > > *filter > > > > :INPUT ACCEPT [0:0] > > :FORWARD ACCEPT [0:0] > > > > -A FORWARD -i eth0 -o eth2 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state > > --state NEW,ESTABLISHED,RELATED -j ACCEPT > > -A FORWARD -i eth0 -o eth3 -s 192.168.0.0/16 -d 0.0.0.0/0 -m state > > --state NEW,ESTABLISHED,RELATED -j ACCEPT > > -A FORWARD -i eth2 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state > > --state ESTABLISHED,RELATED -j ACCEPT > > -A FORWARD -i eth3 -o eth0 -s 0.0.0.0/0 -d 192.168.0.0/16 -m state > > --state ESTABLISHED,RELATED -j ACCEPT > > > > :OUTPUT ACCEPT [0:0] > > > > COMMIT > > # Completed on Sat Dec 27 10:47:54 2003 > > ################################### > > > > Additionally, here is the script I use to set up the multi homed > > routing: > > > > #################################### > > # Add ip rules for routing > > ip rule add from 217.79.103.0/29 table Griffin > > ip rule add from 62.252.21.17 table NTL > > > > # Add routing rules for specific interfaces to insure connectivity > > ip route add to default via 217.79.103.1 dev eth2 table Griffin > > ip route add to default via 62.252.21.254 dev eth3 table NTL > > > > ip route add to 217.79.103.0/29 dev eth2 table Griffin > > ip route add to 62.252.21.0/24 dev eth3 table NTL > > > > # Default route is multi homed > > ip route add to default \ > > nexthop via 217.79.103.1 dev eth2 weight 1 \ > > nexthop via 62.252.21.254 dev eth3 weight 1 > > > > # Commit routing changes > > ip route flush cache > > ############################# > > > > However, looking at tcpdump output from eth2: > > 11:19:27.153771 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 > > > 217.81.134.183.57626: R 0:0(0) ack 2502579442 win 0 (DF) > > 11:19:30.212427 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 > > > 217.81.134.183.57626: R 0:0(0) ack 1 win 0 (DF) > > 11:20:23.928900 cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com.18001 > > > 217.81.134.183.58367: R 0:0(0) ack 2551899092 win 0 (DF) > > > > This is wrong because cpc4-cbly1-3-0-cust17.glfd.cable.ntl.com is > > 62.252.21.17, which is the IP address of eth3. > > > > Similarly, tcpdump from eth3 says things like: > > 11:18:32.787404 217.79.103.2.adsl.griffin.net.uk.18001 > > > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 4066315873 win 0 (DF) > > 11:18:35.683228 217.79.103.2.adsl.griffin.net.uk.18001 > > > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF) > > 11:18:41.744790 217.79.103.2.adsl.griffin.net.uk.18001 > > > p50811062.dip.t-dialin.net.33062: R 0:0(0) ack 1 win 0 (DF) > > > > This is again wrong, because 217.79.103.2.adsl.griffin.net.uk is the > > IP address of eth2. > > > > I am pretty sure the IP rules I set up should work. They assign all > > packets with source IP of a particular interface to a routing table > > that is routed out via the correct gateway. However, some packets > > (from what I have been able to tell, only the masqueraded packets, > > but the test was not exhaustive) get sent out of the wrong interface. > > > > Can anybody see a problem with this setup? > > > > TIA. > > > > Gordan > > _______________________________________________ > > LARTC mailing list / LARTC@mailman.ds9a.nl > > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Monday 05 Jan 2004 02:55, Rio Martin wrote:> Bobic having the problem similar to what i got with one of my server > running kernel-2.4.20. All the interface i have are under the same brand > (Realtek), eth0 would be for clients, eth1 for DSLCable, eth2 for Wireless > 2.4Ghz. Weirdly, several of my clients set up correctly to use both eth1 > and eth2, but there are many clients having the wrong route packets just as > Bobic. > > This problem can be solved if i change to use SNAT instead of MASQUERADE. > Try it Bobic.Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other things?> This Masquerade problem didnt appeared under my Linux 2.4.21So, it is a 2.4.20 kernel problem? I was hoping to avoid home-brewing a kernel again like I did back with my old setup (RH 7.0, 2.2 kernel), and stick with a standard release kernel. Thanks. Gordan _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Gordan Bobic wrote:> Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other > things?-j SNAT your_ip man iptables _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Monday 05 Jan 2004 11:28, Artūras Šlajus wrote:> Gordan Bobic wrote: > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other > > things? > > -j SNAT your_ipOr rather -j SNAT --to-source your_ip. I get it. I''ll check if that works better than masquerading. Thanks. Gordan _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Monday 05 Jan 2004 11:54, Gordan Bobic wrote:> On Monday 05 Jan 2004 11:28, Artūras Šlajus wrote: > > Gordan Bobic wrote: > > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other > > > things? > > > > -j SNAT your_ip > > Or rather -j SNAT --to-source your_ip. I get it. I''ll check if that works > better than masquerading.Just tried it - no difference. Packets still come out with source IP address not matching the interface. :-( Gordan _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi all, I dont understando what the problem is, can you explain so i can help you out. []´s Anderson> On Monday 05 Jan 2004 02:55, Rio Martin wrote: > > > Bobic having the problem similar to what i got with one of my server> > running kernel-2.4.20. All the interface i have are under the same brand> > (Realtek), eth0 would be for clients, eth1 for DSLCable, eth2 for Wireless> > 2.4Ghz. Weirdly, several of my clients set up correctly to use both eth1> > and eth2, but there are many clients having the wrongroute packets just as> > Bobic. > > > > This problem can be solved if i change to use SNAT instead of MASQUERADE.> > Try it Bobic. > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break other> things? > > > This Masquerade problem didnt appeared under my Linux2.4.21> > So, it is a 2.4.20 kernel problem? I was hoping to avoid home-brewing a kernel> again like I did back with my old setup (RH 7.0, 2.2 kernel), and stick with> a standard release kernel. > > Thanks. > > Gordan > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/>__________________________________________________________________________ Acabe com aquelas janelinhas que pulam na sua tela. AntiPop-up UOL - É grátis! http://antipopup.uol.com.br/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Monday 05 Jan 2004 8:32 pm, andybr wrote:> Hi all, > > I dont understando what the problem is, can you explain > so i can help you out.2 interfaces, eth2 and eth3. ip rules ip route are set up correctly. Yet, for some reason, there are packets coming out of eth2 with the IP address of eth3 and vice versa. This seems to be related mainly to forwarded/NATed connections (which tend to break quite quickly). It is almost as if the connection tracking code in the kernel is broken. I am not pretty certain it is a kernel bug in the shipping RH9 kernel. There was an update released today by RH, but I''m not holding much hope it will behave any differently. But it doesn''t matter, as I have a fresh patched kernel that will hopefully work to try in a bit. Thanks for the help. Gordan _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Monday 05 January 2004 19:06, Gordan Bobic wrote:> On Monday 05 Jan 2004 11:54, Gordan Bobic wrote: > > On Monday 05 Jan 2004 11:28, Artūras Šlajus wrote: > > > Gordan Bobic wrote: > > > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break > > > > other things? > > > -j SNAT your_ip > > Or rather -j SNAT --to-source your_ip. I get it. I''ll check if that works > > better than masquerading. > Just tried it - no difference. Packets still come out with source IP > address not matching the interface. :-(Try it switch manually, first you set up without iproute. Remove all the tables you have created and flush it. Try with ISP1 first. Do SNAT --to ip.of.ISP1 Is it work? Okay, now switch to the ISP2. Do SNAT --to ip.of.ISP2. It should be work, otherwise something wrong with the kernel or iptables you had on your machine. Finish this step first, report back to the list. Regards, Rio Martin. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Tuesday 06 Jan 2004 01:49, Rio Martin wrote:> > > > > Hmm. Just replace -j MASQUERADE with -j SNAT? Will that not break > > > > > other things? > > > > > > > > -j SNAT your_ip > > > > > > Or rather -j SNAT --to-source your_ip. I get it. I''ll check if that > > > works better than masquerading. > > > > Just tried it - no difference. Packets still come out with source IP > > address not matching the interface. :-( > > Try it switch manually, first you set up without iproute. Remove all the > tables you have created and flush it. Try with ISP1 first. Do SNAT --to > ip.of.ISP1 > Is it work? Okay, now switch to the ISP2. Do SNAT --to ip.of.ISP2. > It should be work, otherwise something wrong with the kernel or iptables > you had on your machine. > > Finish this step first, report back to the list.If one of the default routes is removed, everything works OK. However, if there are two default routes, packets get misdirected. ChangeLog for 2.4.21 lists a few conntrack bug fixes, which I suspect to be the cause of this. Basically, the non-deterministic default route selection/rotation seems to take precedence over maintaining the same interface for serving a particular established connection through the firewall. I''m compiling a new clean 2.4.24 with the jumbo routes patch at the moment, which will hopefully fix things. I''m hoping to try it out tonight. And BTW, the latest RH9 kernel released yesterday (2.4.20-28.9 IIRC), is still broken as far as routing is concerned. Gordan _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Tuesday 06 January 2004 02:25 am, Gordan Bobic wrote:> If one of the default routes is removed, everything works OK. However, if > there are two default routes, packets get misdirected. ChangeLog for 2.4.21 > lists a few conntrack bug fixes, which I suspect to be the cause of this. > Basically, the non-deterministic default route selection/rotation seems to > take precedence over maintaining the same interface for serving a > particular established connection through the firewall.You are right. This is because the routing core is often queried more than once to set up a usable route cache entry for a given connection/session. Have a look at ip_route_connect() in linux/include/net/route.h as an example: static inline int ip_route_connect(struct rtable **rp, u32 dst, u32 src, u32 tos, int oif) { int err; err = ip_route_output(rp, dst, src, tos, oif); if (err || (dst && src)) return err; dst = (*rp)->rt_dst; src = (*rp)->rt_src; ip_rt_put(*rp); *rp = NULL; return ip_route_output(rp, dst, src, tos, oif); } Consider when this function is called with src==0, which happens for locally generated output (SNAT is similar I believe). The first ip_route_output() call returns a pointer to a route cache entry, which includes a src ip in (*rp)->rt_src. The first route cache entry doesn''t work for us, because its ''key'' has src==0 and so won''t match subsequent traffic. So a second ip_route_output() is called using the new src as part of its key. The new key matches no existing route cache and as a result the default multipath route is again consulted and a nexthop is determined. This latter process does not use src in its processing so there is no guarantee that the nexthop returned is the same as that returned by the first query. Hence, src ip is not guaranteed to match outbound interface. Julian Anastasov''s patches, noted earlier in this thread, provide a solution to this problem. He allows for additional route rules and route tables that are matched by the second route query in preference to the default route so the src ip and outbound interface can be forced to be consistent. I''m still pretty new to all this, so I hope Julian or someone else can correct any errors I have made. The example above is in the non-NAT case of locally generated traffic, but I believe it''s representative of what happens in the SNAT case as well.> I''m compiling a new clean 2.4.24 with the jumbo routes patch at the moment, > which will hopefully fix things. I''m hoping to try it out tonight. And BTW, > the latest RH9 kernel released yesterday (2.4.20-28.9 IIRC), is still > broken as far as routing is concerned.I haven''t looked at RedHat''s route patch; it''d be killer if they solved this without requiring the additional route rules and tables setups as required by Julian''s patches. Let us know the outcome, would you? The reason for this behavior makes sense from a code perspective, but not IMO from a route administration perspective. I have a patch in its infancy that attempts to address this problem without requiring extra route administration (rules and tables). It works in the non-nat case, but there is still much more testing to go before it''s worth publishing. If it survives the next few weeks of testing, I''d be happy to pass it on to anyone else who might be interested in playing with it. Best Regards, Steve _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/