Been running this for quite a while and noticed that have intermittent problems getting out. Find that if I ping the same site from 2 computers it may work on one and fail on the other. Also was surprised that some time they are going out different interfaces at the same time. Seems to work all the time from the firewall. Running 2.6.10 kernel with the multipath routing patches on a debian sarge system. # ip rule 0: from all lookup local 60: from all lookup main 200: from all lookup 200 201: from 216.170.136.0/24 lookup isp1 201: from 24.196.120.28/30 lookup isp2 222: from all lookup multi 222: from all lookup multi 32766: from all lookup main 32767: from all lookup default cat /etc/iproute2/rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # 1 inr.ruhep 201 isp1 202 isp2 222 multi root@fonroute:~# ip route list table 200 192.168.0.0/16 via 192.168.2.254 dev eth0 root@fonroute:~# ip route list table 201 default via 216.170.136.1 dev eth1 proto static src 216.170.136.82 prohibit default proto static metric 1 root@fonroute:~# ip route list table 202 default via 24.196.120.29 dev eth2 proto static src 24.196.120.30 prohibit default proto static metric 1 root@fonroute:~# ip route list table 222 default proto static nexthop via 216.170.136.1 dev eth1 weight 1 nexthop via 24.196.120.29 dev eth2 weight 4 using shorewall to setup rules. iptable -L Chain PREROUTING (policy ACCEPT) target prot opt source destination net_dnat all -- anywhere anywhere net_dnat all -- anywhere anywhere loc_dnat all -- anywhere anywhere Chain POSTROUTING (policy ACCEPT) target prot opt source destination eth1_masq all -- anywhere anywhere eth2_masq all -- anywhere anywhere Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain eth1_masq (1 references) target prot opt source destination masq2 all -- 192.168.2.0/24 anywhere Chain eth2_masq (1 references) target prot opt source destination masq1 all -- 192.168.2.0/24 anywhere Chain loc_dnat (1 references) target prot opt source destination REDIRECT tcp -- anywhere anywhere tcp dpt:www redir ports 3128 Chain masq1 (1 references) target prot opt source destination RETURN all -- anywhere 192.168.0.0/16 RETURN all -- fonroute.advocap.org anywhere SNAT all -- anywhere anywhere to:24.196.120.30 Chain masq2 (1 references) target prot opt source destination RETURN all -- anywhere 192.168.0.0/16 RETURN all -- fonroute.advocap.org anywhere SNAT all -- anywhere anywhere to:216.170.136.73 Chain net_dnat (2 references) target prot opt source destination DNAT tcp -- !192.168.0.0/16 anywhere multiport dports ssh,www to:192.168.2.1 DNAT tcp -- !192.168.0.0/16 anywhere multiport dports smtp,imaps,https to:192.168.2.10 DNAT tcp -- !192.168.0.0/16 anywhere tcp dpt:2525 to:192.168.2.10:25 DNAT tcp -- !192.168.0.0/16 anywhere tcp dpt:8000 to:192.168.2.12:443 DNAT tcp -- !192.168.0.0/16 anywhere tcp dpt:9000 to:192.168.2.12:22 REDIRECT tcp -- anywhere anywhere tcp dpt:1022 redir ports 22 Have snat on both interfaces Have rules to keep vpn traffic from getting snated. Any solution? Any way to troubleshoot? John _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Hi John. On 7/28/05, John McMonagle <johnm@advocap.org> wrote:> Find that if I ping the same site from 2 computers it may work on one > and fail on the other. > Also was surprised that some time they are going out different > interfaces at the same time.Same symptoms I had.> Have snat on both interfacesWhen you SNAT incoming packets, you need to do something different from what is in the HOWTO ([4]) because SNAT is done before the routing desition (check the Kernel Packet Traveling Diagram[5]). I had the same problem [1]. The solution is to use conntrack and mark packets on arrival, and then route them back using the fwmark[2]. There''s no need to tell you I had a hard time with this. There should be a warning about this in the HOWTO (in this page [4]). The proposed solution I quote in [2] worked for me for the multiple uplink providers + SNAT problem. It is (Using the same variables that are in the HOWTO [4]): 1) Mark packages on arrival: iptables -t mangle -A PREROUTING -m conntrack --ctorigdst $IP1 -j MARK --set-mark=1 iptables -t mangle -A PREROUTING -m conntrack --ctorigdst $IP2 -j MARK --set-mark=2 And then use the mark to route the outgoing packages correctly. ip rule add fwmark 1 table T1 ip rule add fwmark 2 table T2 Regards, Nelson.- PD : I solved my problem with IPVS and multiple uplink providers (see [3]). [1] http://mailman.ds9a.nl/pipermail/lartc/2005q2/016171.html [2] http://mailman.ds9a.nl/pipermail/lartc/2005q2/016441.html [3] http://arhuaco.blogspot.com/2005/07/ipvs-and-conntrack.html [4] http://lartc.org/howto/lartc.rpdb.multiple-links.html [5] http://www.docum.org/docum.org/kptd/ -- Homepage : http://geocities.com/arhuaco The first principle is that you must not fool yourself and you are the easiest person to fool. -- Richard Feynman.
I think I said something wrong in my last message. You DNAT incoming packets and then SNAT them when they come back if your Linux router has some server behind it. I don''t know if this is your case (having servers behind the router). (I needed to top-post here --- maybe not). On 7/28/05, Nelson Castillo <nelsoneci@gmail.com> wrote:> Hi John. > > On 7/28/05, John McMonagle <johnm@advocap.org> wrote: > > > Find that if I ping the same site from 2 computers it may work on one > > and fail on the other. > > Also was surprised that some time they are going out different > > interfaces at the same time. > > Same symptoms I had. > > > Have snat on both interfaces > > When you SNAT incoming packets, you need to do something different > from what is in the HOWTO ([4]) because SNAT is done before the > routing desition (check the Kernel Packet Traveling Diagram[5]). > > I had the same problem [1]. The solution is to use conntrack and mark > packets on arrival, and then route them back using the fwmark[2]. > > There''s no need to tell you I had a hard time with this. There should > be a warning about this in the HOWTO (in this page [4]). > > The proposed solution I quote in [2] worked for me for the > multiple uplink providers + SNAT problem. > > It is (Using the same variables that are in the HOWTO [4]): > > 1) Mark packages on arrival: > > iptables -t mangle -A PREROUTING -m conntrack --ctorigdst $IP1 -j > MARK --set-mark=1 > iptables -t mangle -A PREROUTING -m conntrack --ctorigdst $IP2 -j > MARK --set-mark=2 > > And then use the mark to route the outgoing packages correctly. > > ip rule add fwmark 1 table T1 > ip rule add fwmark 2 table T2 > > Regards, > Nelson.- > > PD : I solved my problem with IPVS and multiple uplink providers (see [3]). > > [1] http://mailman.ds9a.nl/pipermail/lartc/2005q2/016171.html > [2] http://mailman.ds9a.nl/pipermail/lartc/2005q2/016441.html > [3] http://arhuaco.blogspot.com/2005/07/ipvs-and-conntrack.html > [4] http://lartc.org/howto/lartc.rpdb.multiple-links.html > [5] http://www.docum.org/docum.org/kptd/ > > -- > Homepage : http://geocities.com/arhuaco > > The first principle is that you must not fool yourself > and you are the easiest person to fool. > -- Richard Feynman.
Nelson Not been having any problems with incoming DNAT I''m using a bit different solution. Noticed that the returning packets wanted to head for the correct interface probably because of conntracking stuff. Problem was they would be routed then the correct interface to whatever one was the default. What I basically added rules that said if it''s from an interfaces ip go out that interface. This is the setup for one of the isp interfaces: IP="24.196.120.30" NET="24.196.120.28" LENGTH=30 ROUTER="24.196.120.29" BRD="24.196.120.31" ip link set $IFACE up ip addr flush dev $IFACE ip addr add $IP/$LENGTH brd $BRD dev $IFACE ip rule add prio 201 from $NET/$LENGTH table isp2 ip route add default via $ROUTER dev $IFACE src $IP proto static table isp2 ip route append prohibit default table isp2 metric 1 proto static # call something to fixup default route /etc/network/defroute Doing some simular tricks to get ipsec vpn works outgoing from the firewall. IP="192.168.2.254" NET="192.168.2.0" LENGTH=24 BRD="192.168.2.255" ip link set $IFACE up ip addr flush dev $IFACE ip addr add $IP/$LENGTH brd $BRD dev $IFACE #next is to make sure local 192.168. goes via eth0 ip rule delete prio 200 table 220 ip route del table 200 ip route add 192.168.0.0/16 via $IP dev $IFACE table 200 ip rule add prio 200 table 200 Nelson Castillo wrote:>I think I said something wrong in my last message. >You DNAT incoming packets and then SNAT them when >they come back if your Linux router has some server behind it. >I don''t know if this is your case (having servers behind the router). > >(I needed to top-post here --- maybe not). > >On 7/28/05, Nelson Castillo <nelsoneci@gmail.com> wrote: > > >>Hi John. >> >>On 7/28/05, John McMonagle <johnm@advocap.org> wrote: >> >> >> >>>Find that if I ping the same site from 2 computers it may work on one >>>and fail on the other. >>>Also was surprised that some time they are going out different >>>interfaces at the same time. >>> >>> >>Same symptoms I had. >> >> >> >>>Have snat on both interfaces >>> >>> >>When you SNAT incoming packets, you need to do something different >>from what is in the HOWTO ([4]) because SNAT is done before the >>routing desition (check the Kernel Packet Traveling Diagram[5]). >> >>I had the same problem [1]. The solution is to use conntrack and mark >>packets on arrival, and then route them back using the fwmark[2]. >> >>There''s no need to tell you I had a hard time with this. There should >>be a warning about this in the HOWTO (in this page [4]). >> >> The proposed solution I quote in [2] worked for me for the >> multiple uplink providers + SNAT problem. >> >> It is (Using the same variables that are in the HOWTO [4]): >> >> 1) Mark packages on arrival: >> >> iptables -t mangle -A PREROUTING -m conntrack --ctorigdst $IP1 -j >>MARK --set-mark=1 >> iptables -t mangle -A PREROUTING -m conntrack --ctorigdst $IP2 -j >>MARK --set-mark=2 >> >> And then use the mark to route the outgoing packages correctly. >> >> ip rule add fwmark 1 table T1 >> ip rule add fwmark 2 table T2 >> >>Regards, >>Nelson.- >> >>PD : I solved my problem with IPVS and multiple uplink providers (see [3]). >> >>[1] http://mailman.ds9a.nl/pipermail/lartc/2005q2/016171.html >>[2] http://mailman.ds9a.nl/pipermail/lartc/2005q2/016441.html >>[3] http://arhuaco.blogspot.com/2005/07/ipvs-and-conntrack.html >>[4] http://lartc.org/howto/lartc.rpdb.multiple-links.html >>[5] http://www.docum.org/docum.org/kptd/ >> >>-- >>Homepage : http://geocities.com/arhuaco >> >>The first principle is that you must not fool yourself >>and you are the easiest person to fool. >> -- Richard Feynman. >> >>