LARTC - Mar 2004 - split route questions

I am working on a split route and ShoreWall system.  I reviewed the
lartc documentation but have a few areas that I still need help on. 

 

Here is my network:

 

64.xxx.xxx.1/25             66.xxx.xxx.129/26

                        |                                   |

#################################################

#          Eth2 64.xxx.xxx.2          eth0 66.xxx.xxx.130       #

#          eth2:1 64.xxx.xxx.3       eth0:1 66.xxx.xxx.131    #

#                      eth1 192.168.xxx.1
#

#################################################

                                    192.168.xxx.0/24

 

 

Here are the rules as referenced from the lartc split route section:

 

These are added in /etc/iproute2/rt_tables

 

ip route add 64.xxx.xxx.0/25 dev eth2 src 64.xxx.xxx.2 table T1

ip route add default via 64.xxx.xxx.1 table T1

ip route add 66.xxx.xxx.0/26 dev eth0 src 66.xxx.xxx.130 table T2

ip route add default via 66.xxx.xxx.129 table T2

 

 

main routing tables

 

ip route add 64.xxx.xxx.0/25 dev eth2 src 64.xxx.xxx.2

ip route add 66.xxx.xxx.0/26 dev eth0 src 66.xxx.xxx.130

 

 

default route

 

ip route add default via 64.xxx.xxx.1

 

 

routing rules

 

ip rule add from 64.xxx.xxx.2 table T1

ip rule add from 64.xxx.xxx.130 table T2

 

 

ip route add 192.168.122.0/24 dev eth1 table T1

ip route add 66.xxx.xxx.0/26 dev eth0 table T1

ip route add 127.0.0.0/8 dev lo table T1

ip route add 192.168.xxx.0/24 dev eth1 table T2

ip route add 64.xxx.xxx.0/25 dev eth2 table T2

ip route add 127.0.0.0/8 dev lo table T2

 

 

I run web servers behind this firewall and will be using split route as
a transition tool from old circuit to new circuits I also may keep eth0
network haven''t decided yet.

 

Here are my questions:

 

On each wan interface I am running alias'' as referenced in the above
diagram eth2:1, eth2:2 and so on where do these need to be specified in
my above rule set is there a way to just set the route via interface
instead of specifying each ip?

 

Do all of the rules in each section above go into rt_tables and if not
where do they go respectively it doesn''t seem to be stated clearly in
the lartc section on split routes?

 

Is there anything I am missing?

 

Thanks very much for your help.

Hello Thad,

[ I''m sending this to the LARTC list for the benefit of others; Thad
and I
  had a few exchanges offlist, but I thought the list might benefit from
  this exchange.  It''s more on the issue of handling multiple Internet
  links for inbound connections. ]

 : 64.xxx.xxx.1/25             66.xxx.xxx.129/26
 :
 :        |                             |
 :
 : #################################################
 : #  Eth2 64.xxx.xxx.2    eth0 66.xxx.xxx.130     #
 : #  eth2:1 64.xxx.xxx.3  eth0:1 66.xxx.xxx.131   #
 : #              eth1 192.168.xxx.1               #
 : #################################################
 :                         |
 :                192.168.xxx.0/24

[ lots of other unrelated stuff snipped ]

 : Consider also taking advantage of the conntrack match on the PREROUTING
 : chain to catch anything with ctorigdst of an IP on link 1 and using
 : fwmark (-j MARK --set-mark 1) along with routing based on fwmark.

OK!  Here''s what I mean... I''ll assume the following:

 internal server:  192.168.122.1    $intserver
  external IP T1:  64.81.128.17     $ip_t1
  external IP T2:  66.92.124.18     $ip_t2

Assuming that you wish for both of these public IPs to redirect to
internal IP on server 192.168.122.1, here''s what I''d suggest. 
Be
forewarned, I have never done this myself before, although I have done
many things very similar.  Regardless, you may have a bit of tweaking and
testing to do once you have this basic configuration in place.

  - kernel which supports routing by fwmark (CONFIG_IP_ROUTE_FWMARK=y)
    (What distribution are you using?)
  - completely "normal" main routing table, along with your preferred
    default route
  - routing table T1 with routes to the internal network, the local
    networks, and a default route through provider on T1
  - routing table T2 with routes to the internal network, the local
    networks, and a default route through provider on T2
  - iptables rules as follows:

  iptables -t PREROUTING -i $if_t1 -d $ip_t1 -j DNAT --to-dest $intserver
  iptables -t PREROUTING -i $if_t2 -d $ip_t2 -j DNAT --to-dest $intserver

  - These two rules (above) handle the DNAT to get the packets addressed
    correctly for your LAN.

  iptables -t nat -I PREROUTING -i $internal -s $intserver \
    --match conntrack --ctorigdst $ip_t1 -j MARK --set-mark 1
  iptables -t nat -I PREROUTING -i $internal -s $intserver \
    --match conntrack --ctorigdst $ip_t2 -j MARK --set-mark 2

   - These two rules (above) put a mark (meta-data) on the packet as it
     travels through the firewall.  Any packet which was originally
     destined for $ip_t1 will be marked with fwmark 1.  Any packet
     originally destined for $ip_t2 will be marked with fwmark 2.  Note
     that this happens in the PREROUTING table.

   - And finally, the magic routing rules:

  ip rule add from $intserver fwmark 1 table T1
  ip rule add from $intserver fwmark 2 table T2

  [ more stuff snipped ]

I am hoping that somebody else on the LARTC list has tried this and lived
through the pitfalls of using conntrack in this fashion--perhaps somebody
can even point out if I''m leading you down the wrong path.

I''ll include some comments below for posterity.....

 : These are added in /etc/iproute2/rt_tables

 : ip route add 64.xxx.xxx.0/25 dev eth2 src 64.xxx.xxx.2 table T1
 : ip route add default via 64.xxx.xxx.1 table T1
 : ip route add 66.xxx.xxx.0/26 dev eth0 src 66.xxx.xxx.130 table T2
 : ip route add default via 66.xxx.xxx.129 table T2

Actually, /etc/iproute2/rt_tables is not a script which gets excuted, but
rather a mapping of route table names to numbers.  It usually looks
something more like this:

      #
      # reserved values
      #
      255    local
      254    main
      253    default
      0      unspec

      #
      # local
      #
      #1      inr.ruhep


  [ snip ]

 : routing rules
 : ip rule add from 64.xxx.xxx.2 table T1
 : ip rule add from 64.xxx.xxx.130 table T2

The second of these rules looks wrong....did you really mean to have that
be 64.xxx.xxx.130?  Shouldn''t that have been 66.xxx.xxx.130?  Even so,
you
could actually use the following.  See also the nano HOWTO for more detail
[0].

  [ snip ]

 : I run web servers behind this firewall and will be using split route as
 : a transition tool from old circuit to new circuits I also may keep eth0
 : network haven''t decided yet.

So, if I understand correctly, you want to have the internal webserver(s)
reachable on both T1 links simultaneously.  There are two solutions.  The
first involves fully separate sets of IPs from inside to outside [1].  The
other option is to take advantage of the iptables conntrack match module.
See a partial example here [2], although any fairly recent iptables will
have conntrack, and you can look at the options and probably figure out
how to make it work.*

 : On each wan interface I am running alias'' as referenced in the above
 : diagram eth2:1, eth2:2 and so on where do these need to be specified in
 : my above rule set is there a way to just set the route via interface
 : instead of specifying each ip?

And another note about "aliased interfaces".  These don''t
really exist as
far as the kernel is concerned.  To see what I mean, simply do the
following:

# ip addr show dev eth2

The "label eth2:1" parameter on addresses other than the first address
are
used for compatibility with ifconfig.  See also [3].

  [ snip ]

Best of luck,

-Martin

 [0] http://www.ssi.bg/~ja/nano.txt
 [1] http://linux-ip.net/html/adv-multi-internet.html#adv-multi-internet-inbound
 [2] http://mailman.ds9a.nl/pipermail/lartc/2003q2/008173.html
 [3] http://linux-ip.net/html/tools-ip-address.html#ex-tools-ip-address-del

* snipped from "/sbin/iptables -m conntrack --help"

conntrack match v1.2.7a options:
 [!] --ctstate [INVALID|ESTABLISHED|NEW|RELATED|SNAT|DNAT][,...]
                                State(s) to match
 [!] --ctproto  proto           Protocol to match; by number or name, eg.
`tcp''
     --ctorigsrc  [!] address[/mask]
                                Original source specification
     --ctorigdst  [!] address[/mask]
                                Original destination specification
     --ctreplsrc  [!] address[/mask]
                                Reply source specification
     --ctrepldst  [!] address[/mask]
                                Reply destination specification
 [!] --ctstatus [NONE|EXPECTED|SEEN_REPLY|ASSURED][,...]
                                Status(es) to match
 [!] --ctexpire time[:time]     Match remaining lifetime in seconds against
                                value or range of values (inclusive)




-- 
Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

> I am hoping that somebody else on the LARTC list has tried this and lived
> through the pitfalls of using conntrack in this fashion--perhaps somebody
> can even point out if I''m leading you down the wrong path.
I''ve used such a setup recently, worked fine and was 95% similar to
what
you listed. Found it googling around.

One pitfall is dual connection protocols like PPTP and active FTP, where
the inside server opens a connection back to external client. I suppose
that if a helper is available, RELATED will make things work; I solved
this by using an IP alias on LAN so connection inbound from one link would
go to one IP, the other will go other IP and both are aliased on the same
server. The simpler protocols used the fwmark so firewall rules could be
made short.


Rubens


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

>
>   iptables -t nat -I PREROUTING -i $internal -s $intserver \
>     --match conntrack --ctorigdst $ip_t1 -j MARK --set-mark 1
>   iptables -t nat -I PREROUTING -i $internal -s $intserver \
>     --match conntrack --ctorigdst $ip_t2 -j MARK --set-mark 2
>
>   ip rule add from $intserver fwmark 1 table T1
>   ip rule add from $intserver fwmark 2 table T2
>
> I am hoping that somebody else on the LARTC list has tried this and lived
> through the pitfalls of using conntrack in this fashion--perhaps somebody
> can even point out if I''m leading you down the wrong path.
>
Firstly, those iptables command won''t even work. I suspect you mean
iptables -t mangle ( and not -t nat !!!).

Secondly, using this method, if it works, I have to do it for every internal
server which I have. If I have 10 servers, I will have to execute many of
those commands.

I saw some earlier post (by Daniel Chemko) which says he has done this
using CONNMARK, basically what he did it by doing this :-

iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
iptables -t mangle -A PREROUTING -m state --state NEW -i
${OUTSIDE_DEVICE} -j MARK
--set-mark 1
iptables -t mangle -A PREROUTING -m state --state NEW -i
\${OUTSIDE_DEVICE2} -j MARK
--set-mark 2
iptables -t mangle -A PREROUTING -j CONNMARK --save-mark

I haven''t got a chance to try all these. But if this is working, it is
much
cleaner
than this approach using fwmark.





_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

OK I have sufficient evidence now that my split route
( multipath routing ) is causing kernel panic and also
frequent connection lost.

I have set up the split route according to 





_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/