LARTC - Oct 2005 - Advice....

Hi,

  I''m a relative newbie to LARTC but I have read Matthew
Marsh''s book and
lurked on this list for a while.... I still seem to be missing a few key
ideas here.... So... Maybe folks on the list will be kind enough to help.

  I have two different ISPs. Cogent and Bell. I have three different
firewalls (2 PIX and 1 IPCop). And I have an Ubuntu Linux box doing LARTC
for around 40 LANs (all using 192.168.x.x/24).... Here''s a picture:

                      /----------------------|
                     /                       |
                    3         |-----IPCop----|---cogent
distribution----0-ubuntu-1----|              |
                    2         |-----PIX------|
                    |         |
     ^             n/a        |-----PIX----------bell
     |                        ^
192.168.x.x/24         192.168.25.x/24

distribution upstream if:      192.168.24.1/24
ubuntu eth0:                   192.168.24.2/24
ubuntu eth1:                   192.168.25.1
ipcop downsteam ip:            192.168.25.25
pix1 downstream if:            192.168.25.20
pix2 downstream if:            192.168.25.21

The Ubuntu machine also has an interface in the cogent DMZ using a public
ip address (xxx.xxx.xxx.75).

Here''e the result of ip addr ls:

1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:01:03:cf:26:6d brd ff:ff:ff:ff:ff:ff
    inet 192.168.24.2/24 brd 192.168.24.255 scope global eth0
    inet6 fe80::201:3ff:fecf:266d/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:10:4b:99:1a:84 brd ff:ff:ff:ff:ff:ff
    inet 192.168.25.1/24 brd 192.168.25.255 scope global eth1
    inet6 fe80::210:4bff:fe99:1a84/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:10:4b:74:99:e3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.26.1/24 brd 192.168.26.255 scope global eth2
    inet6 fe80::210:4bff:fe74:99e3/64 scope link
       valid_lft forever preferred_lft forever
5: eth3: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000
    link/ether 00:b0:d0:b6:5d:65 brd ff:ff:ff:ff:ff:ff
    inet xxx.xxx.xxx.75/27 brd xxx.xxx.xxx.255 scope global eth3
6: sit0: <NOARP> mtu 1480 qdisc noop
    link/sit 0.0.0.0 brd 0.0.0.0


I have set up 4 new routing tables in /etc/iproute2/rt_tables:

# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep

100     management
110     cogent
120     bell
130     ipcop

and I have created a simple script to use all this stuff the way I want
to:

#!/bin/sh
##
##First let''s set up a rule and route for the management traffic
##
ip rule add from xxx.xxx.xxx.75 priority 10000 table management
ip route add default via xxx.xxx.xxx.xxx proto static table management
#          Cogent router ^^^^^^^^^^^^^^^
##
##Next let''s set up a route for all Cogent traffic and define rules for
Cogent
##
## First a default route for Cogent:
#
ip route add default via 192.168.25.20 proto static table cogent
#
## Then rules to use the Cogent route:
#
ip rule add from 192.168.1.0/24 priority 11000 table cogent
ip rule add from 192.168.2.0/24 priority 11010 table cogent
#
##
##Then let''s set up a route for all Bell traffic and define rules for
Bell
##
#
## First a default route for Bell:
#
ip route add default via 192.168.25.21 proto static table bell
#
## Then rules to use the Bell route:
#
ip rule add from 192.168.20.0/24 priority 12022 table bell
#
## End of Bell rules
#
## And now lets add a route to use the IP Cop firewall
#
ip route add default via 192.168.25.25 proto static table ipcop
#
## Then rules to use the IP Cop route:
#
# No rules yet
#
# Routes for local sites connected via ethernet
#
ip route add 192.168.1.0/24 via 192.168.24.1 table main
ip route add 192.168.2.0/24 via 192.168.24.1 table main
ip route add 192.168.20.0/24 via 192.168.24.1 table main
#
ip route add default via 192.168.25.20 table main
#
ip route flush cache
#
# End of script



Here are my questions:

1) I want traffic coming from 192.168.20.x/24 to egress via the Bell PIX
and Bell to the Internet, but I also want to connect from  192.168.20.x/24
to devices in the Cogent DMZ (including the DMZ interface of the Ubuntu
LARTC router)... Which doesn''t seem to want to work... WHY?? Traffic
should go out Bell and route through the Internet to the Cogent DMZ LAN
which has public address space.... but it does not... I can ping and
traceroute to any other host on the Internet from 192.168.20.x/24 and
traceroute shows the traffic going out Bell just fine... but traceroute
to a device in the Cogent DMZ stops at the distribution router.... It must
be because the Ubuntu LARTC router has an interface in the Cogent DMZ LAN....
How do I tell the Ubuntu box to ignore this local interface in routing
decisions??

2) What''s the best way to monitor the traffic passing through the
Ubuntu
LARTC router? I am using jnettop and darkstat but maybe there are better
tools?? Advice? I would like to monitor traffic volume by distribution
LAN... Unfortunately, the distribution equipment (Cisco 4000) does not
support SNMP so I cannot use MRTG to poll this equipment...

3) One objective of this setup is to be able to shift traffic between the
two Cogent firewalls as a testbed to evaluate different firewall
technology and proxy services using live traffic on a dynamic basis... Any
advice from more knowledgable folks on doing this??

4) Any other thoughts about what I am tring to do? Any better way??


TIA and kind regards,

Dave

> 1) I want traffic coming from 192.168.20.x/24 to egress via the Bell PIX
> and Bell to the Internet, but I also want to connect from  192.168.20.x/24
> to devices in the Cogent DMZ (including the DMZ interface of the Ubuntu
> LARTC router)... Which doesn''t seem to want to work... WHY??
Traffic
> should go out Bell and route through the Internet to the Cogent DMZ LAN
> which has public address space.... but it does not... I can ping and
> traceroute to any other host on the Internet from 192.168.20.x/24 and
> traceroute shows the traffic going out Bell just fine... but traceroute
> to a device in the Cogent DMZ stops at the distribution router.... It must
> be because the Ubuntu LARTC router has an interface in the Cogent DMZ
LAN....
> How do I tell the Ubuntu box to ignore this local interface in routing
> decisions??
I take it that the Ubuntu Linux box is the ""Distribution Router that
you are speaking of?

I know you have provided a list of the rules / routes that you are populating
your various routing tables with but I''d like a an output of your
routing tables and your routing rules.

Also can we get an example of a (scrubbed) trace route from a 192.168.20.x
client computer.

As I''m sitting here thinking about it I bet you can not send returning
(outbound) traffic that came in the eth3 b/c your system is using Cogent as
it''s default route to the internet.  To solve this you may need to mark
the traffic that comes in eth3 so that you can test for the IP Mark to use the
Management routing table.
> 2) What''s the best way to monitor the traffic passing through the
Ubuntu
> LARTC router? I am using jnettop and darkstat but maybe there are better
> tools?? Advice? I would like to monitor traffic volume by distribution
> LAN... Unfortunately, the distribution equipment (Cisco 4000) does not
> support SNMP so I cannot use MRTG to poll this equipment...
You could easily setup rules in your FORWARD table that are used soly as
counters...
> 3) One objective of this setup is to be able to shift traffic between the
> two Cogent firewalls as a testbed to evaluate different firewall
> technology and proxy services using live traffic on a dynamic basis... Any
> advice from more knowledgable folks on doing this??
Rather than just changing the default route in your Cogent table I''d
set up two Cogent tables, one with one router as the default and the other table
with the other router.  I think this would allow you to shift load from one
router over to the other with out breaking existing streams.  To do this you
would have a couple of rules that would decide which Cogent routing table to use
based on the marks in the packet / stream.  If a packet / stream is not marked
you would know that it is new and could just start going out the router that you
want and thus be marked for said router on the way out.

If packet / stream marked as Cogent_1 use table Cogent_1.
If packet / stream marked as Cogent_2 use table Cogent_2.
If packet / stream unmarked (new) use table Cogent_(what) and mark as such.
> 4) Any other thoughts about what I am tring to do? Any better way??
That depends on what you are ultimately trying to do.  From the sounds of it you
are still in a testing phase and don''t have a final direction to go
yet.  (This may just be me misunderstanding you though.)



Grant. . . .