LARTC - Nov 2000 - A complicated routing scenario (for me at least)

Greetings:

I''ve been pouring over every piece of documentaiton on the subject I
can
find, and I''ve concluded that advanced routing + ipchains is what I
need. It also appears that the people that can answer some of the more
in depth questions I have, read this list.

I''m fairly new to this so please forgive me if I ask stupid questions.
Anyway.

Here is what I''m trying to accomplish in ASCII art:

                                 LAN
                                  | (172...)
                                  |
         _/\__/\_             +---+----+            _/\__/\_
        /        \   (63...)  |        | (204...)  /        \
       ( Internet )-----------+ Router +----------( Internet )
        \_  __  _/            |        |           \_  __  _/
          \/  \/              +----+---+             \/  \/ 
                                   | 63..
                                   | 204..
                                   |
                 --+---------------+----------+--  <---single physical
net
                   |                          |        (i.e. one hub)
                   |                          |
               +---+---+ 63..1            +---+---+ 63..2
               | Linux | 63..4            | Linux | 63..3
               +-------+ 204..1           +-------+ 204..2
                         204..4                     204..3

The desired end result is a redundant connection to two dns/mail servers
from the 
internet. I''m willing to fight through this, but I have a few specific
question''s 
for now that I''m hoping someone can answer.

1. From a server''s perspective, do incoming packets get responded to on
the same interface they came in on? Does it matter if the interface in
question is an alias?

2. if I tag a packet coming into the router/firwall above with ipchains
for routing purposes, will the tag persist to the reply packets? Or do I
have to tag the reply packets with ipchains from the responding server?

3. In reading the various documentation, I noticed several references to
files in the /etc/iproute2 directory. Related to this directory:
	a) is there any documentation on the names and syntax of the various
files other than the source code, and the casual references I''ve found?
	b) I don''t currently have this directory in /etc. Is it something I
just create, and does it''s existance imply that the kernel will read
configureation data from it on boot?

Incidentally I''m currently using kernel 2.2.17 on my boxes. Since
I''m
already going to be meticulously documenting my setup, I''d be willing
to
do so in a format that could be posted as a HOWTO or as an example in
someone else''s HOWTO, whatever would be most usefull.

Suggestions in this area are much appreciated since I have no experiance
in HOWTO writing. (Plenty of writing experiance, just not howtos)

Anyway, Thanks for the help.

-Andrew

On Mon, 13 Nov 2000, Andrew wrote:
> Greetings:
> I''ve been pouring over every piece of documentaiton on the subject
I can
> find, and I''ve concluded that advanced routing + ipchains is what
I
> need. It also appears that the people that can answer some of the more
> in depth questions I have, read this list.
Might be true, might be true. ;)
> I''m fairly new to this so please forgive me if I ask stupid
questions.
> Anyway.
We''re all fairly new to this. It isn''t older than a year or
two, three at the
most.
> Here is what I''m trying to accomplish in ASCII art:
>                                  LAN
>                                   | (172...)
>                                   |
>          _/\__/\_             +---+----+            _/\__/\_
>         /        \   (63...)  |        | (204...)  /        \
>        ( Internet )-----------+ Router +----------( Internet )
>         \_  __  _/            |        |           \_  __  _/
>           \/  \/              +----+---+             \/  \/ 
>                                    | 63..
>                                    | 204..
>                                    |
>                  --+---------------+----------+--  <---single physical
> net
>                    |                          |        (i.e. one hub)
>                    |                          |
>                +---+---+ 63..1            +---+---+ 63..2
>                | Linux | 63..4            | Linux | 63..3
>                +-------+ 204..1           +-------+ 204..2
>                          204..4                     204..3
Hmmm. Right.
> The desired end result is a redundant connection to two dns/mail servers
> from the internet. I''m willing to fight through this, but I have a
> few specific question''s for now that I''m hoping someone
can answer.
> 1. From a server''s perspective, do incoming packets get responded
to on
> the same interface they came in on? 
No, unfortunately not. In fact, unless you specifically set things up so
that it will go right the kernel will even play haywire with which interface
packets with certain IP addresses are routed out.
> Does it matter if the interface in
> question is an alias?
No. By the way, drop the mental concept of alias. With the iproute2 ip tool
it only serves to confuse matters. Interfaces kan have multiple IP addresses
attached to them. This is *much* more portable to IPv6... :)
> 2. if I tag a packet coming into the router/firwall above with ipchains
> for routing purposes, will the tag persist to the reply packets? 
No.
> Or do I have to tag the reply packets with ipchains from the responding 
> server?
Yes.
> 3. In reading the various documentation, I noticed several references to
> files in the /etc/iproute2 directory. Related to this directory:
> 	a) is there any documentation on the names and syntax of the various
> files other than the source code, and the casual references I''ve
found?
> 	b) I don''t currently have this directory in /etc. Is it something
I
> just create, and does it''s existance imply that the kernel will
read
> configureation data from it on boot?
a. There are examples for these files in /usr/doc/iproute-2.2.4/iproute2/ on
   RedHat 6.2 systems with iproute2 installed and in
   /usr/share/doc/iproute-2.2.4/iproute2/ on RedHat 7.0 systems.
   These files all have names starting with rt_, and should also be in
   the iproute2 tarball, but I''m too lazy to check :).
b. The directory can contain the files rt_dsfield, rt_protos, rt_realms,
   rt_scopes and rt_tables. Most of the values in these files are user
   settable, and will be read when the files exist. If they do not exist you
   do not get nice names and have to deal with the raw numbers. Note: they
   are *not* necessary for operation, just useful from a user''s point
of
   view.
> Incidentally I''m currently using kernel 2.2.17 on my boxes. Since
I''m
> already going to be meticulously documenting my setup, I''d be
willing to
> do so in a format that could be posted as a HOWTO or as an example in
> someone else''s HOWTO, whatever would be most usefull.
> Suggestions in this area are much appreciated since I have no experiance
> in HOWTO writing. (Plenty of writing experiance, just not howtos)
Well, an ASCII-gram such as the one above and step by step explanations of
your setup and *why* you''ve taken those steps would be great. ;)
> Anyway, Thanks for the help.
No problem.

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

On Mon, 13 Nov 2000, Andrew wrote:
> 2. if I tag a packet coming into the router/firwall above with ipchains
> for routing purposes, will the tag persist to the reply packets? Or do I
> have to tag the reply packets with ipchains from the responding server?
You''ll have to tag them whenever they enter your gateway. The reply
packets are just packets, at ip level there is no difference between them.
Only the very first packets, which initiate the connection, carry some
extra flags.
> 3. In reading the various documentation, I noticed several references to
> files in the /etc/iproute2 directory. Related to this directory:
> 	b) I don''t currently have this directory in /etc. Is it something
I
> just create, and does it''s existance imply that the kernel will
read
> configureation data from it on boot?
I believe you can just create that directory and the files.
> Suggestions in this area are much appreciated since I have no experiance
> in HOWTO writing. (Plenty of writing experiance, just not howtos)
There is an HOWTO about writing HOWTO''s, I think. Try
http://www.linuxdoc.org/ for more elaborate info.

Salukes,

Tung

Thanks for the reply, Here are a few more questions
> >                                  LAN
> >                                   | (172...)
> >                                   |  (eth1)
> >          _/\__/\_             +---+----+            _/\__/\_
> >         /        \   (63...)  |        | (204...)  /        \
> >        ( Internet )-----------+ Router +----------( Internet )
> >         \_  __  _/    (eth0)  |        |  (eth2)   \_  __  _/
> >           \/  \/              +----+---+             \/  \/
> >                              (eth3)| 63..
> >                                    | 204..
> >                                    |
> >                  --+---------------+----------+--  <---single
physical
> > net
> >                    |                          |        (i.e. one hub)
> >                    |                          |
> >                +---+---+ 63..1            +---+---+ 63..2
> >                | Linux | 63..4            | Linux | 63..3
> >                +-------+ 204..1           +-------+ 204..2
> >                          204..4                     204..3
> a. There are examples for these files in /usr/doc/iproute-2.2.4/iproute2/
on
>    RedHat 6.2 systems with iproute2 installed and in
>    /usr/share/doc/iproute-2.2.4/iproute2/ on RedHat 7.0 systems.
>    These files all have names starting with rt_, and should also be in
>    the iproute2 tarball, but I''m too lazy to check :).
> b. The directory can contain the files rt_dsfield, rt_protos, rt_realms,
>    rt_scopes and rt_tables. Most of the values in these files are user
>    settable, and will be read when the files exist. If they do not exist
you
>    do not get nice names and have to deal with the raw numbers. Note: they
>    are *not* necessary for operation, just useful from a user''s
point of
>    view.
OK.. someplace else to look. Are the examples the only thing available
in the way of file syntax? Also could someone help me understand how
these files are read at startup if they exist. (what code/script is
responsible for doing it, and what happens if there are syntax errors. I
have a redhat 6.2 system.)
> Well, an ASCII-gram such as the one above and step by step explanations of
> your setup and *why* you''ve taken those steps would be great. ;)
If/When I get this written up, is this mailing list the place to post
it? Would there be any value in puting it into a separate (mini)-HOWTO?

Based on what everyone''s said, here is what I''m contemplating
(this
still assumes that
that packets are answered on the same interface they come in on. I''m
getting conflicting information for this. Someone said they do, and
someone else said they don''t. I suppose if they don''t I could
use an
explicit source address hint in a routing table entry.):

Packets coming in on eth0,1, and 2 would be marked with
different TOS values based on what interface they came in on using
ipchains, and routed through the proper address on eth3 using the
advanced routing.

Since the TOS field doesn''t change (an assumption, is this true?) I
would know which interface to route the packet back through while at the
same time being able to reset the TOS field back to zero for routing on
the internet.

Comments? Suggestions?

Thanks,

-Andrew
depaan@bibleinfo.com

Andrew,

I''m awfully glad you''re asking these questions, since
I''m contemplating a
similar setup and am not yet to that wonderful place where this presently
obscure new technology resolves into clarity. So I''ll be trying to
follow
along closely as you work out your scheme.

One question I have, in addition to those you''re asking, is whether
iproute2
in itself is adequate to handle fallover routing when one of the outside
links goes down - that is, can it just be set up with lower priority rules
to route the other way, or does something like mon
(http://www.kernel.org/software/mon/) need to be set up to actively change
the routing tables when an outside line goes down?

 \/\/ I-I I T 
 Blauvelt
 whit@transpect.com

On Tue, 14 Nov 2000, Andrew wrote:
> OK.. someplace else to look. Are the examples the only thing available
> in the way of file syntax? Also could someone help me understand how
> these files are read at startup if they exist. (what code/script is
> responsible for doing it, and what happens if there are syntax errors. I
> have a redhat 6.2 system.)
Maybe you should try it first and experiment a bit. It''s rather hard to
answer on vague questions.
> Since the TOS field doesn''t change (an assumption, is this true?)
I
> would know which interface to route the packet back through while at the
> same time being able to reset the TOS field back to zero for routing on
> the internet.
As far as I understand your question: you can change the TOS based on the
incoming interface (and lots of other options). And I think it isn''t
usefull to reset the TOS field when you send out the packets into the
internet, because most routers won''t use the field and simply ignore
it.
When a packets comes back, you should route it dependand on the
destination, not on the TOS field because that can be anything.

I hope I haven''t told complete nonsense.

Tung

On Tue, 14 Nov 2000, Whit wrote:
> Andrew,
> I''m awfully glad you''re asking these questions, since
I''m contemplating a
> similar setup and am not yet to that wonderful place where this presently
> obscure new technology resolves into clarity. So I''ll be trying to
follow
> along closely as you work out your scheme.
Hey, I''ve never done stuff like Andrew''s proposing either.
It''s cool. I want
to know if it''ll work. :)
> One question I have, in addition to those you''re asking, is
whether iproute2
> in itself is adequate to handle fallover routing when one of the outside
> links goes down - that is, can it just be set up with lower priority rules
> to route the other way, or does something like mon
> (http://www.kernel.org/software/mon/) need to be set up to actively change
> the routing tables when an outside line goes down?
Hmmm. Methinks that with the preferences associated with routes and the
effects of neighbor unreachability detection this should work. This 
remark is wholly untested, though.

Doei, Arthur. 

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

On Wed, 15 Nov 2000, Wingtung.Leung wrote:
> 
> On Tue, 14 Nov 2000, Andrew wrote:
> 
> > OK.. someplace else to look. Are the examples the only thing available
> > in the way of file syntax? Also could someone help me understand how
> > these files are read at startup if they exist. (what code/script is
> > responsible for doing it, and what happens if there are syntax errors.
I
> > have a redhat 6.2 system.)
> 
> Maybe you should try it first and experiment a bit. It''s rather
hard to
> answer on vague questions.
But not impossible. There is the iproute2 code that does the actual
parsing of the files in /etc/iproute2, and there is some information in the
ip-cref doc. The parser is the function rtnl_tab_initialize in
lib/rt_names.c of the iproute2 source tree. The syntax is one of the
following for scanf formats (read the manpage for scanf if you can''t
parse
these by sight):

0x%x %s\n
0x%x %s #
%d %s\n
%d %s #

Implied is basically a key-value table, with decimal or hexadecimal numbers
as keys and strings as values. Comments are allowed and whitespace is mostly
ignored. There has to be some between keys and values though. :)

If there are syntax errors you will only see numbers instead of nice user
friendly names, after the ip or tc tool has complained about ''Database
<xyz>
is corrupted at <foobar>'', with one of the files instead of
<xyz> and part
of the file instead of <foobar>

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Greetings

I am new to iproute2 and therefore would not try to give a direct answer to
any of your questions. I have, however, been working on what seems to me to
be a similar problem and will offer it here in case it assists.

I two have two links to the internet though different isps. The requirements
were that users on the lan would be as oblivious as possible to any one link
going down.

The solution I came up with involved two new linux boxes. Both using iproute
and one using masq (nat with 2.4 when released). One box was placed between
the two isps with source routing and a third network card linking to the
dns,www etc (I call it a crossrouter). The second box also links the two
isps and uses source routing and masq to offer redundancy (redundancy box).
An attempt at text art may help

isp1                     +redundancy box+                         isp2
|                            |                             |
|
+---------------------+                           +---------------------+
                             |                             |
                             +   Crossrouter     +
                                            |
     ---+---------+---------------+----------------------------------Lan
      www      dns                              etc


I made may attempts to use one box instead of two without any luck. I had
bad results with marking that would have made a single box solution
possible. The result was the two boxes. On the routers to the isp I have a
second default route to the redundancy box with a higher matrix. If one link
fails the data is routed to the redundancy box where it is masq on an ip
(one assigned by the isp with the good link) address and sent out through
the other isp. Every test I have done is showing the solution to work but it
has not been fully deployed yet.

I would imagine that this type of solution would be worth using is you link
to two different isps and BGP is not available. I intend doubling up on the
crossrouter as it will become a single point of failure.

If anyone knows of a better solution I would be interested to know.

Hope this helps

Regards
Warren

Hello,

I appreciate all the comments, Let''s keep it up. I promise
I''ll post the whole
thing when I have it all figured out. My mental picture is slowly filling out.
Three specific things I''ve got to know though: (diagrams are just for
reference)

 > >                                  LAN
> >                                   | (172...)
> >                                   |  (eth1)
> >          _/\__/\_             +---+----+            _/\__/\_
> >         /        \   (63...)  |        | (204...)  /        \
> >        ( Internet )-----------+ Router +----------( Internet )
> >         \_  __  _/    (eth0)  |        |  (eth2)   \_  __  _/
> >           \/  \/              +----+---+             \/  \/
> >                              (eth3)| 63..
> >                                    | 204..
> >                                    |
> >                  --+---------------+----------+--  <---single
physical
> >                    |                          |             net
> >                    |                          |        (i.e. one hub)
> >                    |                          |
> >                +---+---+ 63..1            +---+---+ 63..2
> >                | Linux | 63..4            | Linux | 63..3
> >                +-------+ 204..1           +-------+ 204..2
> >                          204..4                     204..3
         .---------------------------------------------------------------.
         |Version|  IHL  |Type of Service|          Total Length         |
         |-------+-------+---------------+-------------------------------|
         |         Identification        |Flags|      Fragment Offset    |
         |---------------+---------------+-------------------------------|
         |  Time to Live |    Protocol   |         Header Checksum       |
         |---------------+---------------+-------------------------------|
         |                       Source Address                          |
         |---------------------------------------------------------------|
         |                    Destination Address                        |
        
`---------------------------------------------------------------''

1. If I modify the TOS field on incomming packets with IPchains, and then route
those packets to the proper internal server on my protected network, when that
server replies, what do the TOS fields in the reply packets look like? Are they
copies of the ones that came from the outside initiating the connection routed
through my router? (I hope that''s the case) Or does the kernel on the
replying
machine invent some new TOS value for those packets?

2. On a machine with two IP addresses on the same nic (A and B). If someone on
the internet makes a connection to IP address A. What is the source address of
reply packets in the IP header diagram above? A? (I hope that''s the
case too.)

3. Finally, is it necessary for the router box above to have two addresses on
the internal network. Does it matter that the default gateway that packets,
having a 204.. source address, go to might be a 63.. address? (or vice versa, as
long as all machines are connected to the same physical network) If it
doesn''t
then there is no need for a second IP address on the internal side of the router
(eth3 above). If it does matter, can I put in some fancy next-hop routes on the
servers based on what the source address of reply packets ends up being? (This
would essentially completely ignore the default route)

Something to remember (it may or may not matter to you): Valid packets coming in
on eth2 (204... above) will have a 204... destination address. Also Valid
packets coming in on eth0 (63... above) will have a 63... destination address.
Marking the packets as they come in is merely so that we can route them back on
the same interface they came in on. I don''t want assymetric routes.

Here''s some more ascii art:
                                   local
                                  process
                                    | ^
eth0 \                              | |                                / eth0
eth1 |                              | |                                | eth1
eth2 +--incomming                   | |                      outgoing--+ eth2
eth3 |   packets                    | |                       packets  | eth3
etc. /                              | |                                \ etc.
|                                   | |                                    ^
|                                   | |                                    |
|                                   | |                                    |
|        ---------------------------|-|-----------------(lo interface)<----|
|        |                          | |                                    |
|        V                          V |        _______       _______       |
---> C --> S --> ______ --> D --> ~~~~~~~~
-->|forward|---->|       |-------
     h     a    |input |    e    {Routing }   |Chain  |     |output |ACCEPT
     e     n    |Chain |    m    {Decision}   |_______| --->|Chain  |
     c     i    |______|    a     ~~~~~~~~        |     | ->|_______|
     k     t       |        s       |             |     | |     |   
     s     y       |        q     local           v     | |     |   
     u     |       V        e    process         DENY/  | |     v   
     m     |     DENY/      r    packets        REJECT  | |   DENY/
     |     V    REJECT      a       |                   | |  REJECT
     |   DENY               d       --------------------- |
     V                      e -----------------------------
    DENY

I''ve slightly modified the packet flow diagram from the ipchains howto
according
to my understanding of how things work (in other words, correct me if
I''m
wrong). What I want to know is what the packet looks like once it gets to the
routing decision block above from a local process. (Assume that this packet is a
response to a connection established from the internet. I alredy know how the
source address is decided upon for connections originating from a machine. This
is farely well documented on page 48 of the IP command reference, which is part
of the iproute2 distribution.) I especially want to know what the source and
destination addresses are at this point. I''m sure you all can see why.
If not
let me know and I''ll clarify.


Thanks again for the help and comments.

-- Andrew
depaan@bibleinfo.com
--------------------------------------------------------------
Want answers to life''s big questions? Visit www.bibleinfo.com.

On Wed, 15 Nov 2000, Andrew wrote:
> Hello,
> 
> I appreciate all the comments, Let''s keep it up. I promise
I''ll post the whole
> thing when I have it all figured out. My mental picture is slowly filling
out.
> Three specific things I''ve got to know though: (diagrams are just
for reference)
>  > >                                  LAN
> > >                                   | (172...)
> > >                                   |  (eth1)
> > >          _/\__/\_             +---+----+            _/\__/\_
> > >         /        \   (63...)  |        | (204...)  /        \
> > >        ( Internet )-----------+ Router +----------( Internet )
> > >         \_  __  _/    (eth0)  |        |  (eth2)   \_  __  _/
> > >           \/  \/              +----+---+             \/  \/
> > >                              (eth3)| 63..
> > >                                    | 204..
> > >                                    |
> > >                  --+---------------+----------+--  <---single
physical
> > >                    |                          |             net
> > >                    |                          |        (i.e. one
hub)
> > >                    |                          |
> > >                +---+---+ 63..1            +---+---+ 63..2
> > >                | Linux | 63..4            | Linux | 63..3
> > >                +-------+ 204..1           +-------+ 204..2
> > >                          204..4                     204..3
> 1. If I modify the TOS field on incomming packets with IPchains, and then
route
> those packets to the proper internal server on my protected network, when
that
> server replies, what do the TOS fields in the reply packets look like? Are
they
> copies of the ones that came from the outside initiating the connection
routed
> through my router? (I hope that''s the case) Or does the kernel on
the replying
> machine invent some new TOS value for those packets?
I really do not know.
> 2. On a machine with two IP addresses on the same nic (A and B). If someone
on
> the internet makes a connection to IP address A. What is the source address
of
> reply packets in the IP header diagram above? A? (I hope that''s
the case too.)
Yes, reply packets originate from the exact same IP address as the
originating packets were sent to. At least as far as I understand the
IP RFC''s
> 3. Finally, is it necessary for the router box above to have two addresses
on
> the internal network. Does it matter that the default gateway that packets,
> having a 204.. source address, go to might be a 63.. address? (or vice
versa, as
> long as all machines are connected to the same physical network) If it
doesn''t
> then there is no need for a second IP address on the internal side of the
router
> (eth3 above). If it does matter, can I put in some fancy next-hop routes on
the
> servers based on what the source address of reply packets ends up being?
(This
> would essentially completely ignore the default route)
As long as there is a route from the nodes having a 204... address via the
router, the IP address of the router does not matter. Let me clarify a
little more. Suppose the Router only has a 63... address on the eth3
interface. To route 204 addresses from the internal servers you would set it
up so that *all* addresses *but* the address of that specific internal server 
are routed via the Router. In the Router you can then decide where to send
the packets. They will have the correct source and destination addresses, so
that is not a big problem.
> Something to remember (it may or may not matter to you): Valid packets
coming in
> on eth2 (204... above) will have a 204... destination address. Also Valid
> packets coming in on eth0 (63... above) will have a 63... destination
address.
> Marking the packets as they come in is merely so that we can route them
back on
> the same interface they came in on. I don''t want assymetric
routes.
Oh, but in that case you won''t even need to do the marking. The
information
you need is already fully encapsulated in the IP addresses inside the
packet. Ofcourse, I''m assuming there''s no nasty machine doing
dastardly
deeds on that internal network of yours... :)

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

> >  > >                                  LAN
> > > >                                   | (172...)
> > > >                                   |  (eth1)
> > > >          _/\__/\_             +---+----+            _/\__/\_
> > > >         /        \   (63...)  |        | (204...)  /       
\
> > > >        ( Internet )-----------+ Router +----------( Internet
)
> > > >         \_  __  _/    (eth0)  |        |  (eth2)   \_  __ 
_/
> > > >           \/  \/              +----+---+             \/  \/
> > > >                              (eth3)| 63..
> > > >                                    | 204..
> > > >                                    |
> > > >                  --+---------------+----------+-- 
<---single physical
> > > >                    |                          |            
net
> > > >                    |                          |        (i.e.
one hub)
> > > >                    |                          |
> > > >                +---+---+ 63..1            +---+---+ 63..2
> > > >                | Linux | 63..4            | Linux | 63..3
> > > >                +-------+ 204..1           +-------+ 204..2
> > > >                          204..4                     204..3
> 
> > 1. If I modify the TOS field on incomming packets with IPchains, and
then route
> > those packets to the proper internal server on my protected network,
when that
> > server replies, what do the TOS fields in the reply packets look like?
Are they
> > copies of the ones that came from the outside initiating the
connection routed
> > through my router? (I hope that''s the case) Or does the
kernel on the replying
> > machine invent some new TOS value for those packets?
> 
> I really do not know.
Based on your comments below, It appears to be a moot
point.
> 
> > 2. On a machine with two IP addresses on the same nic (A and B). If
someone on
> > the internet makes a connection to IP address A. What is the source
address of
> > reply packets in the IP header diagram above? A? (I hope
that''s the case too.)
> 
> Yes, reply packets originate from the exact same IP address as the
> originating packets were sent to. At least as far as I understand the
> IP RFC''s
> 
> > 3. Finally, is it necessary for the router box above to have two
addresses on
> > the internal network. Does it matter that the default gateway that
packets,
> > having a 204.. source address, go to might be a 63.. address? (or vice
versa, as
> > long as all machines are connected to the same physical network) If it
doesn''t
> > then there is no need for a second IP address on the internal side of
the router
> > (eth3 above). If it does matter, can I put in some fancy next-hop
routes on the
> > servers based on what the source address of reply packets ends up
being? (This
> > would essentially completely ignore the default route)
> 
> As long as there is a route from the nodes having a 204... address via the
> router, the IP address of the router does not matter. Let me clarify a
> little more. Suppose the Router only has a 63... address on the eth3
> interface. To route 204 addresses from the internal servers you would set
it
> up so that *all* addresses *but* the address of that specific internal
server
> are routed via the Router. In the Router you can then decide where to send
> the packets. They will have the correct source and destination addresses,
so
> that is not a big problem.
Wonderful... 
> 
> > Something to remember (it may or may not matter to you): Valid packets
coming in
> > on eth2 (204... above) will have a 204... destination address. Also
Valid
> > packets coming in on eth0 (63... above) will have a 63... destination
address.
> > Marking the packets as they come in is merely so that we can route
them back on
> > the same interface they came in on. I don''t want assymetric
routes.
> 
> Oh, but in that case you won''t even need to do the marking. The
information
> you need is already fully encapsulated in the IP addresses inside the
> packet. Ofcourse, I''m assuming there''s no nasty machine
doing dastardly
> deeds on that internal network of yours... :)
You''re right. Since packets get responded to on the exact same address
they came
in on, All I have to do in the router is look at the source address. If it is a
204.. address then route it out the 204.. pipe. If it is a 63.. address then
route it out the 63... pipe. Cool!

Hey, I''m working on the rules and routes to implement what
I''ve been talking
about, and I''ve got a small question about the ip rule add.

I''m trying to add a blackhole route, and ip rule add seems to insist
that I
provide a lookup table with the route.

For instance If I execute the command:

"ip rule add from 0.0.0.0/0 type blackhole."

when I look at what it did with "ip rule ls" I see

"from all lookup main blackhole"

The only way it seems to get rid of main in the example above is
to define a dummy table and re-add the route like so:

"ip rule add from 0.0.0.0/0 lookup bit-bucket type blackhole."

Then when I see what''s done with "ip rule ls" it says:

"from all lookup bit-bucket blackhole"

I suppose that''s OK. Just seems a bit stilted. Does it matter that the
bit-bucket table might not contain any routes? In the case of a blackhole route,
does the kernel even perform the table lookup? Would leaving the default main in
have been OK?

-Andrew

Warren Flemmer wrote:
> 
> Greetings
> 
> I am new to iproute2 and therefore would not try to give a direct answer to
> any of your questions. I have, however, been working on what seems to me to
> be a similar problem and will offer it here in case it assists.
> 
> I two have two links to the internet though different isps. The
requirements
> were that users on the lan would be as oblivious as possible to any one
link
> going down.
> 
> The solution I came up with involved two new linux boxes. Both using
iproute
> and one using masq (nat with 2.4 when released). One box was placed between
> the two isps with source routing and a third network card linking to the
> dns,www etc (I call it a crossrouter). The second box also links the two
> isps and uses source routing and masq to offer redundancy (redundancy box).
> An attempt at text art may help
> 
> isp1                     +redundancy box+                         isp2
> |                            |                             |
> |
> +---------------------+                           +---------------------+
>                              |                             |
>                              +   Crossrouter     +
>                                             |
>      ---+---------+---------------+----------------------------------Lan
>       www      dns                              etc
> 
> I made may attempts to use one box instead of two without any luck. I had
> bad results with marking that would have made a single box solution
> possible. The result was the two boxes. On the routers to the isp I have a
> second default route to the redundancy box with a higher matrix. If one
link
> fails the data is routed to the redundancy box where it is masq on an ip
> (one assigned by the isp with the good link) address and sent out through
> the other isp. Every test I have done is showing the solution to work but
it
> has not been fully deployed yet.
> 
> I would imagine that this type of solution would be worth using is you link
> to two different isps and BGP is not available. I intend doubling up on the
> crossrouter as it will become a single point of failure.
> 
> If anyone knows of a better solution I would be interested to know.
> 
> Hope this helps
> 
> Regards
> Warren
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/
I think in this case, you may be able to do what you want with a single box,
unless you need to route based on fwmark or something set in ipchains.

If all you need to base the information on is source address, or something
already in the packet, a single box solution should be feasible.

Can you elaborate on your current setup?  How/what detects that the ISP has gone
down?  What switches you to forward to the other router?  Are you running any
routing/status_monitoring daemons?

So far, the only way I''ve been able to detect that the ISP is down is a
ping.

Andrew wrote:
> 
> Hey, I''m working on the rules and routes to implement what
I''ve been talking
> about, and I''ve got a small question about the ip rule add.
> 
> I''m trying to add a blackhole route, and ip rule add seems to
insist that I
> provide a lookup table with the route.
> 
> For instance If I execute the command:
> 
> "ip rule add from 0.0.0.0/0 type blackhole."
> 
> when I look at what it did with "ip rule ls" I see
> 
> "from all lookup main blackhole"
> 
> The only way it seems to get rid of main in the example above is
> to define a dummy table and re-add the route like so:
> 
> "ip rule add from 0.0.0.0/0 lookup bit-bucket type blackhole."
> 
> Then when I see what''s done with "ip rule ls" it says:
> 
> "from all lookup bit-bucket blackhole"
> 
> I suppose that''s OK. Just seems a bit stilted. Does it matter that
the
> bit-bucket table might not contain any routes? In the case of a blackhole
route,
> does the kernel even perform the table lookup? Would leaving the default
main in
> have been OK?
> 
> -Andrew
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/
Why not use the "default" table?  It''s empty anyway, it just
seems "right".
Anyone know something about this I haven''t thought of??

On Thu, 16 Nov 2000, Andrew wrote:
> Hey, I''m working on the rules and routes to implement what
I''ve been talking
> about, and I''ve got a small question about the ip rule add.
> I''m trying to add a blackhole route, and ip rule add seems to
insist that I
> provide a lookup table with the route.
> 
> For instance If I execute the command:
> 
> "ip rule add from 0.0.0.0/0 type blackhole."
>
> when I look at what it did with "ip rule ls" I see
> 
> "from all lookup main blackhole"
Try using   ip rule add priority 256 from 0.0.0.0/0 type blackhole

Doei, Arthur. (There''s some hints as to the priority being required in
the
               IP command reference)

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Arthur van Leeuwen wrote:
> 
> On Thu, 16 Nov 2000, Andrew wrote:
> 
> > Hey, I''m working on the rules and routes to implement what
I''ve been talking
> > about, and I''ve got a small question about the ip rule add.
> 
> > I''m trying to add a blackhole route, and ip rule add seems to
insist that I
> > provide a lookup table with the route.
> >
> > For instance If I execute the command:
> >
> > "ip rule add from 0.0.0.0/0 type blackhole."
> >
> > when I look at what it did with "ip rule ls" I see
> >
> > "from all lookup main blackhole"
> 
> Try using   ip rule add priority 256 from 0.0.0.0/0 type blackhole
> 
> Doei, Arthur. (There''s some hints as to the priority being
required in the
>                IP command reference)
ip rule add lookup 256 from 0.0.0.0/0 type blackhole

or replace "lookup" with "table", same thing.

If you use priority, you are affecting the location of that rule in the rule
search.  In the above case, it should blackhole everything, unless there are any
"ip rule"s with a priority below 256.  This is in theory, I
didn''t want to try
it on my one linux box.

I rely on the routers to detect the downed link. If the route through the
serial is default and down then the second default (matrix 2) is used (this
has been tested on intel routers, and I am yet to see the intel do something
that the cisco can''t).
The Single Box: I tried and tried and tried. Eventually I went to two boxes
and the problem as solved.
Reason for 2 boxes.
The crossrouter does source routing based on which source ip is used (If the
ip of the source belongs to isp1 then the data should go out though isp1,
the data could quite easily go through isp2 but would return through isp1,
making it imperitive that both links are up). This makes load balancing
possible (for me at least), my dial-ins are assign alternating ips from the
different isps. Since I rely on the routers to handle the routing for a
downed link, if the data is sent back to the same box, the box has no idea
the link is down and it would send it back to the same downed link, causing
a loop. I initially thought that marking data as it went out would enable
the one box to adjust the routes (route based on fwmark) if the same marked
data was retrieved, but I could not get this to work. The second box
(redundancy) instead gets the data from the router (if link down). It then
MASQ the data on the other isps ip and sends the data out.
My first impression was that 1 one box solution was feasible, but without
some way of knowing the data has been sent back by the router this is not
possible. New features in kernel 2.4 may permit this. If I managed to get
the marking to work (I managed to mark data and send data based on mark but
the returned data from the router was not always marked!), one box might
have been possible.
Even if a one box solution were possible I think I may stay with the two
boxes for now. I intend to upgrade to kernel 2.4 and use SNAT instead of
MASQ. The box I use is old and out of date for any real processing function
but works great for the app. Keeping the boxes separate would make upgrading
easy. The solution is also considerably simpler and easier to test.
-----Original Message-----
From: Mike Fedyk <mfedyk@matchmail.com>
To: Warren Flemmer <warren@netlab.co.za>
Cc: LARTC@mailman.ds9a.nl <LARTC@mailman.ds9a.nl>
Date: Friday, November 17, 2000 2:04 PM
Subject: Re: [LARTC] A complicated routing scenario (for me at least)
>Warren Flemmer wrote:
>>
>> Greetings
>>
>> I am new to iproute2 and therefore would not try to give a direct
answer
to
>> any of your questions. I have, however, been working on what seems to
me
to
>> be a similar problem and will offer it here in case it assists.
>>
>> I two have two links to the internet though different isps. The
requirements
>> were that users on the lan would be as oblivious as possible to any one
link
>> going down.
>>
>> The solution I came up with involved two new linux boxes. Both using
iproute
>> and one using masq (nat with 2.4 when released). One box was placed
between
>> the two isps with source routing and a third network card linking to
the
>> dns,www etc (I call it a crossrouter). The second box also links the
two
>> isps and uses source routing and masq to offer redundancy (redundancy
box).
>> An attempt at text art may help
>>
>> isp1                     +redundancy box+                         isp2
>> |                            |                             |
>> |
>> +---------------------+                          
+---------------------+
>>                              |                             |
>>                              +   Crossrouter     +
>>                                             |
>>     
---+---------+---------------+----------------------------------Lan
>>       www      dns                              etc
>>
>> I made may attempts to use one box instead of two without any luck. I
had
>> bad results with marking that would have made a single box solution
>> possible. The result was the two boxes. On the routers to the isp I
have
a
>> second default route to the redundancy box with a higher matrix. If one
link
>> fails the data is routed to the redundancy box where it is masq on an
ip
>> (one assigned by the isp with the good link) address and sent out
through
>> the other isp. Every test I have done is showing the solution to work
but
it
>> has not been fully deployed yet.
>>
>> I would imagine that this type of solution would be worth using is you
link
>> to two different isps and BGP is not available. I intend doubling up on
the
>> crossrouter as it will become a single point of failure.
>>
>> If anyone knows of a better solution I would be interested to know.
>>
>> Hope this helps
>>
>> Regards
>> Warren
>>
>> _______________________________________________
>> LARTC mailing list / LARTC@mailman.ds9a.nl
>> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/
>
>I think in this case, you may be able to do what you want with a single
box,
>unless you need to route based on fwmark or something set in ipchains.
>
>If all you need to base the information on is source address, or something
>already in the packet, a single box solution should be feasible.
>
>Can you elaborate on your current setup?  How/what detects that the ISP has
gone
>down?  What switches you to forward to the other router?  Are you running
any
>routing/status_monitoring daemons?
>
>So far, the only way I''ve been able to detect that the ISP is down
is a
ping.

Season''s Greetings:

If anyone is still watching this thread, I''m still working on my setup.
Today I
have a very specific question:

I execute the following command at the prompt:

ip rule add from 100.100.100.80/29 to 10.5.5.0/24 iif eth0 lookup dmz-lan
priority 230

Then I run the following:
>ip rule ls
Which among other things lists the following for the rule I just added above:
>230:  from 100.100.100.80/24 to 10.5.5.0/24 iif eth0 lookup dmz-lan  (it IS
typed correctly)
Furthermore if I try to run the following going off the output of ip rule ls
above:
>ip rule delete from 100.100.100.80/24 to 10.5.5.0/24 iif eth0
I get the following answer:
>RTNETLINK answers: no such process
But if I run the command modified slightly as follows:
>ip rule delete from 100.100.100.80/29 to 10.5.5.0/24 iif eth0
It works (deletes the rule) and ip rule ls verfies that it no longer exists.


What is going on?! It seems to me that ip rule ls should match what I put in
with ip rule add.
Why isn''t it?

BTW. all the above IP addresses are completely bogus.

Thanks for the help!

-Andrew
-- 
depaan@bibleinfo.com
--------------------------------------------------------------
Want answers to life''s big questions? Visit www.bibleinfo.com.