LARTC - Aug 2007 - Dead Gateway Detection & BGP

Greetings to all,

 

To start I’ll firstly lay down the foundation to what I have done so far and
if those of you on the list can provide further insight, tips, links etc.

 

This scenario consists of 2 firewalls (both running Debian “etch”), 2 Cisco
routers (unsure of model numbers) connected together like so in the diagram
below.

 

 

 
-----------------------

 
|  Uplink Provider  |

 
-----------------------

 
|

 
|

 
-----------------------

 
|                                    |

 
-------------------    --------------------

                                                                | Cisco
Router  |   |  Cisco Router   |

 
------------------      --------------------

 
|                                    |

 
|                                    |

 
-------------------    --------------------

                                                                |
Firewall 1     |   |      Firewall 2     |

 
-------------------     --------------------

 

Initially, the first task I was designated was to setup BGP routing on 2
firewalls.  Each firewall is connected to its own Cisco router provided by
the uplink provider and the uplink provider is only providing a default
gateway/router to each of the firewalls.  Now, having had minimal experience
with BGP (minimal in terms of the broadness of what is possible with BGP)
and using the information provided by the uplink provider I have setup BGP.

 

What I have been recently informed of is that the 2 firewalls must do some
sort of failover between them when either of the default gateway’s are no
longer responsive.  I had initially looked into using heartbeat (which I am
still considering) to do the failover or possibly using vrrpd (Virtual
Router Redundancy Protocol Daemon).  This however isn’t what I am contacting
this list about.  What I need to do at minimal, is at least for the
failover, is to detect when the default gateway of (say) firewall 1 is no
longer available and perform failover to firewall 2 and vice versa.  As far
as  I am aware the only DGD support available is still through the patches
that Julian Anastasov wrote for the 2.4 kernel series or by writing a script
that uses arping to determine the last hop available. 

 

What other options are there?

 

I have done a fair amount of searching the internet only to come back to
these 2 possibilities.  Surely there must be something else ….

 

Thanks in advance to anyone that replies as I know that this topic seems to
be coming up more and more frequently on the lists and must be getting
somewhat tedious for most.

 

Regards,

 

Rangi


No virus found in this outgoing message.
Checked by AVG Free Edition. 
Version: 7.5.484 / Virus Database: 269.12.8/973 - Release Date: 8/25/2007
5:00 PM
 


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

On 08/26/07 12:29, Rangi Biddle wrote:
> Greetings to all,
> 
> To start I’ll firstly lay down the foundation to what I have done so 
> far and if those of you on the list can provide further insight, 
> tips, links etc.
> 
> This scenario consists of 2 firewalls (both running Debian “etch”), 2 
> Cisco routers (unsure of model numbers) connected together like so in 
> the diagram below.
> 
>          +-----------------+
>          | Uplink Provider |
>          +--------+--------+
>                   |
>         +---------+---------+
>         |                   |
> +-------+-------+   +-------+-------+
> | Cisco  Router |   | Cisco  Router |
> +-------+-------+   +-------+-------+
>         |                   |
> +-------+-------+   +-------+-------+
> | Firewall # 1  |   | Firewall # 2  |
> +---------------+   +-------+-------+
> 
> Initially, the first task I was designated was to setup BGP routing 
> on 2 firewalls.  Each firewall is connected to its own Cisco router 
> provided by the uplink provider and the uplink provider is only 
> providing a default gateway/router to each of the firewalls.  Now, 
> having had minimal experience with BGP (minimal in terms of the 
> broadness of what is possible with BGP) and using the information 
> provided by the uplink provider I have setup BGP.
> 
> What I have been recently informed of is that the 2 firewalls must do 
> some sort of failover between them when either of the default 
> gateway’s are no longer responsive.  I had initially looked into 
> using heartbeat (which I am still considering) to do the failover or 
> possibly using vrrpd (Virtual Router Redundancy Protocol Daemon). 
> This however isn’t what I am contacting this list about.  What I need 
> to do at minimal, is at least for the failover, is to detect when the 
> default gateway of (say) firewall 1 is no longer available and 
> perform failover to firewall 2 and vice versa.  As far as  I am aware 
> the only DGD support available is still through the patches that 
> Julian Anastasov wrote for the 2.4 kernel series or by writing a 
> script that uses arping to determine the last hop available.
In my experience, Julian''s DGD patch(s) are very good but not needed
for
your scenario.  I have achieved a very similar scenario with a stock 
kernel.  The main thing(s) that Julian''s patches do is provide Dead 
Gateway Detection for (this is the key point) "non-default" routes
while
the kernel its self is capable to providing this for default routes.
> What other options are there?
Add two equal metric default routes in reverse priority.  (It is my 
experience that the route command populates the routing table by pushing 
new routes on to the top to be read before other existing routes.)
> I have done a fair amount of searching the internet only to come back 
> to these 2 possibilities.  Surely there must be something else ….
Well, you are touching on some key points to what needs to be done, but 
there are still other things to be considered for a truly redundant 
scenario.
> Thanks in advance to anyone that replies as I know that this topic 
> seems to be coming up more and more frequently on the lists and must 
> be getting somewhat tedious for most.
You are welcome.



Grant. . . .

After talking with a colleague on the ethics of this message I (/ we) 
decided that I needed to make the same offer to everyone on this mailing 
list that I privately made to Rangi Biddle.

The company that I work for is in business to do many different things, 
included in which is helping with specialized configurations like I 
believe that Rangi Biddle is needing.  As such I offered to consult with 
Rangi Biddle for $1/min on what my company has done in the past to 
generate complete solutions not just pieces of the puzzle leaving Rangi 
Biddle to put them together on his own.

I my self and the company that I work for want to offer as much back to 
the community as it has offered to us.  As such I / we are willing to 
help point people in the right direction and show them some of the 
pieces to the puzzle.  However business being what it is I am not 
allowed to always provide the entire step by step how to guide for many 
different things.  My company has invested time and money in to being 
able to provide solutions using open source products for such things as 
load balancing a medium size network across multiple cable modems, 
redundant fail over routing for globally routable addresses, down to 
segmenting a multi tenant building so that tenants can not cross infect 
each other while sharing one single IP subnet.

I am curious what the community''s reaction is to this and ask for and 
encourage responses with regards to when is it appropriate for 
individuals / companies to move from "free to the public" support to 
"reasonable rate commercial support".

I apologize if my actions offended any one.  However, please if they 
did, contact me either on or off list as I would like to know why they did.



Thank you and have a nice day,

Grant Taylor
Systems Administrator
Riverview Technologies Inc.
2311 East Walnut
Columbia MO  65201
United States of America

Phone:  +1 (573) 442-7151
   Fax:  +1 (573) 442-3062
eMail:  gtaylor (at) riverviewtech (dot) net

Grant Taylor wrote:
> I my self and the company that I work for want to offer as much back to 
> the community as it has offered to us. 
> My company has invested time and money
> I am curious what the community''s reaction is to this and ask for
and
> encourage responses with regards to when is it appropriate for 
> individuals / companies to move from "free to the public" support
to
> "reasonable rate commercial support".
I for one can not speak for the community, but the three points 
highlighted above do not add up. Here is the scoring:

			Community	Your Company

Cost of help offered	  free		   paid
Time/money investment	  large		   large

			    2	    :        1

It is OK to charge for any provided service, good or bad. It is not OK 
to label this as "giving back as much as was offered".

Regards

Peter

On 8/27/2007 12:21 PM, Peter Rabbitson wrote:
> It is OK to charge for any provided service, good or bad. It is not OK 
> to label this as "giving back as much as was offered".
I''m not sure that I completely understand what you are trying to get
at,
therefore I can not comment correctly.

However, I was trying to imply that my company has spent time and money 
to develop a configuration (what) including the order in which things 
are configured in (how).  With the order of configuration (how) being 
more of our information that we are not eager to give up.  We are more 
than willing to list out the components (what) that were used and 
possibly even some of an order, but not all of the order.

With that being said, I think offering up the what for free with out the 
how (below) is fairly good while still protecting our time and money 
investment.

The "what" would consist of the following:
  - Large over all block diagram.
  - List of modules used for each block.
  - List of optional modules used for each block.
  - Explanation of what each module does to fulfill the block.
  - Possibly some how or indicate to follow Read-Me(s).

The "how" would consist of the following:
  - How to configure each module to achieve the desired result.

The "how" is where our company has spent the most time and money to
get
things to work and achieve much larger projects.



Grant. . . .

On 8/27/2007 9:49 PM, Mohan Sundaram wrote:
> Such a service is a much needed complement to forums to aid adoption 
> of FOSS. I was doing this for a fairly long while as a knowhow 
> provider.
*nod*
> There is a very thin line one needs to walk. Forums being used to 
> vend services is frowned upon, rightly so. It is the concept of free 
> sharing that gets violated. Even when I was a consultant, I used to 
> offer complete advice to forums simply because it gave me 
> satisfaction. I''d learnt a lot from the forums and this was my way
of
> returning the coin.
Agreed.  Normally I do tend to offer up the complete solution, 
especially if said solution or one very similar can be found elsewhere 
on the net with a bit of Googleing.  However when the solution in 
question is that of something that was not readily available on the net 
and one that we spent a lot of time putting the puzzle pieces together 
we tend to hold on to some of it.
> There is a definite need and opportunity. Reasonable is dependent on 
> a lot of factors and the same service yields different values to 
> different customers.
Indeed.
> My philosophy: I think it is definitely possible to differentiate 
> between personal time and company time. It is like social work. If 
> you do something on your personal time that does not eat into your 
> co''s biz, I believe it is good to do so free. Even if you did do
it
> such, so long as you do not charge for it, I believe it is not 
> unethical.
I''m not sure what you are trying to get at there.  I think you are 
saying that if you do it as a personal time, then you probably should 
find some other sort of personal gratification.  If you do it as company 
time then it is more understandable if it is charged for.  Am I any 
where close?

I can see how trolling a forum / news group looking for people asking 
questions and posting multiple follow up posts only saying "the company 
that I work for can provide you with a solution for X $s" is not so 
good.  However if you are an active member of a forum / news group and 
offer advice and pointers in the right direction to the solution of the 
question and state that "the company I work for can probably help 
provide a more complete solution contact me if you are interested" is a 
bit different?

I''m not trying to argue any thing here, just completely understand what
you are saying and making sure that you understand what I''m saying 
(making sure that communications is happening both ways) while 
discussing this.

Thank you for taking time to reply to my post.



Grant. . . .

Hi Guys,

Well here''s my two cents worth regarding this whole thing.

Firstly I can appreciate where Grant is coming from.  There are a number of
things that aren''t so commonly done with Linux that the community
currently
doesn''t provide answers for and obviously there are people out there
that
know how to do things that the community cannot answer.  The issue I have
with what Grant wants to provide (re: $1/min rate via email) is that I have
no control over the amount of time that is spent writing an email or seeking
answers to my questions meaning I could spend $100''s if not
$1,000''s of
dollars getting a partial answer (not implying that that would be the case),
but is a point of concern.  I myself have been an active supporter of OSS
and have contributed code and answers to not so common questions or have
gone out of my way to assist others.  Unfortunately, in this instance, it is
I that am seeking help and am now being asked to pay for an answer to my
question.  Sounds somewhat like visiting a shrink.  In some instances, it
doesn''t quite surprise me that Linux isn''t more mainstream and
this being a
primary example of it.  If more of us knew how to do <insert task here> I
believe Linux would become more mainstream because there are more of us
available to actively support Linux systems which, as most of us are aware
of, is the primary concern of most that purchase a Linux solution "Who is
going to look after it if you''re not here or available?".

Bottom line is this, my boss refuses to pay someone that neither he nor I
know.  Primarily because this same person wants to provide a solution to us
for an indeterminate price and if there is an issue at any point we are left
with no way of knowing how to fix the issue and again be left with paying an
indeterminate price for further support.  What my boss is more happy to do
is pay for a commercial solution regardless of price.  It is mainly because
he is aware of what he must pay before he purchases the solution and also
because he knows that it will do what he wants including support if we have
an issue.  Obviously this would mean scrapping Linux out of the picture even
with the amount of high regard I give to it.

So Grant, I''ll put the ball back in your court.

Regards,

Rangi

-----Original Message-----
From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl]
On Behalf Of Grant Taylor
Sent: Wednesday, August 29, 2007 5:40 PM
To: Mail List - Linux Advanced Routing and Traffic Control
Subject: Re: [LARTC] Dead Gateway Detection & BGP

On 8/27/2007 9:49 PM, Mohan Sundaram wrote:
> Such a service is a much needed complement to forums to aid adoption 
> of FOSS. I was doing this for a fairly long while as a knowhow 
> provider.
*nod*
> There is a very thin line one needs to walk. Forums being used to 
> vend services is frowned upon, rightly so. It is the concept of free 
> sharing that gets violated. Even when I was a consultant, I used to 
> offer complete advice to forums simply because it gave me 
> satisfaction. I''d learnt a lot from the forums and this was my way
of
> returning the coin.
Agreed.  Normally I do tend to offer up the complete solution, 
especially if said solution or one very similar can be found elsewhere 
on the net with a bit of Googleing.  However when the solution in 
question is that of something that was not readily available on the net 
and one that we spent a lot of time putting the puzzle pieces together 
we tend to hold on to some of it.
> There is a definite need and opportunity. Reasonable is dependent on 
> a lot of factors and the same service yields different values to 
> different customers.
Indeed.
> My philosophy: I think it is definitely possible to differentiate 
> between personal time and company time. It is like social work. If 
> you do something on your personal time that does not eat into your 
> co''s biz, I believe it is good to do so free. Even if you did do
it
> such, so long as you do not charge for it, I believe it is not 
> unethical.
I''m not sure what you are trying to get at there.  I think you are 
saying that if you do it as a personal time, then you probably should 
find some other sort of personal gratification.  If you do it as company 
time then it is more understandable if it is charged for.  Am I any 
where close?

I can see how trolling a forum / news group looking for people asking 
questions and posting multiple follow up posts only saying "the company 
that I work for can provide you with a solution for X $s" is not so 
good.  However if you are an active member of a forum / news group and 
offer advice and pointers in the right direction to the solution of the 
question and state that "the company I work for can probably help 
provide a more complete solution contact me if you are interested" is a 
bit different?

I''m not trying to argue any thing here, just completely understand what
you are saying and making sure that you understand what I''m saying 
(making sure that communications is happening both ways) while 
discussing this.

Thank you for taking time to reply to my post.



Grant. . . .
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

No virus found in this incoming message.
Checked by AVG Free Edition. 
Version: 7.5.484 / Virus Database: 269.12.10/977 - Release Date: 8/28/2007
4:29 PM
 

No virus found in this outgoing message.
Checked by AVG Free Edition. 
Version: 7.5.484 / Virus Database: 269.12.10/977 - Release Date: 8/28/2007
4:29 PM

On 8/29/2007 8:50 PM, Rangi Biddle wrote:
> Firstly I can appreciate where Grant is coming from.  There are a 
> number of things that aren''t so commonly done with Linux that the 
> community currently doesn''t provide answers for and obviously
there
> are people out there that know how to do things that the community 
> cannot answer.  The issue I have with what Grant wants to provide 
> (re: $1/min rate via email) is that I have no control over the amount 
> of time that is spent writing an email or seeking answers to my 
> questions meaning I could spend $100''s if not $1,000''s of
dollars
> getting a partial answer (not implying that that would be the case), 
> but is a point of concern.  I myself have been an active supporter of 
> OSS and have contributed code and answers to not so common questions 
> or have gone out of my way to assist others.  Unfortunately, in this 
> instance, it is I that am seeking help and am now being asked to pay 
> for an answer to my question.  Sounds somewhat like visiting a 
> shrink.  In some instances, it doesn''t quite surprise me that
Linux
> isn''t more mainstream and this being a primary example of it.  If 
> more of us knew how to do <insert task here> I believe Linux would 
> become more mainstream because there are more of us available to 
> actively support Linux systems which, as most of us are aware of, is 
> the primary concern of most that purchase a Linux solution "Who is 
> going to look after it if you''re not here or available?".
With regards to the amount of time spent on the email(s), I had 
indicated that I expected to spend between 30 minutes and 180 minutes 
total helping.  Usually it takes me about 15 minutes or so to draft a 
detailed email and re-reading / editing it before I send it.  Indeed 
there are a lot of short one liners that take all of 30 seconds to send 
too.  So, I don''t think that there is concern with spending any ware 
near $1,000''s of dollars.  Even after all was said and done, I would 
probably negotiate with you to make sure that what I initially proposed 
to you (or any one else for that matter) was mutually fair, if any thing 
erroring on the low side to make sure that things were fair.

I''m sorry for even remotely making you feel as if you have to pay for
an
answer to your question(s), I was not trying to imply that at all.  At 
the time that I had wrote that I was dealing with a particularly 
difficult problem that I had just spent numerous hours of my personal / 
company time (distinctions are *VERY* gray seeing as how my job is the 
same thing as my hobby).  I would have happily payed what I considered 
to be a nominal rate to be able to talk with someone about what I was 
wanting to accomplish rather than working all those hours.

Look for a follow up email to your original post with more of an answer 
to your question shortly.  At least it will contain what I would us to 
achieve what you are wanting to do, in so far as the logical blocks to 
your problem, not specific configuration instructions, which I leave up 
to an exercise for an educated person (being any one that can read 
readme files and think logically about networking and run a compiler). 
With contrast if I was doing this for a client as I had initially 
offered I would most likely end up giving much closer to step by step 
instructions including how to configure what interface and what MAC 
address to put where rather than leaving it up to said educated individual.
> Bottom line is this, my boss refuses to pay someone that neither he 
> nor I know.  Primarily because this same person wants to provide a 
> solution to us for an indeterminate price and if there is an issue at 
> any point we are left with no way of knowing how to fix the issue and 
> again be left with paying an indeterminate price for further support. 
> What my boss is more happy to do is pay for a commercial solution 
> regardless of price.  It is mainly because he is aware of what he 
> must pay before he purchases the solution and also because he knows 
> that it will do what he wants including support if we have an issue. 
> Obviously this would mean scrapping Linux out of the picture even 
> with the amount of high regard I give to it.
Ah, I think there is some more ambiguity showing through there.  I can 
completely understand you and your bosses lack of willingness to blindly 
enter in to a business arrangement.  First keep in mind that what was 
originally discussed / proposed is not a contractual agreement, simply 
and invitation to discuss things further to see if each party would be 
interested in doing business.  More of a "Hay, here is what I can do, 
call me if you would like more details." type thing.  With regards to 
the indeterminate amount, to me that is not as much as an issue that 
some might think at present because I do not know the true nature of 
what you are trying to accomplish nor have you heard my follow up 
responses that may provide a much better over all solution.  Once we had 
spoken and discussed such things there would be a much more firm 
estimate and / or range of expected time to do what ever as well as 
check points that either side of the agreement could back out gracefully 
with as little egg on their face as possible.

As far as being worried that some consultant would come in and change 
things with out your knowledge (of the reasoning behind the change) or 
consent, in short "That would *NEVER* happen!" as it is quite simply 
unethical.  Myself and my company would much rather help educate you 
along the way so that you can make the changes your self thus learn what 
needed to be done and why and how it effects things.  Thus you would be 
the one doing the work while knowing how to do it and how to support it 
in the long run.  I see my (companies) role in this as a guiding hand 
pointing you in the right direction and as a sounding board to discuss 
what really is the proper thing to do.  That is not to say that I would 
not be willing to log in to systems and make change, though there would 
have to be a very well established relationship prior to any thing 
remotely like that.  I would much rather help educate you so that you 
can do things your self.

I personally would hate to see you have to scrap Linux or any other open 
source solution just because your company does not have the in house 
knowledge set to take full advantage of open source software.
> So Grant, I''ll put the ball back in your court.
I apologize if the first pitch seemed to be a curve and / or knuckle 
ball.  I was more going after a slow pitch softball with a note saying 
that I could offer more tailored support out side of the scope of this 
mailing list verses the more generic support that is usually found here. 
  I.e. what we would do off mailing list would include me having a 
fuller understanding of your network structure including host names and 
interface configurations so that all communications can use such 
information to be as thorough as possible verses the "System A" and 
"System B" approach which is left open to so much interpretation.

Please let me know what you think of this (hopefully) underhanded slow 
pitch softball.  ;)
> Regards,
Likewise.



Grant. . . .

(Before any one questions why I withheld information and went down the 
road that I did, I''d like to say that I had fully intended to respond 
with more detail, however other things going on both at work and home 
prevented me from doing so before now.  I also sort of paused because of 
the discussion that arose out of the road that I did go down.)

On 8/26/2007 12:29 PM, Rangi Biddle wrote:
>          +-----------------+
>          | Uplink Provider |
>          +--------+--------+
>                   |
>         +---------+---------+
>         |                   |
> +-------+-------+   +-------+-------+
> | Cisco  Router |   | Cisco  Router |
> +-------+-------+   +-------+-------+
>         |                   |
> +-------+-------+   +-------+-------+
> | Firewall # 1  |   | Firewall # 2  |
> +---------------+   +-------+-------+
> 
> Initially, the first task I was designated was to setup BGP routing 
> on 2 firewalls.  Each firewall is connected to its own Cisco router 
> provided by the uplink provider and the uplink provider is only 
> providing a default gateway/router to each of the firewalls.  Now, 
> having had minimal experience with BGP (minimal in terms of the 
> broadness of what is possible with BGP) and using the information 
> provided by the uplink provider I have setup BGP.
Question:
  - Are there multiple providers in this situation or one single 
provider that has chosen to do this type of set up.
  - If there are multiple providers, are they in any sort of peering 
relationship between them?
  - Is there suppose to be any sort of redundancy amongst the two Cisco 
routers or are they to be two purely independent non redundant connections?
  - What type of connections are there in to the two Cisco routers?
  - Are the Cisco routers actually routing, or just bridging between two 
layer 1 technologies?
  - Is ethernet being used between the Cisco routers and the Debian 
firewalls?
  - What type of (if any) IP address range overlap are we looking at?

Answers to each of these questions will most likely beget more questions 
until finally a much clearer picture of what ultimately is being done 
emerges.  This is also part of why I was wanting to do this off mailing 
list as some of these answers are not appropriate for a public form that 
is archived and search able.
> What I have been recently informed of is that the 2 firewalls must do 
> some sort of failover between them when either of the default 
> gateway’s are no longer responsive.  I had initially looked into 
> using heartbeat (which I am still considering) to do the failover or 
> possibly using vrrpd (Virtual Router Redundancy Protocol Daemon). 
> This however isn’t what I am contacting this list about.  What I need 
> to do at minimal, is at least for the failover, is to detect when the 
> default gateway of (say) firewall 1 is no longer available and 
> perform failover to firewall 2 and vice versa.  As far as  I am aware 
> the only DGD support available is still through the patches that 
> Julian Anastasov wrote for the 2.4 kernel series or by writing a 
> script that uses arping to determine the last hop available.
Hum.  I''m not entirely sure what is suppose to be redundant here, the 
Cisco routers, the Debian firewalls, a logical router (or routers) that 
are presented to your systems behind the firewalls, what.  Will you 
please clarify?
> What other options are there?
More than you might initially think.
> I have done a fair amount of searching the internet only to come back 
> to these 2 possibilities.  Surely there must be something else ….
Well, in my opinion, what you have proposed is a couple of different 
solutions to the same piece of the puzzle.

Presuming that you are dealing with T-1s from your provider(s), let''s 
start with a modified version of your above network layout.

          +-----------------+
          | Uplink Provider |
          +--------+--------+
                   |
         +---------+---------+
         |                   |
+-------+-------+   +-------+-------+
|   Atlas 550   +---+   Atlas 550   |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
| Cisco  Router +---+ Cisco  Router |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
|    Switch     +---+    Switch     |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
| Firewall # 1  +---+ Firewall # 2  |
+-------+---+---+   +---+---+-------+
         |   |           |   |
         |    \         /    |
         |     \       /     |
         |      \     /      |
         |       \   /       |
         |        \ /        |
         |         X         |
         |        / \        |
         |       /   \       |
         |      /     \      |
         |     /       \     |
         |    /         \    |
         |   |           |   |
+-------+---+---+   +---+---+-------+
|    Switch     +---+    Switch     |
+-------+-------+   +-------+-------+
         |                   |
    ...--+--...--(LAN)--...--+--...

Now that the ASCII art is out of the way, let''s have some explanation
as
to what each piece of the puzzle is for.

Physical Layer
--------------

The "Atlas 550"s are devices to switch / route T-1 on a phone company
/
circuit level.  In other words they can take a T-1 in and give a T-1 out 
based on different conditions with in the circuit on a given interface. 
  In short the Atlas 550 will allow you to route an inbound T-1 the 
primary interface if the equipment that the primary interface is 
connected to is up and handling traffic.  If the equipment that the 
primary interface connected to is not up and handling traffic route the 
T-1 out the secondary interface.  If for some reason the equipment that 
the secondary interface is connected to is not handling traffic route 
the T-1 out the tertiary interface to the backup Atlas in hopes that the 
cabling between the original Atlas and the primary and secondary 
equipment is down and that the backup Atlas has functioning cable.

The Cisco routers are similarly configured with two T-1 WICs each so 
that each can connect to both Atlas 550s.  Also there is a similar setup 
between the Cisco routers and the ethernet switches and each other.

Likewise the switches have a similar set up to connect to the firewall 
boxen as well as the firewall boxen do to the internal LAN switch(es).

Data Layer
----------
Each Atlas 550s can redundantly route their inbound T-1 to two different 
routers configured redundantly for each other or to the other Atlas 550.

Each Cisco router can redundantly route their inbound T-1s to two 
different switches configured redundantly for each other or to the other 
router.

Each switch can redundantly switch their inbound network segments to two 
different firewalls configured redundantly for each other or to the 
other switch.

Each firewall can redundantly filter their inbound network segments to 
two different switches configured redundantly for each other or to the 
other firewall.

Each switch can redundantly switch their inbound network segment to the 
internal LAN or to the other switch.

Network Layer
-------------
Each Atlas 550 would be configured to be able to handle the others T-1 
in the event that the other is unable to reach its desired router.

Each Cisco router would be configured to be able to handle the other 
routers circuit in addition to its own circuit, thus you could have a 
Cisco router die with out adversely effecting your network.  If I could, 
I would probably use HSRP or VRRP between the Cisco routers so that they 
could be redundant for each other.

Each switch is used for basic network connectivity allowing for more 
intermediary equipment.  If this is the only equipment you are going t 
have you could take the core switches out of the mix and go from the 
Cisco routers straight in to the firewalls.  However these switches will 
allow for more future expansion and other options down the road.  For 
example, either of the switches, if managed, would allow you to mirror 
traffic from one port to another for sniffing.

Each firewall would be able to filter traffic for its primary circuit as 
well as backup filter for the other firewalls backup circuit.  I would 
use VRRP to allow multiple physical firewalls to be redundant for each 
others IP address.  For example, make firewall A be primary for IP 1 and 
secondary for IP 2 while making firewall B be primary for IP 2 and 
secondary for IP 1.  Thus each firewall is redundant on its WAN facing 
side.  Do something similar for the LAN facing side.  If you decide that 
one connection from your provider is primary and the other is backup, 
you could route inbound traffic through one firewall while routing 
outbound traffic through the other firewall for load balancing / 
distribution reasons.  If you have the ethernet switches in place you 
could even insert a third firewall ans an inactive backup system to be 
used if either of the primary systems go down.  I would recommend that 
you use ConnTrackd to synchronize the firewall state between the two (or 
more) firewalls.

Each switch is used to allow connectivity between the two (or more) 
firewalls with the internal LAN.

As you can see there really is not a single point of failure between 
where the provider leaves off and the workstations pick up.
> Thanks in advance to anyone that replies as I know that this topic 
> seems to be coming up more and more frequently on the lists and must 
> be getting somewhat tedious for most.
*nod*
> Regards,
*nod*

Chew on this and let me know what you think.



Grant. . . .