CentOS - Aug 2005 - OT: CentOS server with 2 GbE links to 2 GbE switches

Hi all,

I am trying to come up with an architecture that has some redundancy.
The idea is to hook up the two GbE LAN interfaces of a CentOS server to
two Gigabit Ethernet switches. In case one switch goes down, there is a
redundant path (the server is redundant too). Here is the idea:

                             -----------
                            |    GbE    |   
 PCs            ------------|   switch  |------------
  |            |             -----------             |
  |   -----------------                        -----------------
  ---| Workgoup Switch |                      | CentOS/Asterisk |
  |   -----------------                        -----------------
  |            |             -----------             |
 VoIP           ------------|    GbE    |------------
Phones                      |   switch  |   
                             -----------

How would I acomplish this? Can I use IP addresses from one IP network
(say 10.0.0.0/24) to assign to the 2 LAN ports on the CentOS server and
a port on each of the GbE switches and then use something like OSPF on
the switches and the CentOS box to do the routing? Any other ideas?

Many thanks for your suggestions.

Regards,
Patrick

Have a look at 

http://linux-ip.net/html/ether-bonding.html#ether-bonding-ha 

You can only use one link at a time. The second nic is for fail-over.

* Jeff
>>> centos at puzzled.xs4all.nl 8/26/2005 8:36:42 AM >>>
Hi all,

I am trying to come up with an architecture that has some redundancy.
The idea is to hook up the two GbE LAN interfaces of a CentOS server to
two Gigabit Ethernet switches. In case one switch goes down, there is a
redundant path (the server is redundant too). Here is the idea:

                             -----------
                            |    GbE    |   
 PCs            ------------|   switch  |------------
  |            |             -----------             |
  |   -----------------                        -----------------
  ---| Workgoup Switch |                      | CentOS/Asterisk |
  |   -----------------                        -----------------
  |            |             -----------             |
 VoIP           ------------|    GbE    |------------
Phones                      |   switch  |   
                             -----------

How would I acomplish this? Can I use IP addresses from one IP network
(say 10.0.0.0/24) to assign to the 2 LAN ports on the CentOS server and
a port on each of the GbE switches and then use something like OSPF on
the switches and the CentOS box to do the routing? Any other ideas?

Many thanks for your suggestions.

Regards,
Patrick
_______________________________________________
CentOS mailing list
CentOS at centos.org 
http://lists.centos.org/mailman/listinfo/centos

Patrick wrote:
>Hi all,
>
>I am trying to come up with an architecture that has some redundancy.
>The idea is to hook up the two GbE LAN interfaces of a CentOS server to
>two Gigabit Ethernet switches. In case one switch goes down, there is a
>redundant path (the server is redundant too). Here is the idea:
>
>                             -----------
>                            |    GbE    |   
> PCs            ------------|   switch  |------------
>  |            |             -----------             |
>  |   -----------------                        -----------------
>  ---| Workgoup Switch |                      | CentOS/Asterisk |
>  |   -----------------                        -----------------
>  |            |             -----------             |
> VoIP           ------------|    GbE    |------------
>Phones                      |   switch  |   
>                             -----------
>
>  
>

Why don't you just bond the two interfaces?  An alternative would be to 
use one interface as a primary link and periodically check its 
"hearbeat".  If the interface goes down, you just add the route for
the
other interface to your default gateway.  Bonding the 2 channels is 
probably the easier method, though.

Cheers,

Patrick <centos at puzzled.xs4all.nl> wrote:
> I am trying to come up with an architecture that has some
> redundancy.  The idea is to hook up the two GbE LAN
> interfaces of a CentOS server to two Gigabit Ethernet
> switches. In case one switch goes down, there is a
> redundant path (the server is redundant too).
> How would I acomplish this?
First off, doing it at the layer-3/IP level with dynamic
routes is far more overhead than is required.  In your case,
you're just looking for layer-2/802 level.  So leverage what
standard 802 offers if you can.

I'm more of an academic, so the first thing I recommend to
people is that they get familar with the standard
capabilities of 802.  More explicitly, research 802.1d
Spanning Tree Protocol (STP) as well as newer standards like
802.3ad Link Aggregation.  In fact, it's this latter addition
that really makes things very easy.

In the "good old days," you'd setup a single, virtual UNIX
interface bridged to two.  Your system only knows about the
single, virtual UNIX interface.  But it would then leverage
two interfaces, only bringing the other interface up if one
failed.  With support for STP, loops would be avoided.  The
only thing to worry about with STP is the maximum number of
hops in a layer-2 network -- 7.  This, of course, requires
both your host (software) and network stack (firmware) to
support STP.

In the "new, better days" we now have 802.3ad Link
Aggregation.  Now you can get more bandwidth and failover at
the same time.  Again, both your host (NIC firmware) and
network stack (firmware) need to support 802.3ad Link
Aggregation.  But if it does, it becomes very, very easy to
configure a single IP address to a pair of NICs, and
aggregate both to two different ports in a network stack.

Now if you're using cheap network equipment, I don't know
what to tell you.  Layer-2 is probably out then.
> Can I use IP addresses from one IP network (say
10.0.0.0/24)
> to assign to the 2 LAN ports on the CentOS server and a
port
> on each of the GbE switches and then use something like
OSPF
> on the switches and the CentOS box to do the routing?
You can maybe use layer-3 hacks and tweaks to deal with
things, but it's very difficult to handle the failover
without support at the concentrator end.  Your idea to use
different IP addresses and dynamic routing tables is probably
the best way.  But it's definitely not as clean.

Especially with 802.3ad Link Aggregation being supported more
and more.
> Any other ideas?
If you let me know what your networking equipment and/or
budgetary constraints are, I can help you further.  You'd be
surprised how little this actually costs, but if you're using
$200 GbE switches, then I can't help you with layer-2.


-- 
Bryan J. Smith                | Sent from Yahoo Mail
mailto:b.j.smith at ieee.org     |  (please excuse any
http://thebs413.blogspot.com/ |   missing headers)

On Fri, 2005-08-26 at 15:36 +0200, Patrick wrote:
> Hi all,
> 
> I am trying to come up with an architecture that has some redundancy.
> The idea is to hook up the two GbE LAN interfaces of a CentOS server to
> two Gigabit Ethernet switches. In case one switch goes down, there is a
> redundant path (the server is redundant too). Here is the idea:
> 
>                              -----------
>                             |    GbE    |   
>  PCs            ------------|   switch  |------------
>   |            |             -----------             |
>   |   -----------------                        -----------------
>   ---| Workgoup Switch |                      | CentOS/Asterisk |
>   |   -----------------                        -----------------
>   |            |             -----------             |
>  VoIP           ------------|    GbE    |------------
> Phones                      |   switch  |   
>                              -----------
> 
> How would I acomplish this? Can I use IP addresses from one IP network
> (say 10.0.0.0/24) to assign to the 2 LAN ports on the CentOS server and
> a port on each of the GbE switches and then use something like OSPF on
> the switches and the CentOS box to do the routing? Any other ideas?
> 
> Many thanks for your suggestions.
> 
> Regards,
> Patrick
Hi Patrick,

I run a HA(High Availability) technique that I developed myself whereas
2 servers are redundant syncing up bout every 15-20 seconds. Basically
mine is IP oriented instead of machine oriented...either machine can be
the master and will stay the master until the slave deems the master as
having problems and not being able to handle the server
responsibilities. When the slave decides this, it then grabs the roaming
IP and turns on the servers and it is then the master.

If a redundant HA server is machine oriented, that means that, yes, the
slave will take over but as soon as the master comes back on line, the
master takes back the roaming ip and starts up the servers and the slave
machine will assume slave responsibilities again.

these are just some ideas of what you are looking for. Also, google
around using High Availability as a keyword...a bunch out there I think,
I am just about to the point to start building up my web-site where I
will cover my system in depth.

Incidently, over the last month or so, the main server seems to loose it
and decides to reboot in which case, the slave takes over...I have never
been around when it actually happened and sometimes I never even
realized it for a day or so. It's kinda seamless and the switch takes
bout 20 seconds. My point, I do know it works as it should!

John Rose

On Fri, 2005-08-26 at 09:36, Patrick wrote:
> Hi all,
> 
> I am trying to come up with an architecture that has some redundancy.
> The idea is to hook up the two GbE LAN interfaces of a CentOS server to
> two Gigabit Ethernet switches. In case one switch goes down, there is a
> redundant path (the server is redundant too). Here is the idea:
> 
>                              -----------
>                             |    GbE    |   
>  PCs            ------------|   switch  |------------
>   |            |             -----------             |
>   |   -----------------                        -----------------
>   ---| Workgoup Switch |                      | CentOS/Asterisk |
>   |   -----------------                        -----------------
>   |            |             -----------             |
>  VoIP           ------------|    GbE    |------------
> Phones                      |   switch  |   
>                              -----------
> 
> How would I acomplish this? Can I use IP addresses from one IP network
> (say 10.0.0.0/24) to assign to the 2 LAN ports on the CentOS server and
> a port on each of the GbE switches and then use something like OSPF on
> the switches and the CentOS box to do the routing? Any other ideas?
> 
> Many thanks for your suggestions.
The setup you describe has several single points of failure.  Are the
GbE switches you are using that fragile and likely to fail?



In the network you describe above the workgroup switch and the Asterisk
box are single points of failure.  If you want a redundant system then
you need to eliminate the single points of failure.  You may want to
look at using HSRP or VRRP (HSRP is Cisco specific, VRRP is more
generic) for HA type network solutions.

For the server you will need to look at cluster solutions.

As others have mentioned you can try bonding the interfaces on the
server to provide higher bandwidth but I believe you need to have a
switch that understands bonding as well.

When designing for redundancy and high availability start by identifying
the critical parts of your infrastructure and determine the type of
disasters you want to protect against as well as the likely hood of such
a disaster.  Many things while possible are unlikely or have little or
now impact.  Concentrate on those things that are likely to happen and
have major impact to your systems.

And remember that adding more hardware or making your network more
complex can sometimes increase the likely hood of having a failure cause
service interruptions.

Depending on the costs of taking an outage you may be better off having
a cold spare handy to replace the switch or device that fails.