Currently I have a setup that involves connections from several different
providers ranging from 6 mbit DSL''s to 10 mbit fractional DS3
connections tied together to provide multipath high availability internet in our
area for schools and businesses. We recently extended our coverage to another
area which has several other ISP''s. The box that is doing multiwan
right now is the only gateway for the entire network and for some sites it is
several hops back to that gateway. My goal is for us to purchase bandwidth in
other locations, install a linux box and have multiple gateways available, but I
think this is easier said than done. I will diagram this as best I can for you
in current and future setup modes.
Currently:
ISP 1 ISP 2 ISP 3 etc...
| | | |
Multiwan box
|
Area 1
/ | \
Area 2 Area 3 Area 4
Goal:
Currently:
ISP 1 ISP 2 ISP 3 etc...
| | | |
Multiwan box
|
Area 1
/ | \
Area 2 Area 3 Area 4
| |
Linux Box Linux Box
| |
ISP x ISP y
My goal is for people in area 1-4 to all be able to use any of the connections
to ISP 1-z and if one fails it passes the traffic through the network to an
available route. Area 1-4 don''t current have any linux hardware but it
has been a long standing plan to put some in place in those areas as well so if
that is required it could be done. These areas are all serviced by different
small ISP''s and would make acceptable alternate routes say if a
backhual failed for some reason so that is the goal.
Thanks for any input.
sound like you need a routing protocol, if you have access to manage all the routers On Wed, Oct 10, 2007 at 09:44:09PM -0600, Tony Hagans wrote:> Currently I have a setup that involves connections from several different providers ranging from 6 mbit DSL''s to 10 mbit fractional DS3 connections tied together to provide multipath high availability internet in our area for schools and businesses. We recently extended our coverage to another area which has several other ISP''s. The box that is doing multiwan right now is the only gateway for the entire network and for some sites it is several hops back to that gateway. My goal is for us to purchase bandwidth in other locations, install a linux box and have multiple gateways available, but I think this is easier said than done. I will diagram this as best I can for you in current and future setup modes. > > Currently: > > ISP 1 ISP 2 ISP 3 etc... > | | | | > > Multiwan box > > | > > Area 1 > > / | \ > > Area 2 Area 3 Area 4 > > Goal: > > Currently: > > ISP 1 ISP 2 ISP 3 etc... > | | | | > > Multiwan box > > | > > Area 1 > > / | \ > > Area 2 Area 3 Area 4 > > | | > Linux Box Linux Box > | | > ISP x ISP y > > My goal is for people in area 1-4 to all be able to use any of the connections to ISP 1-z and if one fails it passes the traffic through the network to an available route. Area 1-4 don''t current have any linux hardware but it has been a long standing plan to put some in place in those areas as well so if that is required it could be done. These areas are all serviced by different small ISP''s and would make acceptable alternate routes say if a backhual failed for some reason so that is the goal. > > Thanks for any input. > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >_______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
On 10/10/07 22:44, Tony Hagans wrote:> Currently I have a setup that involves connections from several > different providers ranging from 6 mbit DSL''s to 10 mbit fractional > DS3 connections tied together to provide multipath high availability > internet in our area for schools and businesses. We recently > extended our coverage to another area which has several other ISP''s. > The box that is doing multiwan right now is the only gateway for the > entire network and for some sites it is several hops back to that > gateway. My goal is for us to purchase bandwidth in other locations, > install a linux box and have multiple gateways available, but I think > this is easier said than done. I will diagram this as best I can for > you in current and future setup modes....> My goal is for people in area 1-4 to all be able to use any of the > connections to ISP 1-z and if one fails it passes the traffic through > the network to an available route. Area 1-4 don''t current have any > linux hardware but it has been a long standing plan to put some in > place in those areas as well so if that is required it could be done. > These areas are all serviced by different small ISP''s and would make > acceptable alternate routes say if a backhual failed for some reason > so that is the goal.Like Alex said, I think you need at least an interior routing protocol. I''d also recommend that you look in to getting your own address space from your regional provider. Ideally you would get multiple if not all of the ISPs to BGP advertise your address space out to the world. In addition to the world knowing the better / best routes in to you depending on where the traffic is originating from, your network would know the better / best routes to take out to destinations with relation to your ingress / egress points. Additionally as long as you are doing NATing much closer to the customer, or the customer is doing it them selves, you could have multiple routes to / from the customer with out worrying about changing NAT IPs.> Thanks for any input.You are welcome. Grant. . . .
Thanks for the input Grant, we have kicked around the idea of BGP or a similar method for the routing but the issue is that NAT takes place at the box doing the multiwan. We only have a few external IP addresses and all of our connections are from different vendors so they are spread across several different IP ranges. My first though was to create a tunnels between the machines at each location and put the tunnel adapters in a routing table with a prio > the local routing table. This would allow data to: client --> main gateway --> routing table mpath to any gateway on the network --> internet as available I have a feeling this would cause some very very bad problems with return paths and I don''t know what would happen if a tunnel were to fail or something of that nature. The system is basically many many wireless repeaters, access points, and clients spread across about 60 square miles. It crosses 3 LATAs and is within range of at least 10 different ISP''s who all use different providers. The idea was if we could instead of bringing the traffic all back to where it goes now be able to go wherever was closest/least busy/etc and hop on a DSL/Cable/Wireless/whatever we can pick up for cheap bandwidth it would take some of the traffic from individuals doing whatever they do off the DS3 durring the day for businesses and schools to make use of. Tony
On 10/11/07 14:07, Tony Hagans wrote:> Thanks for the input Grant, we have kicked around the idea of BGP or > a similar method for the routing but the issue is that NAT takes > place at the box doing the multiwan. We only have a few external IP > addresses and all of our connections are from different vendors so > they are spread across several different IP ranges. My first though > was to create a tunnels between the machines at each location and put > the tunnel adapters in a routing table with a prio > the local > routing table. This would allow data to:You are welcome. Just because you are doing NATing at the core egress points instead of the edge ingress points does not mean that you can not still use some sort of routing protocol. Do NATing where you are but use some sort of routing protocol that will keep all your routers in sync with each other. If you use some sort of routing protocol that includes connection state and possibly load, you could easily shift where traffic is routed out of your network based on load and / or availability. Your customers would talk to the upstream router that you direct them to use, which will then send the traffic out the appropriate way.> client --> main gateway --> routing table mpath to any gateway on the > network --> internet as availableUsing some sort of routing protocol that includes load / cost of a particular connection, you could easily just let routers pick the best route to go out based on the routing protocol. Your routers that have a single connection would just do standard NATing. Your routers that have multiple connections would use the basic multipath routing described in the LARTC How-To. Use your routing protocols to pick which rout to go out. I would also make sure to educate each router be aware of the subnets that the provider has. There is no reason to ever go out provider B to get to a server on provider A''s network unless the link with provider A is down. It sounds like you don''t have a bunch of IPs, say one for each client, but could you get extra IPs from each provider, say one ip for each router in your network from each provider. So if you had 5 providers, get 5 IPs from each, for a total of 25 IPs. If you could do this, you could have each router be able to connect directly to the links from each provider and load balance out with multipath routing if you so chose. Granted this would be dependent on your network structure and its capabilities. One advantage of this is that you would have more IPs to hid services behind. If one IP became black listed for some reason, you could shift traffic off of it and use another one with out much hassle at all.> I have a feeling this would cause some very very bad problems with > return paths and I don''t know what would happen if a tunnel were to > fail or something of that nature. The system is basically many many > wireless repeaters, access points, and clients spread across about 60 > square miles. It crosses 3 LATAs and is within range of at least 10 > different ISP''s who all use different providers. The idea was if we > could instead of bringing the traffic all back to where it goes now > be able to go wherever was closest/least busy/etc and hop on a > DSL/Cable/Wireless/whatever we can pick up for cheap bandwidth it > would take some of the traffic from individuals doing whatever they > do off the DS3 durring the day for businesses and schools to make use > of.I think this is possible to do. I think you will have better luck if you have traffic predominantly use the closest router and not try to load balance a given client across multiple providers but rather load balance by picking which provider a given client uses. You could even use something as devious as VRRP and virtual routers to on the fly change which router was a given IP with out requiring clients to re-configure any thing. Granted you run a chance of in progress sessions being messed up, but that is a problem you will have with just about any multipath setup where you are NATing at the core. If you will provide more information on what your actual network topology is including if things are a layer 2 or layer 3 link and what subnets are where, I''ll be glad to help. Grant. . . .