im in the process of configurating our network to have two upstream providers,
it will be loadbalanced
under normal operation and a complete failover if one of the lines would fail.
internet internet
| |
border border
| |
|- core router - |
|
lan
the "problem" im having is that i will not do nat on the core router,
but on the border routers.
the multipath default route is on the core router. from what i understand, could
be totally wrong,
you have to have nat, at least connection tracking on the core to make the
multipath route per
flow and not per packet.
any insight of this?
-tomas bonnedahl
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Wed, 2003-06-25 at 04:35, Tomas Bonnedahl wrote:> the "problem" im having is that i will not do nat on the core router, but on the border routers.I was faced with the same problem and ended up doing two rounds of NAT/PAT. The next step to that is to stop doing any NAT on the routers and let the core router deal with all that. From my experience a properly designed and dialed in Linux router can perform better than most other name brand dedicated routers. Now I am not saying it will be out a $100,000 Cisco router. The performance should easily be equal to or greater than your existing routers. For example when I had my setup in CA my Linux router through put latency was about half that of my Cisco 827 ADSL router, or either of my Netopia SDSL routers.> the multipath default route is on the core router.Linux router, correct.> from what i understand, could be totally wrong, > you have to have nat, at least connection tracking on the core to make the multipath route per > flow and not per packet.Correct, sort of. NAT will keep the path in cache, which will allow packets to keep traveling the same router. The word flow is much better than connection. You will not get per connection load balancing. Either way using multipath it will be per packet load balancing. However with NAT and Julian''s patches the NAT routes are cached which will allow further packets to flow or traverse the same path. I have seen others, I think even Julian, said that it is possible to accomplish without NAT. That has not been my experience. Based on my experience I would say that NAT is a must. -- Sincerely, William L. Thomson Jr. Support Group Obsidian-Studios, Inc. 3548 Jamestown Ln. Jacksonville, FL 32223 Phone/Fax 904.260.2445 http://www.obsidian-studios.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Perhaps I missed the original point of the first message, but why exactly don''t you just use BGP, as it was basically designed for this purpose? There are at least two good implementations of BGP for Linux, one of which is easy to use, the other obfuscated. (Zebra and GateD) Of course, that requires having globally routable address space in the first place, but I assume that you do. Is there a reason not to use BGP in this case? Aaron On Wed, 25 Jun 2003, William L. Thomson Jr. wrote:> On Wed, 2003-06-25 at 04:35, Tomas Bonnedahl wrote: > > > the "problem" im having is that i will not do nat on the core router, but on the border routers. > > I was faced with the same problem and ended up doing two rounds of > NAT/PAT. The next step to that is to stop doing any NAT on the routers > and let the core router deal with all that. From my experience a > properly designed and dialed in Linux router can perform better than > most other name brand dedicated routers. > > Now I am not saying it will be out a $100,000 Cisco router. The > performance should easily be equal to or greater than your existing > routers. > > For example when I had my setup in CA my Linux router through put > latency was about half that of my Cisco 827 ADSL router, or either of my > Netopia SDSL routers. > > > the multipath default route is on the core router. > > Linux router, correct. > > > from what i understand, could be totally wrong, > > you have to have nat, at least connection tracking on the core to make the multipath route per > > flow and not per packet. > > Correct, sort of. NAT will keep the path in cache, which will allow > packets to keep traveling the same router. > > The word flow is much better than connection. You will not get per > connection load balancing. Either way using multipath it will be per > packet load balancing. However with NAT and Julian''s patches the NAT > routes are cached which will allow further packets to flow or traverse > the same path. > > I have seen others, I think even Julian, said that it is possible to > accomplish without NAT. That has not been my experience. Based on my > experience I would say that NAT is a must. > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Wed, 2003-06-25 at 13:44, Aaron Dewell wrote:> Of course, that > requires having globally routable address space in the first place, but I > assume that you do.Most of the time this is the exact reason why BGP is not used. I have considered purchasing a block. However I doubt many ISP''s would want to route those IP''s to DSL lines or equivalent. Maybe a T-1. So if you have DSL lines or equivalent and a small block of static or dynamic IP addresses then BGP is not an option. -- Sincerely, William L. Thomson Jr. Support Group Obsidian-Studios, Inc. 3548 Jamestown Ln. Jacksonville, FL 32223 Phone/Fax 904.260.2445 http://www.obsidian-studios.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Wed, 2003-06-25 at 19:02, Aaron Dewell wrote:> DSL can be an option, if you have at least a /24, which if you have 3 routers, > you probably do. Some ISPs will think about it over DSL (business class).Covad no BellSouth no PacBell no assume mom and pops Sonic no ATG no So for most no. Not at least for DSL> If you are using residential class (1 IP, static or not), then sure, you have > to do something really, really obscure to multi-home, but then you are kinda > getting what you deserve then, IMHO.True, but those still may have the need or want.> The proper solution is to get more IP addresses from your provider, then > announce those via BGP. That''s usually not a big problem unless you have a > perceived lack of BGP clue, or a crappy provider.Yes but part of the idea is multiple providers. Next step for me will be to load balance a SDSL and CDMA wireless connection. At my previous location in CA I had two separate SDSL providers and a separate ADSL provider. All had different networks, but would still travel the same few miles to the CO. The wireless way will provide a further level of redundancy in case a physical line is broken between the CO and me.> You don''t need to purchase the block from ARIN, just get some from your > provider.I do and they allocate them to you. However most providers have down time for servicing and etc. Especially on DSL lines business class or not. So multiple providers is an added benefit since both usually are not down at the same time. Except for the scenario above thus wireless. One provider will not route for the other provider. So getting a block from one and having another route them for you as well is not an option. If you know of any ISPs that differ please let me know. -- Sincerely, William L. Thomson Jr. Support Group Obsidian-Studios, Inc. 3548 Jamestown Ln. Jacksonville, FL 32223 Phone/Fax 904.260.2445 http://www.obsidian-studios.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hello, On Wed, 25 Jun 2003, Tomas Bonnedahl wrote:> im in the process of configurating our network to have two upstream > providers, it will be loadbalanced > under normal operation and a complete failover if one of the lines would > fail. > > internet internet > | | > border border > | | > |- core router - | > | > lanThe problem is that the multipath route assumes one packet can go through any of the defined nexthops. This is valid for setups with many ISPs allowing spoofing, you can safely send packet with saddr=IP1 via ISP2 and this works as long as ISP1 is UP to receive the replies from remote hosts. But for your setup there is one requirement: once one connection is routed via Border X all its packets should go via the same border router for NAT purposes. Even the related ICMP traffic. And this works ... for some time until the routing cache is flushed and different Border router is selected from the multipath route. The connection usually dies.> the "problem" im having is that i will not do nat on the core router, but on the border routers. > the multipath default route is on the core router. from what i understand, could be totally wrong, > you have to have nat, at least connection tracking on the core to make the multipath route per > flow and not per packet.I''m not sure the Linux conntracking has such feature. IIRC, the netfilter follows the routing decision, i.e. nobody guarantees that packets from one connection will continue to use same nexthop while it is alive. Considering the fact that all cache entries expire (on user request or on timeout) the current implementation can not guarantee such persistence for the forwarded connections that have multiple possible paths. This works only if the core router performs NAT and when the NAT implementation does not blindly follow the multipath route decision. The problem can be solved in this way: the NAT connection requests route with saddr=the external IP address attached for this connection. As result, the multipath route is avoided and the traffic for one connection is bound to one ISP (usually): http://www.ssi.bg/~ja/#routes> any insight of this? > > > -tomas bonnedahlRegards -- Julian Anastasov <ja@ssi.bg> _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Thu, Jun 26, 2003 at 09:50:45AM -0600, Aaron Dewell wrote:> On Thu, 26 Jun 2003, Tomas Bonnedahl wrote: > > i dont have any addresses nor do i own an AS, i know there are private ASNs to > > use but this seems like a more complicated solution than a mere multipath default > > route to the two upstream providers.> An ASN can be gotten from ARIN with the justification "I''m multihomed to ASN #X > and #Y" and $500. Or you can use a private AS and have your upstreams filter > it out, also reasonably common.i didnt know it was that easy really, this might be an option.> BGP is not complicated at all to use, that''s a myth. It''s a fairly simple > protocol, and even easier to set up. Define one external peer per router, one > internal peer (each other), this is all done by AS. Set up the routes you want > advertised. In this case, you want everything, so no inbound filtering. Done. > 3 configuration options in Zebra''s bgpd. Less complicated than setting up NAT.i assume i will only advertise the core (some /28) since the lan is still a private network. i probably wont be able to get a whole /24 from my upstream.> Think about it - if you have two IP addresses total, one assigned by each > upstream, and using two default routes, anything connection-oriented is > broken immediately (TCP comes to mind). Anything connectionless (i.e. UDP) > will likely work fine. Web, ssh, IMAP, POP3, SMTP are all TCP. Those not > working make it basically useless.why wont it work? from what i understand, you could get a "per flow" with julians patches so the core-router doesnt varies on a per packet basis and thus make established connections to fail.> Otherwise, you have to have selective routes. Route this block of the internet > through provider X, that block through provider Y. No failover, no redundancy, > no point. Or, you could point default and provider X and a lower priority to > provider Y, but then you have to learn by IGP at your core when provider X dies. > That means advertising default from the borders with your IGP, which is a > workable solution, but could get messy if you''re not pretty good at whatever > IGP you are using, making the assumption that your IGP will do it. However, > two problems: 1. Your second connection is idle until the primary fails, thus > wasting money. 2. All TCP connections reset when you fail over to the backup, > and reset again when you resume to the primary.i thought the multihop path was designed to solve this issue with redundancy and failover? my very first thought in this was to use ospf as IGP but i couldnt come up with something to use upstream to see if the providers still were under normal operation. just to sum it up: use something like ospf as IGP and use BGP upstream. were you assuming that i would get a /24 from my isp and use for lan or should i do nat on the core router from the lan? thanks, tomas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/