LARTC - Dec 2000 - load balance/redundancy

I have looked through the archives, and I can''t find the answer(not
that
it isn''t there)

I have two connections to the net.  I want load balancing and redundancy.

cable      adsl
(24.141.)  (64.229)
|          |
|          |
|          |
|          |
------------
|  linux   |
|redhat 7.0|

I have no idea were to even start.  I would like equal access to both
connections.  Please help.  I would assume that alot of people do this
already, or want to.  A sample config would be very nice.


Thanks in advance for your help!

--Paul

On Sun, 10 Dec 2000, Paul wrote:
> I have looked through the archives, and I can''t find the
answer(not that
> it isn''t there)
> 
> I have two connections to the net.  I want load balancing and redundancy.
> 
> cable      adsl
> (24.141.)  (64.229)
> |          |
> |          |
> |          |
> |          |
> ------------
> |  linux   |
> |redhat 7.0|
> 
> I have no idea were to even start.  I would like equal access to both
> connections.  Please help.  I would assume that alot of people do this
> already, or want to.  A sample config would be very nice.
Hmm. Try reading the ip route 2 documentation on the ip command, and look
at the ''nexthop'' arguments to route add. This will set up
multipath routes
which do most of what you want.

There does seem to be something of a discrepancy between the ordinary
default route code and the multipath code in the handling of down
links that might bite you though when one of the links goes down...

Doei, Arthur. (Still figuring that last one out)

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Well, I think I have the redundancy down... In a kludgy way.  Right now I
have two default routes.

route add default gw 24.141.xxx.1
route add default gw 64.229.xxx.1

I can write a script that if one of connections goes down for x number of
pings, then remove the route.

Now for the load balancing.

Can you point me to documentation on iproute?  Other then the Advanced
Routing HOWTO, unless I am missing something in it.

Thanks again,
Paul

On Sun, 10 Dec 2000, Arthur van Leeuwen wrote:
> On Sun, 10 Dec 2000, Paul wrote:
> 
> > I have looked through the archives, and I can''t find the
answer(not that
> > it isn''t there)
> > 
> > I have two connections to the net.  I want load balancing and
redundancy.
> > 
> > cable      adsl
> > (24.141.)  (64.229)
> > |          |
> > |          |
> > |          |
> > |          |
> > ------------
> > |  linux   |
> > |redhat 7.0|
> > 
> > I have no idea were to even start.  I would like equal access to both
> > connections.  Please help.  I would assume that alot of people do this
> > already, or want to.  A sample config would be very nice.
> 
> Hmm. Try reading the ip route 2 documentation on the ip command, and look
> at the ''nexthop'' arguments to route add. This will set up
multipath routes
> which do most of what you want.
> 
> There does seem to be something of a discrepancy between the ordinary
> default route code and the multipath code in the handling of down
> links that might bite you though when one of the links goes down...
> 
> Doei, Arthur. (Still figuring that last one out)
> 
> 
-- 
HEY!  I''m a guy like me!

		--Homer

On Sun, 10 Dec 2000 hesselsp@ashaman.dhs.org wrote:
> Well, I think I have the redundancy down... In a kludgy way.  Right now I
> have two default routes.
> route add default gw 24.141.xxx.1
> route add default gw 64.229.xxx.1
Actually, if I understand the code correctly what you do here is in fact
create a multipath route. A cleaner way to do so is using ip route 2 as
follows:

ip route add default nexthop via 24.141.xxx.1 nexthop via 64.229.xxx.1

This explicitly states that you will be using a multipath route and balances
outgoing routes over the two interfaces.
> I can write a script that if one of connections goes down for x number of
> pings, then remove the route.
And here''s the crux. It would be nice for the kernel to use its idea of
gateway reachability (in the neighbor cache) to automatically ignore an
upstream hop in case it is dead. However, for multipath routes, no
death detection is done on the gateways in the different hops. This is
quite somewhat different from the single default route behaviour 
> Now for the load balancing.
The kernel balances outgoing routes over the upstream interfaces. Unless all
your packets go to the same address, that should balance your traffic
already. Other than that there''s some code based on EQL that will flush
the
route cache after every packet, thereby smashing the route-balancing down to
packet-level load balancing. I can''t seem to find the URL for that
code,
though, unfortunately.
> Can you point me to documentation on iproute?  Other then the Advanced
> Routing HOWTO, unless I am missing something in it.
http://snafu.freedom.org/linux2.2/iproute-notes.html#doc

There''s some more interesting stuff on
http://snafu.freedom.org/linux2.2/
as well. 

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Arthur,

Thanks for your help.  I tried what you suggested, here is the result:

# /sbin/ip route add default nexthop via 24.141.xxx.1 nexthop via
64.229.xxx.1
RTNETLINK answers: Network is unreachable

So instead, I tried:

# /sbin/ip route add default nexthop via 24.141.xxx.1 dev eth0 nexthop via
64.229.xxx.1 dev ppp0

# ip route ls
64.229.xxx.1 dev ppp0  proto kernel  scope link  src 64.229.xxx.12
24.141.xxx.0/22 dev eth0  proto kernel  scope link  src 24.141.xxx.89
127.0.0.0/8 dev lo  scope link
default
        nexthop via 24.141.xxx.1  dev eth0 weight 1
        nexthop via 64.229.xxx.1  dev ppp0 weight 1

and that seemed to work, so I tried:

# /sbin/ip route add default nexthop dev eth0 nexthop dev ppp0

# ip route ls
64.229.xxx.1 dev ppp0  proto kernel  scope link  src 64.229.xxx.12
24.141.xxx.0/22 dev eth0  proto kernel  scope link  src 24.141.xxx.89
127.0.0.0/8 dev lo  scope link
default
        nexthop dev eth0 weight 1
        nexthop dev ppp0 weight 1


Now this seems to work.  Is there anything wrong with this?  Can you take
a guess at why what you suggested didn''t work?  I think I tried all the
different permutations with addresses, I think the problem must come from
the pppoe interface.

Now all I have to do is kludge something together for my redundancy, set
up firewalling, and masqerading.

Thanks,
Paul


On Sun, 10 Dec 2000, Arthur van Leeuwen wrote:
> On Sun, 10 Dec 2000 hesselsp@ashaman.dhs.org wrote:
> 
> > Well, I think I have the redundancy down... In a kludgy way.  Right
now I
> > have two default routes.
> 
> > route add default gw 24.141.xxx.1
> > route add default gw 64.229.xxx.1
> 
> Actually, if I understand the code correctly what you do here is in fact
> create a multipath route. A cleaner way to do so is using ip route 2 as
> follows:
> 
> ip route add default nexthop via 24.141.xxx.1 nexthop via 64.229.xxx.1
> 
> This explicitly states that you will be using a multipath route and
balances
> outgoing routes over the two interfaces.
> 
> > I can write a script that if one of connections goes down for x number
of
> > pings, then remove the route.
> 
> And here''s the crux. It would be nice for the kernel to use its
idea of
> gateway reachability (in the neighbor cache) to automatically ignore an
> upstream hop in case it is dead. However, for multipath routes, no
> death detection is done on the gateways in the different hops. This is
> quite somewhat different from the single default route behaviour 
> 
> > Now for the load balancing.
> 
> The kernel balances outgoing routes over the upstream interfaces. Unless
all
> your packets go to the same address, that should balance your traffic
> already. Other than that there''s some code based on EQL that will
flush the
> route cache after every packet, thereby smashing the route-balancing down
to
> packet-level load balancing. I can''t seem to find the URL for that
code,
> though, unfortunately.
> 
> > Can you point me to documentation on iproute?  Other then the Advanced
> > Routing HOWTO, unless I am missing something in it.
> 
> http://snafu.freedom.org/linux2.2/iproute-notes.html#doc
> 
> There''s some more interesting stuff on
http://snafu.freedom.org/linux2.2/
> as well. 
> 
> Doei, Arthur.
> 
> 
-- 
HEY!  I''m a guy like me!

		--Homer

I take part of this back:

On Sun, 10 Dec 2000, Paul wrote:
> Arthur,
> 
> Thanks for your help.  I tried what you suggested, here is the result:
> 
> # /sbin/ip route add default nexthop via 24.141.xxx.1 nexthop via
> 64.229.xxx.1
> RTNETLINK answers: Network is unreachable
> 
> So instead, I tried:
> 
> # /sbin/ip route add default nexthop via 24.141.xxx.1 dev eth0 nexthop via
> 64.229.xxx.1 dev ppp0
> 
> # ip route ls
> 64.229.xxx.1 dev ppp0  proto kernel  scope link  src 64.229.xxx.12
> 24.141.xxx.0/22 dev eth0  proto kernel  scope link  src 24.141.xxx.89
> 127.0.0.0/8 dev lo  scope link
> default
>         nexthop via 24.141.xxx.1  dev eth0 weight 1
>         nexthop via 64.229.xxx.1  dev ppp0 weight 1
This DOES work.

> 
> and that seemed to work, so I tried:
> 
> # /sbin/ip route add default nexthop dev eth0 nexthop dev ppp0
> 
> # ip route ls
> 64.229.xxx.1 dev ppp0  proto kernel  scope link  src 64.229.xxx.12
> 24.141.xxx.0/22 dev eth0  proto kernel  scope link  src 24.141.xxx.89
> 127.0.0.0/8 dev lo  scope link
> default
>         nexthop dev eth0 weight 1
>         nexthop dev ppp0 weight 1
> 
This does WEIRD things.  I can traceroute to ftp.cdrom.com, but I can''t
ping it, nor can I ftp to it.

> 
> Now this seems to work.  Is there anything wrong with this?  Can you take
> a guess at why what you suggested didn''t work?  I think I tried
all the
> different permutations with addresses, I think the problem must come from
> the pppoe interface.
> 
> Now all I have to do is kludge something together for my redundancy, set
> up firewalling, and masqerading.
> 
> Thanks,
> Paul
> 
> 
> On Sun, 10 Dec 2000, Arthur van Leeuwen wrote:
> 
> > On Sun, 10 Dec 2000 hesselsp@ashaman.dhs.org wrote:
> > 
> > > Well, I think I have the redundancy down... In a kludgy way. 
Right now I
> > > have two default routes.
> > 
> > > route add default gw 24.141.xxx.1
> > > route add default gw 64.229.xxx.1
> > 
> > Actually, if I understand the code correctly what you do here is in
fact
> > create a multipath route. A cleaner way to do so is using ip route 2
as
> > follows:
> > 
> > ip route add default nexthop via 24.141.xxx.1 nexthop via 64.229.xxx.1
> > 
> > This explicitly states that you will be using a multipath route and
balances
> > outgoing routes over the two interfaces.
> > 
> > > I can write a script that if one of connections goes down for x
number of
> > > pings, then remove the route.
> > 
> > And here''s the crux. It would be nice for the kernel to use
its idea of
> > gateway reachability (in the neighbor cache) to automatically ignore
an
> > upstream hop in case it is dead. However, for multipath routes, no
> > death detection is done on the gateways in the different hops. This is
> > quite somewhat different from the single default route behaviour 
> > 
> > > Now for the load balancing.
> > 
> > The kernel balances outgoing routes over the upstream interfaces.
Unless all
> > your packets go to the same address, that should balance your traffic
> > already. Other than that there''s some code based on EQL that
will flush the
> > route cache after every packet, thereby smashing the route-balancing
down to
> > packet-level load balancing. I can''t seem to find the URL for
that code,
> > though, unfortunately.
> > 
> > > Can you point me to documentation on iproute?  Other then the
Advanced
> > > Routing HOWTO, unless I am missing something in it.
> > 
> > http://snafu.freedom.org/linux2.2/iproute-notes.html#doc
> > 
> > There''s some more interesting stuff on
http://snafu.freedom.org/linux2.2/
> > as well. 
> > 
> > Doei, Arthur.
> > 
> > 
> 
> 
-- 
HEY!  I''m a guy like me!

		--Homer

On Sun, 10 Dec 2000, Paul wrote:
> Arthur,
> 
> Thanks for your help.  I tried what you suggested, here is the result:
> 
> # /sbin/ip route add default nexthop via 24.141.xxx.1 nexthop via
> 64.229.xxx.1
> RTNETLINK answers: Network is unreachable
You forgot to setup host routes to 64.229.xxx.1 and 24.141.xxx.1 over the 
correct interfaces.
> So instead, I tried:
> 
> # /sbin/ip route add default nexthop via 24.141.xxx.1 dev eth0 nexthop via
> 64.229.xxx.1 dev ppp0
Well, that works as well. No host routes needed. :)
> # ip route ls
> 64.229.xxx.1 dev ppp0  proto kernel  scope link  src 64.229.xxx.12
> 24.141.xxx.0/22 dev eth0  proto kernel  scope link  src 24.141.xxx.89
> 127.0.0.0/8 dev lo  scope link
> default
>         nexthop dev eth0 weight 1
>         nexthop dev ppp0 weight 1
Ah, yes. You might get problems with the selection of the source address you
come from though. The situation that might occur is a route going over ppp0
using 24.141.xxx.89 as source address. That will probably *not* work.
> Now this seems to work.  Is there anything wrong with this?  Can you take
> a guess at why what you suggested didn''t work?  I think I tried
all the
> different permutations with addresses, I think the problem must come from
> the pppoe interface.
I don''t think the problem comes from the pppoe interface. I''ve
set up stuff
like this with ppp over pptp over ethernet links (*silly* ADSL setup), and
that works as well.
> Now all I have to do is kludge something together for my redundancy, set
> up firewalling, and masqerading.
Note: you may want to set up policy rules for dealing with packets that
already have a source or destination address. This will let servers running
on your system that bind to 0.0.0.0/0 answer to requests from both links
over the correct link. Browse through the mailinglist archives for some
hints on how to set that up.

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Be attentive : you don''t use 24.141.xxx.1 -> xxx should be a number
and the "24.141.xxx.1" should be the gateway you''re using
with the 24.141. ip adress (.videotron.ca?)
and the 64.226.xxx.1 should be the gw your 64.226. ip adress uses (pacbell.net ?
:)))) )

Best Regards
   Dan Caescu
> > 
> > # /sbin/ip route add default nexthop via 24.141.xxx.1 nexthop via
> > 64.229.xxx.1
> > RTNETLINK answers: Network is unreachable
> 
> You forgot to setup host routes to 64.229.xxx.1 and 24.141.xxx.1 over the 
> correct interfaces.
> 
> > So instead, I tried:
> > 
> > # /sbin/ip route add default nexthop via 24.141.xxx.1 dev eth0 nexthop
via
> > 64.229.xxx.1 dev ppp0
> 
> Well, that works as well. No host routes needed. :)
> 
> > # ip route ls
> > 64.229.xxx.1 dev ppp0  proto kernel  scope link  src 64.229.xxx.12
> > 24.141.xxx.0/22 dev eth0  proto kernel  scope link  src 24.141.xxx.89
> > 127.0.0.0/8 dev lo  scope link
> > default
> >         nexthop dev eth0 weight 1
> >         nexthop dev ppp0 weight 1
> 
> Ah, yes. You might get problems with the selection of the source address
you
> come from though. The situation that might occur is a route going over ppp0
> using 24.141.xxx.89 as source address. That will probably *not* work.
> 
> > Now this seems to work.  Is there anything wrong with this?  Can you
take
> > a guess at why what you suggested didn''t work?  I think I
tried all the
> > different permutations with addresses, I think the problem must come
from
> > the pppoe interface.
> 
> I don''t think the problem comes from the pppoe interface.
I''ve set up stuff
> like this with ppp over pptp over ethernet links (*silly* ADSL setup), and
> that works as well.
> 
> > Now all I have to do is kludge something together for my redundancy,
set
> > up firewalling, and masqerading.
> 
> Note: you may want to set up policy rules for dealing with packets that
> already have a source or destination address. This will let servers running
> on your system that bind to 0.0.0.0/0 answer to requests from both links
> over the correct link. Browse through the mailinglist archives for some
> hints on how to set that up.
> 
> Doei, Arthur.
> 
> -- 
>   /\    / |      arthurvl@sci.kun.nl      | Work like you don''t
need the money
>  /__\  /  | A friend is someone with whom | Love like you have never been
hurt
> /    \/__ | you can dare to be yourself   | Dance like there''s
nobody watching
> 
> 
> _______________________________________________
> LARTC mailing list / LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO:
http://ds9a.nl/2.4Routing/