LARTC - May 2004 - Re: Load Balancing 4 cable modems, followed nano.txt

Hello,

On Mon, 17 May 2004, Charles-Etienne.Dube wrote:
> I did some tests with 2 cable modems, but now it is installed in a
production
> environment with 4 cable modems.  At first, everything seemed to work 
fine..
> But now I had a couple of users tell me that some web pages were
> not available while others were, and it semms to be a masquerading problem
> since when I downgraded the system to one cable modem and normal routing
> table with only one default gateway everything started working fine again
(like
> before). I am sure that the non-working URLs were valid sites that were
available
> at the time (ex: www.yahoo.com; www.google.com; www.hotmail.com).
>
>
> Here are my questions :
>
>
> 1 - Did I use to right patch ? I used "routes-2.4.20-9.diff"
	It is the right one
> I tried to apply "05_nf_reroute-2.4.20-9.diff" Even if
I''m used to
> patching kernels, I don''t understand that much about this process,
and
> the patch programm was giving error messages, while the one I use was
applying
> without any error messages.
	Stick with "routes-*.diff" because it is not enough to apply
05_nf* patch, there are two diffs to apply before it.
> 2- Should I  use :
>
> iptables -t nat -A POSTROUTING -o IFE1 -s NWI/NMI -j SNAT --to IPE1
> iptables -t nat -A POSTROUTING -o IFE2 -s NWI/NMI -j SNAT --to IPE2
>
> instead of :
>
> iptables --table nat --append POSTROUTING --out-interface eth1 -j
MASQUERADE
	-j MASQUERADE works for dynamic IPs and is recommended
for complex routing situations when using the "routes-*" patches
> Can anybody elaborate on the difference between these 2 commands.
>
> 3 -  The 4 cable modems have a different IP, but two of them obtained an
> ip in the same address class so they both have the same gateway. In my
> script I treated them as if they were each on different gateways and I see
> traffic going on the 4 modems so I don''t think this is an issue
but I''d
> like to have some advice about this.  What happens if all the IPs
> are renewed at the same time and all 4 modems obtain an IP in the same
> class with same gateway... will it still work ?
	If the gateways are same I assume they are on different
interfaces.
> 4 - do you think 4 cable modems is too much, has it been tested ?
	I do not remember how many nexthops allows the ip utility for
multipath routes but 16 should be possible.
> Help with this case would really be appreaciated,  I am ready to give
> any details that would be necessary for you to guide me further..
ip addr
ip rule
ip route list table all
> Thank You
>
> Charles-Etienne Dube
Regards

--
Julian Anastasov <ja@ssi.bg>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Hello,

On Wed, 19 May 2004, Charles-Etienne.Dube wrote:
> Hi, thank you for the fast answer
>
>        Monday I configured the system again with multiple cable modems,
> but only 2 that time. Until now, one modem transfered 2 gig and the
> second one 1 gig.  No complaints that time, so i''ll leave it that
way
> till the end of the week to see if that setup is stable, but  i still
> need to add at least a third modem to obtain more troughput for peak hours.
	More internal users -> better sharing according to the weights :)
>        About the masquerade thing you said :
>
> -j MASQUERADE works for dynamic IPs and is recommended
> for complex routing situations when using the "routes-*" patches
> -------
>
> I''ll stick with that,  but a question about dynamic ip,  the
> documentation in nano.txt seems to be a setup for static ip. The cable
> modem modem I use will change ip about 2-3 times a year so no big deal
> for this.  Right now I hardcoded their ip in my script,  but  please
> confirm this :
>
>     If I want everything to work with no maintenance (almost)  If those
> cable modems change ip, i''ll need a script that detects it and
then
> dynamically change the routing table, because I can''t do that kind
of
> routing with only interface names.
	I created a script that can manage addresses, ip rules,
routing tables and next hops. It reads arrays with data from
config file. It appears to work in initial tests for static
addresses and routes but I understand that it would be good to
modify the next hop definitions, eg. when ppp device names change
or the IP addresses are dynamic. So far, I have written a little
bit of the documentation but now I''ll upload the current stuff here:

http://www.ssi.bg/~ja/#routes

If you are looking for something new for testing you can look for mpath/
and mpath-X.Y.tar.gz. It now contains settings for my tests. We should
find a way to propagate next hops changes (and may be changes for ip
rules). For now, it can be done by regenerating the config based on
ip-up/ip-down scripts and notifying mpath.sh about this next hop event.
I''m publishing it now just to see more ideas how the users actually
prefer to manage their setup.
> ---------
>
> Each modems are connected to their own network card on the server, so
> yes they each have different interface.  Is it possible to have it
> working with for example, GWE1 & GWE2 be the same ip adresse ?
	Yes, the kernel normally lookups by output device but
the patched kernel will match next hops by gateway too. In fact,
MASQUERADE wins for setups that have two nexthops with same output
device but different gateway IP because IIRC -j SNAT can not
match gateway IP.
> ip rule :
>
> 0:      from all lookup local
> 50:     from all lookup main
> 201:    from 24.201.150.0/24 lookup 201
> 202:    from 24.200.191.0/24 lookup 202
> 222:    from all lookup 222
> 32766:  from all lookup main
> 32767:  from all lookup 253
	May be your ip rules need more work. I want to see
what IP addresses are allowed for each uplink. You can follow
such rules:

- specify correct "from" and "to" in all ip rules and use
only
default routes in all tables when the routes are via gateway.

- put in same routing table all next hops that can serve same
source addresses, for example, can you put nexthops for eth1 and
eth4 in a multipath route in same table and for which addresses?

- provide different table for each uplink which is distinct
from all others and provides internet for different subnet

	So, I''m not sure whether you need 3 or 4 tables
for public IPs/subnets. The default table remains, of course.
> So there if you check eth1 and eth4 are on the same network and share
> the same gateway, could this be a problem ?
	Sort of. You must know what traffic you can send via
each uplink and to explain it with ip rules and tables. The
patched NAT strictly follows these rules.

	So, it is more correct when you use NAT to strictly
define correct rules. For example:

- know which IP or subnet can use particular uplink. I do not see
specific ''from'' ip rules for sources such as 24.201.150.222
and
24.201.150.250. May be each of them can use only its modem?
You must know which table will be used for each source address
and to tell it to the routing.
> Thank You
>
> Charles-Etienne Dube
Regards

--
Julian Anastasov <ja@ssi.bg>
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/