LARTC - Apr 2002 - Per-connection routing for multiple uplinks/providers ??

Hi - I''m a new subscriber to this list.

I have been digging through the Lartc documentation as well as Netfilter,
etc. and haven''t found much on per-connection routing for multiple
uplinks/providers.

What I would like to do is cleanly move packets out to the Internet over
two (maybe 3) separate interfaces, utilizing all of the bandwidth, and
avoiding snags.

I could use a round-robin scheduler, which would put consecutive packets on
different interfaces. I think this will run into problems when the reply
packets come back. Maybe not ??

I read through Arthur Leeuwen''s documentation
    (http://lartc.org/HOWTO//cvs/2.4routing/html/x247.html )
on a scheme for dividing the outgoing packets on a per-route basis. Packets
going to the same destination will go through the same interface. This gets
around the round-robin problem, but I think this is not ''fair''
in the sense
that one interface might accumulate more routes than the other, and there
does not seem to be a mechanism (other than periodically flushing the route
tables) for evening out the flows.  It is pretty simple though and I will
use this as a first chop solution.

Another approach to the problem would be to do a round-robin on a
per-connection basis. Each new connection would go out of the
''next''
interface.

I don''t know exactly how to do this though. Perhaps marking the
''NEW'' state
packet and routing on the mark (even marks go to the left interface, odd
marks to the right,... mod N for more than 2 interfaces).

Of course, it would be nice to allocate connections on an available
bandwidth basis.  Also do some QoS for ftp vs interactive (am looking at
the wondershaper..)

Also would be nice to periodically grab statistics so that I could
determine whether I need to get rid of an ISP (for non-competitive price/bw
stats). The stats could also be used to ''close the loop''
around the routing
to ensure that the best bandwidth is being achieved.

Also would be nice to energize a dial-up connection if the other 2 die for
some reason.

Does such a beast exist? Is it possible to build with current
ip/tc/netfilter technology?  I am running a near stock RH 7.2 at the
moment. Each ISP line is going through a separate (proprietary/black-box)
firewall/router and then into the RH7.2 box.

Thanks for your time.

BobG

On Mon, 15 Apr 2002, Bob Gustafson wrote:
> What I would like to do is cleanly move packets out to the Internet over
> two (maybe 3) separate interfaces, utilizing all of the bandwidth, and
> avoiding snags.
[snip, lots of nice interesting wishes]
> Does such a beast exist? Is it possible to build with current
> ip/tc/netfilter technology?  I am running a near stock RH 7.2 at the
> moment. Each ISP line is going through a separate (proprietary/black-box)
> firewall/router and then into the RH7.2 box.
QoS and routing are basically independent of each other. Orthogonal,
mathematicians would say. Furthermore, there *is* in fact a patch floating
around somewhere on the internet that flushes the route cache after every
packet... that might help. I''ve never used it, as I don''t
trust it to keep
TCP connections intact, but still, it might prove useful.

You''ll have to do some digging on your own though...

Doei, Arthur. (Oh, and it is Arthur van Leeuwen. The surname is
               ''van Leeuwen''. Kinda like in ''Ludwig
van Beethoven'')

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Hello All
Just to check the mail is working.
After reading the Advanced routing HOWTO 5 times I think I am getting a
basic understanding of shaping and policing. [see, got the terminology
right so far]
Can anybody point me in the direction of more docs.man HOWTO RFC don''t
care?
Fascinating stuff. My wife is looking at me though I am mad though. 

"You find that interesting!!"
regards 
Robert




-- 
Robert Slater
Synaptic Solutions
148 Mitchell Street
Northcote 3070
VIC
03 9486 5821
mobile 0438 133 664
robert@synapticserver.com
www.synapticserver.com

>On Mon, 15 Apr 2002, Bob Gustafson wrote:
>
>> What I would like to do is cleanly move packets out to the Internet
over
>> two (maybe 3) separate interfaces, utilizing all of the bandwidth, and
>> avoiding snags.
>
>[snip, lots of nice interesting wishes]
>
>> Does such a beast exist? Is it possible to build with current
>> ip/tc/netfilter technology?  I am running a near stock RH 7.2 at the
>> moment. Each ISP line is going through a separate
(proprietary/black-box)
>> firewall/router and then into the RH7.2 box.
>
>QoS and routing are basically independent of each other. Orthogonal,
>mathematicians would say. Furthermore, there *is* in fact a patch floating
>around somewhere on the internet that flushes the route cache after every
>packet... that might help. I''ve never used it, as I don''t
trust it to keep
>TCP connections intact, but still, it might prove useful.
>
>You''ll have to do some digging on your own though...
I was afraid of that. But this list was the first place to dig, yes?
>
>Doei, Arthur. (Oh, and it is Arthur van Leeuwen. The surname is
>               ''van Leeuwen''. Kinda like in
''Ludwig van Beethoven'')
Ahh, learn something every day. The van *isn''t* just like a middle name
then.
(I was pretty careful about the spelling of Leeuwen though :-)

Thanks much for the feedback

BobG

On Tue, 16 Apr 2002, Bob Gustafson wrote:
> >On Mon, 15 Apr 2002, Bob Gustafson wrote:
> >
> >> What I would like to do is cleanly move packets out to the
Internet over
> >> two (maybe 3) separate interfaces, utilizing all of the bandwidth,
and
> >> avoiding snags.
> >
> >[snip, lots of nice interesting wishes]
> >
> >> Does such a beast exist? Is it possible to build with current
> >> ip/tc/netfilter technology?  I am running a near stock RH 7.2 at
the
> >> moment. Each ISP line is going through a separate
(proprietary/black-box)
> >> firewall/router and then into the RH7.2 box.
> >
> >QoS and routing are basically independent of each other. Orthogonal,
> >mathematicians would say. Furthermore, there *is* in fact a patch
floating
> >around somewhere on the internet that flushes the route cache after
every
> >packet... that might help. I''ve never used it, as I
don''t trust it to keep
> >TCP connections intact, but still, it might prove useful.
> >
> >You''ll have to do some digging on your own though...
>
> I was afraid of that. But this list was the first place to dig, yes?
Yes, nowadays it is. :) At least you know there''s something out there.
Oh, as I recall the patch is by some weird guy in Utrecht, The Netherlands.
Can''t seem to find it using Google at the moment though.

The stuff on http://www-2.cs.cmu.edu/~jcl/programs/ipeql/ipeql.html
does seem like something you might want to look at (even though it is
*old*).
> >
> >Doei, Arthur. (Oh, and it is Arthur van Leeuwen. The surname is
> >               ''van Leeuwen''. Kinda like in
''Ludwig van Beethoven'')
>
> Ahh, learn something every day. The van *isn''t* just like a middle
> name then.
Nope. It would''ve been capitalized if it were. :)
> (I was pretty careful about the spelling of Leeuwen though :-)
Don''t computers make life *easy*? :P
> Thanks much for the feedback
My pleasure.

Doei, Arthur.

-- 
  /\    / |      arthurvl@sci.kun.nl      | Work like you don''t need
the money
 /__\  /  | A friend is someone with whom | Love like you have never been hurt
/    \/__ | you can dare to be yourself   | Dance like there''s nobody
watching

Bob Gustafson wrote:
>>On Mon, 15 Apr 2002, Bob Gustafson wrote:
>>
>>
>>>What I would like to do is cleanly move packets out to the Internet
over
>>>two (maybe 3) separate interfaces, utilizing all of the bandwidth,
and
>>>avoiding snags.
>>
>>[snip, lots of nice interesting wishes]
>>
>>
>>>Does such a beast exist? Is it possible to build with current
>>>ip/tc/netfilter technology?  I am running a near stock RH 7.2 at the
>>>moment. Each ISP line is going through a separate
(proprietary/black-box)
>>>firewall/router and then into the RH7.2 box.
>>
>>QoS and routing are basically independent of each other. Orthogonal,
>>mathematicians would say. Furthermore, there *is* in fact a patch
floating
>>around somewhere on the internet that flushes the route cache after
every
>>packet... that might help. I''ve never used it, as I
don''t trust it to keep
>>TCP connections intact, but still, it might prove useful.
The patch is called equalize, i''ve adapted it to 2.4.18 and fixed some 
locking issued some time ago, it''s available at 
ftp://sliepen.warande.net/pub/eql/ . I haven''t found the time to 
experience a lot with it, but i didn''t notice any broken tcp
connections.

Bye,
Patrick

On Mon, 15 Apr 2002, Bob Gustafson wrote:
> Hi - I''m a new subscriber to this list.
>
> I have been digging through the Lartc documentation as well as Netfilter,
> etc. and haven''t found much on per-connection routing for multiple
> uplinks/providers.
>
<snip>

I wont recommend the round robin packet load balancing as that would do
more problems with TCP connections, asymetric routing , RTT values.

Cisco recommends the per destination load balancing solution. I think if
you enable uqual cost multipath in kernel and add the proper routing
entries with ip route the kernel would do the rest (building the FIB
entries by a round robin per destination fashion).

If you have time it would be nice to give it a try and let us know how it
worked :)

----------------------------
Mihai RUSU

Disclaimer: Any views or opinions presented within this e-mail are solely
those of the author and do not necessarily represent those of any company,
unless otherwise specifically stated.

Mihai RUSU wrote:
> I wont recommend the round robin packet load balancing as that would do
> more problems with TCP connections, asymetric routing , RTT values.
I guess if you''d use multiple connections to the same ISP (and same 
access point) that would solve the problems of different RTT values.
Otherwise i can confirm there are problems, i had a lot more (needless) 
retransmits using a dsl + two isdn links which was probably due to very 
different RTT on both links.
> Cisco recommends the per destination load balancing solution. I think if
It''s not always practical. For example in my case one of the links is 
asymetric (256k upstresm/1mbit downstream), the other isn''t. So 
distributing traffic with a normal multipath route would result in some
connections (maybe downloads) beeing on the link with slow downstream.

Bye,
Patrick

On Tue, 16 Apr 2002, Patrick McHardy wrote:
> Mihai RUSU wrote:
>
> > I wont recommend the round robin packet load balancing as that would
do
> > more problems with TCP connections, asymetric routing , RTT values.
>
> I guess if you''d use multiple connections to the same ISP (and
same
> access point) that would solve the problems of different RTT values.
> Otherwise i can confirm there are problems, i had a lot more (needless)
> retransmits using a dsl + two isdn links which was probably due to very
> different RTT on both links.
>
Yep, also you mentioned about that patch, which would do if having the
same provider as yuo said. In theory the lower layer u do the load
balancing the more equal it is (think about per destination, per packet,
multilink ppp, ATM ...)
> > Cisco recommends the per destination load balancing solution. I think
if
>
> It''s not always practical. For example in my case one of the links
is
> asymetric (256k upstresm/1mbit downstream), the other isn''t. So
> distributing traffic with a normal multipath route would result in some
> connections (maybe downloads) beeing on the link with slow downstream.
>
I guess (as always) that there is a price to be paid. If this situations
with traffic being routed on the slow link happens too often some static
routes would help :)

----------------------------
Mihai RUSU

Disclaimer: Any views or opinions presented within this e-mail are solely
those of the author and do not necessarily represent those of any company,
unless otherwise specifically stated.

my cousin has 3 ISP connections (DSL, ISDN, HIS)
and he made "load balancing" using iptables CONNMARK.
this patch allows marking all packets belonging to connection
with one mark, anyway this patch is rather outtdated but
my cousin was able to make it working with 2.4.18 kernel
(there''s problem with connmark''s mark - it doesn''t
work
with mark-based routing, but MARK module works so
packets can be marked with CONNMARK and then with MARK
so thar routing works)

mikee
__________________________
ignore ads below this line



-------------------------------------------------
zrób sympatii niespodziankę i wyślij logo AA.101431   
SMS na numer 7116  koszt 1 zl + Vat :: Idea, Era i Plus Gsm  www.toplogo.pl

From: "andrew@foxtrot.kiev.ua"
<andrew@foxtrot.kiev.ua>
>   Why don''t he(she) use nth patch from the pom for load
ballancing?
>   It works fine for me.
>   
>   Regards,
>   
>   Andrew Kozachenko
i haven''t said it clearly but he has links to 3 DIFFERENT ISP (3
different IP),
so nth packet marking isn''t enough - all outgoing packets belonging to
connection
must be sent through same link to have uniform "external" ip
regards,

mikee
__________________________
ignore ads below this line


-------------------------------------------------
zrób sympatii niespodziankę i wyślij logo AA.101431   
SMS na numer 7116  koszt 1 zl + Vat :: Idea, Era i Plus Gsm  www.toplogo.pl