Shorewall users - Nov 2005 - [Off-topic] Two provider-setup breaks long-running TCP-connections

Sorry for asking this, as I believe it to be a kernel-related rather
than Shorewall-related problem.  But some of you guys seem to have a
lot of experience with these kind of things.

I''m setting up a NAT''ing router with two ISP lines.

At first sight, everything works as expected, however when the local
machines try to keep a TCP-connection open for a long time, it
disconnects after a period.

Using the 2.6.12 kernel, the disconnection occurs after 10 to 20
minutes.  If I patch it using Julian Anastasov''s router patches
(www.ssi.bg/~ja/), it disconnects after 1 to 2 hours.

Now, two questions for the list:

1) Has anyone tried keeping tcp-connections open for a long time on a
two-provider setup?  Success or failure?
2) Does anyone know whether using Julian''s patches are a good idea?  I
can''t make heads or tails of his descriptions.



Rune
rune.kock@gmail.com


My providers entries are:

#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
LINE1   1        1       main      eth0      10.0.0.1     balance    eth2
LINE2   2        2       main      eth1      192.168.1.1 balance    eth2

I don''t use "track", as I don''t need to accept
incoming connections.


-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

On Monday 21 November 2005 11:31, Rune Kock wrote:
>>
> My providers entries are:
>
> #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
> LINE1   1        1       main      eth0      10.0.0.1     balance    eth2
> LINE2   2        2       main      eth1      192.168.1.1 balance    eth2
>
> I don''t use "track", as I don''t need to accept
incoming connections.
>
I suggest that you use ''track'', even though you don''t
need to accept incoming
connections. I suspect that what is happening is that during periods of 
inactivity, the route cache entry corresponding to the connection is expiring 
and when traffic resumes, you have only a 50-50 chance of the correct route 
being picked. With ''track'', the routing table corresponding to
the interface
that was used previously will be used again and the connection should stay 
alive.

Please let us know if this corrects your problem so that we can update the 
documentation.

Thanks,
-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

Tom Eastep wrote on 21/11/2005 17:48:31:
> On Monday 21 November 2005 11:31, Rune Kock wrote:
> 
> >>
> > My providers entries are:
> >
> > #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
> > LINE1   1        1       main      eth0      10.0.0.1     balance eth2
> > LINE2   2        2       main      eth1      192.168.1.1 balance eth2
> >
> > I don''t use "track", as I don''t need to
accept incoming connections.
> >
> 
> I suggest that you use ''track'', even though you
don''t need to accept
incoming 
> connections. I suspect that what is happening is that during periods of 
> inactivity, the route cache entry corresponding to the connection is 
expiring 
> and when traffic resumes, you have only a 50-50 chance of the correct 
route 
> being picked. With ''track'', the routing table
corresponding to the
interface 
> that was used previously will be used again and the connection should 
stay 
> alive.
> 
> Please let us know if this corrects your problem so that we can update 
the 
> documentation.
> 
> Thanks,
> -Tom
I was wondering if there wasn''t some kind of
''keepalive'' option in the
traffic you are running...

cheers,

-- 
Eduardo

Thanks for the "track" suggestion, Tom.  I''ll try that later
this week.

And to Eduardo -- the disconnects occur even though the connection is
relatively active; the periods of inactivity are less than 1 minute.


Rune

On 11/21/05, Eduardo Ferreira <duda@icatu.com.br>
wrote:
>
> Tom Eastep wrote on 21/11/2005 17:48:31:
>
>  > On Monday 21 November 2005 11:31, Rune Kock wrote:
>  >
>  > >>
>  > > My providers entries are:
>  > >
>  > > #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
>  > > LINE1   1        1       main      eth0      10.0.0.1    
balance
> eth2
>  > > LINE2   2        2       main      eth1      192.168.1.1 balance
> eth2
>  > >
>  > > I don''t use "track", as I don''t need
to accept incoming connections.
>  > >
>  >
>  > I suggest that you use ''track'', even though you
don''t need to accept
> incoming
>  > connections. I suspect that what is happening is that during periods
of
>  > inactivity, the route cache entry corresponding to the connection is
> expiring
>  > and when traffic resumes, you have only a 50-50 chance of the correct
> route
>  > being picked. With ''track'', the routing table
corresponding to the
> interface
>  > that was used previously will be used again and the connection should
> stay
>  > alive.
>  >
>  > Please let us know if this corrects your problem so that we can
update
> the
>  > documentation.
>  >
>  > Thanks,
>  > -Tom
>
> I was wondering if there wasn''t some kind of
''keepalive'' option in the
> traffic you are running...
>
> cheers,
>
> --
> Eduardo

-------------------------------------------------------
This SF.Net email is sponsored by the JBoss Inc.  Get Certified Today
Register for a JBoss Training Course.  Free Certification Exam
for All Training Attendees Through End of 2005. For more info visit:
http://ads.osdn.com/?ad_idv28&alloc_id845&op=click

I''ve tried testing the "track" suggestion.  However my setup
is
questionable in so many other ways that I''ve been unable to tell
whether it helps or not.  It seems more stable, but some disconnects
still occur.  I don''t know for certain whether the disconnects are
related to the multi-provider part.  I guess I will have to work on it
some more...


Rune

On 11/21/05, Tom Eastep <teastep@shorewall.net>
wrote:
> On Monday 21 November 2005 11:31, Rune Kock wrote:
>
> >>
> > My providers entries are:
> >
> > #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
> > LINE1   1        1       main      eth0      10.0.0.1     balance   
eth2
> > LINE2   2        2       main      eth1      192.168.1.1 balance   
eth2
> >
> > I don''t use "track", as I don''t need to
accept incoming connections.
> >
>
> I suggest that you use ''track'', even though you
don''t need to accept incoming
> connections. I suspect that what is happening is that during periods of
> inactivity, the route cache entry corresponding to the connection is
expiring
> and when traffic resumes, you have only a 50-50 chance of the correct route
> being picked. With ''track'', the routing table
corresponding to the interface
> that was used previously will be used again and the connection should stay
> alive.
>
> Please let us know if this corrects your problem so that we can update the
> documentation.
>
> Thanks,
> -Tom
> --
> Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
> Shoreline,     \ http://shorewall.net
> Washington USA  \ teastep@shorewall.net
> PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key
>
>
>

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click

Hey,

Rune Kock wrote:
> Thanks for the "track" suggestion, Tom.  I''ll try that
later this week.
> 
> And to Eduardo -- the disconnects occur even though the connection is
> relatively active; the periods of inactivity are less than 1 minute.
I seem to be having the same issue as Rune here... Lot''s of disconnects
and some other weird stuff(*) using dual providers. Until now, we have 
tried both with and without the ''track'' option without much
gain.
Strangely, even taking out the ''balance'' option
didn''t seem to help.

The thing is that i tried a similar setup from the LARTC how-to, without 
Shorewall a few years ago, using a 2.4 kernel (now 2.6) and some RedHat 
version (now Debian), with exactly the same result... I must be doing 
something terribly wrong here: I''d be glad to hear from other people
who
do not have these problems to confirm that i''m crazy. :)

Rune, care to compare our setups (off-list maybe), to try to figure this 
out?


(*) Unfortunately i cannot recall whether these things already happened 
with both links up, or during failover testing.
-- 
  - Pieter


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_id=7637&alloc_id=16865&op=click

Now I''ve checked things a bit more.  I believe that I have two
separate problems:

1) my two internet lines are unstable -- this has nothing to do with
my Linux router, but it means that I get quite confused during
testing.  So please bear with me for these unprecise posts...

2) the multi-supplier setup causes the Linux router to sometimes break
connections.

One of my users has done some thorough testing using an internet game
that uses both UDP and TCP at the same time.  Without "track", his
connection breaks at exactly 10-minute (sometimes 20-minute)
intervals.  With "track", his connection is significantly more stable.

My own tests using just TCP have not yielded any clear results.

So I may very well have been wrong from the start in calling this a
TCP-problem.  It may just as well be UDP that breaks, which somehow
seems more logical to me.

The bottom line is that I would recommend everyone using "track" in
their multi-supplier setups.  Thanks again for that suggestion, Tom --
and for the fantastic Shorewall as well.


Rune

On 11/24/05, Rune Kock <rune.kock@gmail.com>
wrote:
> I''ve tried testing the "track" suggestion.  However my
setup is
> questionable in so many other ways that I''ve been unable to tell
> whether it helps or not.  It seems more stable, but some disconnects
> still occur.  I don''t know for certain whether the disconnects are
> related to the multi-provider part.  I guess I will have to work on it
> some more...
>
>
> Rune
>
> On 11/21/05, Tom Eastep <teastep@shorewall.net> wrote:
> > On Monday 21 November 2005 11:31, Rune Kock wrote:
> >
> > >>
> > > My providers entries are:
> > >
> > > #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
> > > LINE1   1        1       main      eth0      10.0.0.1     balance
eth2
> > > LINE2   2        2       main      eth1      192.168.1.1 balance 
eth2
> > >
> > > I don''t use "track", as I don''t need
to accept incoming connections.
> > >
> >
> > I suggest that you use ''track'', even though you
don''t need to accept incoming
> > connections. I suspect that what is happening is that during periods
of
> > inactivity, the route cache entry corresponding to the connection is
expiring
> > and when traffic resumes, you have only a 50-50 chance of the correct
route
> > being picked. With ''track'', the routing table
corresponding to the interface
> > that was used previously will be used again and the connection should
stay
> > alive.
> >
> > Please let us know if this corrects your problem so that we can update
the
> > documentation.
> >
> > Thanks,
> > -Tom
> > --
> > Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
> > Shoreline,     \ http://shorewall.net
> > Washington USA  \ teastep@shorewall.net
> > PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key
> >
> >
> >
>

-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://ads.osdn.com/?ad_idv37&alloc_id865&op=click

On Monday 02 January 2006 14:57, Rune Kock wrote:
>
> The bottom line is that I would recommend everyone using "track"
in
> their multi-supplier setups.  Thanks again for that suggestion, Tom --
> and for the fantastic Shorewall as well.
>
Thanks for the update, Rune.

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key

On Monday 02 January 2006 15:14, Tom Eastep wrote:
> On Monday 02 January 2006 14:57, Rune Kock wrote:
> > The bottom line is that I would recommend everyone using
"track" in
> > their multi-supplier setups.  Thanks again for that suggestion, Tom --
> > and for the fantastic Shorewall as well.
>
> Thanks for the update, Rune.
To follow up, I''ve added a couple of recommendations to the multi-ISP
docs.
See:

	http://www1.shorewall.net/MultiISP.html

Thanks, again

-Tom
-- 
Tom Eastep    \ Nothing is foolproof to a sufficiently talented fool
Shoreline,     \ http://shorewall.net
Washington USA  \ teastep@shorewall.net
PGP Public Key   \ https://lists.shorewall.net/teastep.pgp.key