Fedora directory users - Aug 2007 - directory server setting fail to terminate idle connections

I ran into issues hitting the max filedescriptors setting and found that
it was because the server never terminates idle connections.  I have an
idle timeout setting of 1200 seconds (20min).  If I make an LDAP request
from a client to the directory server, the tcp connection stays in
ESTABLISHED state on the server side forever.  I ran tcpdump on the
client side and not a single packet of traffic was sent to the server
during for hours.  

 

Any idea why this connection would not be terminated after 1200 sec?

Idle timeout enforcement is passive, it occurs when we''ve polled some 
activity and we''re walking over the connection table.  If you had made 
an additional connection or sent an operation on another established 
connection the idle one should have been disconnected.

Brian Fender wrote:
>
> I ran into issues hitting the max filedescriptors setting and found 
> that it was because the server never terminates idle connections.  I 
> have an idle timeout setting of 1200 seconds (20min).  If I make an 
> LDAP request from a client to the directory server, the tcp connection 
> stays in ESTABLISHED state on the server side forever.  I ran tcpdump 
> on the client side and not a single packet of traffic was sent to the 
> server during for hours. 
>
>  
>
> Any idea why this connection would not be terminated after 1200 sec?
>
> ------------------------------------------------------------------------
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>

This is actually an apache webserver making the connections to directory
server.  What I see through tcpdump and netstat is that apache creates
an LDAP connection to perform a search, and the connection sits idle for
hours in established state.  The webserver eventually re-uses the random
port it made the initial request on to talk to a client, so the LDAP
connection no longer shoes up as established on the client side.  On the
server side, however, it still shows the connection as established
forever.  There are many other apache children talking to the same LDAP
server in parallel, and the number of open filehandles constantly
increases.  

 

I realize that it is possible that the webserver is not properly tearing
the connection down or a firewall may be blocking it, but shouldn''t the
server application notice that that connection was idle for more than
20min and time it out anyway?

 

________________________________

From: fedora-directory-users-bounces@redhat.com
[mailto:fedora-directory-users-bounces@redhat.com] On Behalf Of Ulf
Weltman
Sent: Friday, August 10, 2007 12:49 PM
To: General discussion list for the Fedora Directory server project.
Subject: Re: [Fedora-directory-users] directory server setting fail
toterminate idle connections

 

Idle timeout enforcement is passive, it occurs when we''ve polled some
activity and we''re walking over the connection table.  If you had made
an additional connection or sent an operation on another established
connection the idle one should have been disconnected.

Brian Fender wrote: 

I ran into issues hitting the max filedescriptors setting and found that
it was because the server never terminates idle connections.  I have an
idle timeout setting of 1200 seconds (20min).  If I make an LDAP request
from a client to the directory server, the tcp connection stays in
ESTABLISHED state on the server side forever.  I ran tcpdump on the
client side and not a single packet of traffic was sent to the server
during for hours.  

 

Any idea why this connection would not be terminated after 1200 sec?

 



________________________________



 
--
Fedora-directory-users mailing list
Fedora-directory-users@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-directory-users

Brian Fender wrote:
> This is actually an apache webserver making the connections to directory 
> server.  What I see through tcpdump and netstat is that apache creates 
> an LDAP connection to perform a search, and the connection sits idle for 
> hours in established state.  The webserver eventually re-uses the random 
> port it made the initial request on to talk to a client, so the LDAP 
> connection no longer shoes up as established on the client side.  On the 
> server side, however, it still shows the connection as established 
> forever.  There are many other apache children talking to the same LDAP 
> server in parallel, and the number of open filehandles constantly 
> increases. 
> 
>  
> 
> I realize that it is possible that the webserver is not properly tearing 
> the connection down or a firewall may be blocking it, but shouldn’t the 
> server application notice that that connection was idle for more than 
> 20min and time it out anyway?
>
You want Apache to keep the connections open. It creates a pool of LDAP 
connections to use for all authentication. TCP/IP connections are 
expensive so it keeps them open to issue search and bind requests when 
doing authentication.

There may be a bug in the pooling code but how many connections are we 
talking about?

rob

With a lightly used, it takes about two weeks to hit 8000 connections.
The number of connections does constantly drop, however not nearly as
fast as it increases.  I would expect it to level off at some point, but
this never happens.  

I believe that the reason it is creating new connections is because the
apache processes are hitting max-requests-per-child and respawning
before they ever have a need to re-use their LDAP connection.  I think
the connection is being closed on the client side at this point, and
when another LDAP request actually comes in it makes a new connection.  

The apache server is extremely busy, it just rarely talks to the LDAP
server. 

-----Original Message-----
From: fedora-directory-users-bounces@redhat.com
[mailto:fedora-directory-users-bounces@redhat.com] On Behalf Of Rob
Crittenden
Sent: Tuesday, August 14, 2007 8:31 AM
To: General discussion list for the Fedora Directory server project.
Subject: Re: [Fedora-directory-users] directory server setting fail
toterminateidle connections

Brian Fender wrote:
> This is actually an apache webserver making the connections to
directory 
> server.  What I see through tcpdump and netstat is that apache creates
> an LDAP connection to perform a search, and the connection sits idle
for 
> hours in established state.  The webserver eventually re-uses the
random 
> port it made the initial request on to talk to a client, so the LDAP 
> connection no longer shoes up as established on the client side.  On
the 
> server side, however, it still shows the connection as established 
> forever.  There are many other apache children talking to the same
LDAP 
> server in parallel, and the number of open filehandles constantly 
> increases. 
> 
>  
> 
> I realize that it is possible that the webserver is not properly
tearing 
> the connection down or a firewall may be blocking it, but
shouldn''t
the 
> server application notice that that connection was idle for more than 
> 20min and time it out anyway?
>
You want Apache to keep the connections open. It creates a pool of LDAP 
connections to use for all authentication. TCP/IP connections are 
expensive so it keeps them open to issue search and bind requests when 
doing authentication.

There may be a bug in the pooling code but how many connections are we 
talking about?

rob