dovecot - Feb 2011 - Auth processes/login processes count

Hello Timo,

I'm running dovecot-1.2.12/Maildir/LDAP-passdb/LDAP-prefetched-userdb and
I'm
experiencing "random" latency (which even seem to make squirrelmail
timeout
sometimes) at login without noticing particular load on the server.

LDAP backends are running fine as far as I know so I was investigating maybe
mistuned limits :

Dovecot is configured (for ~3000 users) with

  . each login processed in its own process :

      #login_process_per_connection = yes
      #login_max_connections = 256

   . counts as :

      #login_processes_count = 3
      login_max_processes_count = 1024

      auth default {
      [...]
      count = 5
      }

I've got several questions :

1) how can I count more than login_max_processes_count processes ?

  # ps aux | egrep -i '(pop3|imap)-login' | wc -l
  # 1031

2) could be my auth process count the bottleneck ?
   is there a rule of thumb to deduce it from the login process count ?

3) on dovecot login/master/auth design, is my understanding correct ?

- dovecot-1.2.x :

  . passdb and userdb lookup are done by the auth process, on behalf of either
the
    login process which wants to know if the client's password is correct or
of the
    master process which wants to check if it was actually correct and which
wants
    to have the userdb info.

  . client connection fd is sent to the master which creates the mail process
and
    transfers the fd to it.

- dovecot-2.x :

  . client connection fd does not go through the master anymore but through a
socket in the login/ directory

  . it's the mail process which ask the auth process for the userdb lookup 

4) with LDAP backend using {CRYPT} passwords dans with plaintext auth enabled in
dovecot,
   is the clear password crypted before comparing it to the LDAP crypted
password
   or does dovecot try to simply bind to LDAP with the clear password ?

Thanks

-- 
Thomas Hummel 	    | Institut Pasteur
<hummel at pasteur.fr> | Groupe Exploitation et Infrastructure

On 15.2.2011, at 16.19, Thomas Hummel wrote:
>      login_max_processes_count = 1024
> 
>      auth default {
>      [...]
>      count = 5
>      }
> 
> I've got several questions :
> 
> 1) how can I count more than login_max_processes_count processes ?
> 
>  # ps aux | egrep -i '(pop3|imap)-login' | wc -l
>  # 1031
It's max. 1024 for imap-login and max. 1024 for pop3-login.
> 2) could be my auth process count the bottleneck ?
>   is there a rule of thumb to deduce it from the login process count ?
Well, with v2.0 you couldn't even have more than 1 auth process. LDAP is
used asynchronously, so it should work ok. Although are you using auth binds?
That's less efficient.

I think anyway the problem has to do with login processes. In v1.x new processes
are forked in a bit dummy way, which I just copied from Apache but I don't
think it really helps with anything.. Anyway, it can only double the process
count once every second. So if there's a large burst of connections, some of
them start hanging. I think you should just try if
login_process_per_connection=no solves everything. Yeah, it's less secure
than "yes", but in practice that's pretty much what all large
installations use anyway. There are no known Dovecot or OpenSSL security holes
now, so it should be safe. :)
> 3) on dovecot login/master/auth design, is my understanding correct ?
> 
> - dovecot-1.2.x :
> 
>  . passdb and userdb lookup are done by the auth process, on behalf of
either the
>    login process which wants to know if the client's password is
correct or of the
>    master process which wants to check if it was actually correct and which
wants
>    to have the userdb info.
> 
>  . client connection fd is sent to the master which creates the mail
process and
>    transfers the fd to it.
Yes.
> - dovecot-2.x :
> 
>  . client connection fd does not go through the master anymore but through
a socket in the login/ directory
> 
>  . it's the mail process which ask the auth process for the userdb
lookup
Yep. The code and idea is similar, but now it's running in the mail process
rather than master process. This change also removed the possibility of having
more than one auth process (if it becomes necessary, I guess they could be added
back by having each auth process listen on a differently named unix socket, like
auth-master-<pid>).
> 4) with LDAP backend using {CRYPT} passwords dans with plaintext auth
enabled in dovecot,
>   is the clear password crypted before comparing it to the LDAP crypted
password
Yes, if auth_bind=no.
>   or does dovecot try to simply bind to LDAP with the clear password ?
Yes, if auth_bind=yes.