dovecot - Jun 2013 - Dovecot SLOW in imaptest without any apparent reason

Hello,

I'm migrating a mail server from a centos 5 cluster architecture to a
centos 6 cluster architecture. The new cluster involves faster machines
then the old cluster, and a virtual machine.
I use dovecot-2.0.9-5.el6.x86_64, while the old cluster uses
dovecot-2.0.1-1_118.el5.
Tha mail server uses mysql for the users database, and a local ldap for
authentication.

The storage is also much faster in the new cluster:
OLD SERVER: date; dd if=/dev/zero of=/cl/prova.bin bs=102400 count=10240;
sync; date
Wed Jun 26 15:43:53 CEST 2013
10240+0 records in
10240+0 records out
1048576000 bytes (1.0 GB) copied, 30.0202 seconds, 34.9 MB/s
Wed Jun 26 15:44:28 CEST 2013
--> 1GB / 35 secs = about 29.25 MB/sec

NEW SERVER: date; dd if=/dev/zero of=/cl/prova.bin bs=102400 count=102400;
sync; date
mar 25 giu 2013, 15.46.36, CEST
102400+0 records in
102400+0 records out
10485760000 bytes (10 GB) copied, 62,5143 s, 168 MB/s
mar 25 giu 2013, 15.48.33, CEST
[root at eta ~]#
---> 10GB / 117secs = about 85.47MB/sec


An 'ls -l' in the user direcotry is also much faster, when the ldap/sssd
cache is populated:

OLD SERVER:
real    0m8.540s
user    0m1.229s
sys     0m0.699s

NEW SERER:
real    0m3.938s
user    0m0.250s
sys     0m0.151s
But my imaptest is much slower:

./imaptest user=XXXXXXX pass=XXXXXXX secs=120 seed=123 mbox=./dovecot-crlf
OLD SERVER:

Totals:
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo
100%  50%  50% 100% 100% 100%  50% 100% 100% 100% 100%
                          30%                  5%
1122  559  547 1121 1121 1568  136  900 1121 1121 2244
NEW SERVER
Totals:
Logi List Stat Sele Fetc Fet2 Stor Dele Expu Appe Logo
100%  50%  50% 100% 100% 100%  50% 100% 100% 100% 100%
                          30%                  5%
 390  210  185  389  389  589   49  320  389  389  780

The configuration is exactly the same in the 2 clusters, that is:

-----------
# 2.0.1: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.18-348.3.1.el5 x86_64 Red Hat Enterprise Linux Server
release 5.9 (Tikanga) xfs
auth_cache_size = 1024
auth_cache_ttl = 21600 s
auth_debug = yes
auth_debug_passwords = yes
auth_master_user_separator = *
auth_mechanisms = plain login
auth_socket_path = /var/run/dovecot/auth-userdb
auth_verbose = yes
base_dir = /var/run/dovecot/
disable_plaintext_auth = no
hostname = mail.unimore.it
info_log_path = /var/log/dovecot.log
lda_mailbox_autocreate = yes
log_path = /var/log/dovecot.log
mail_debug = yes
mail_location = maildir:/cl/mail/vhosts/sms.unimo.it/%Ln/Maildir
mail_plugins = $mail_plugins quota
mailbox_idle_check_interval = 60 s
mbox_write_locks = fcntl
namespace {
  inbox = yes
  location   prefix = INBOX.
  separator = .
  type = private
}
passdb {
  args = /usr/local/etc/dovecot.masterusers
  driver = passwd-file
  master = yes
}
passdb {
  args = dovecot
  driver = pam
}
plugin {
  quota = maildir:User quota
  quota_exceeded_message = Quota exceeded (mailbox is full)
  quota_rule = *:storage=200MB
  quota_rule2 = *:messages=100000
  quota_rule3 = INBOX.Trash:storage=+100M
  quota_warning = storage=90%% quota-warning 90 %u
  quota_warning2 = storage=85%% quota-warning 85 %u
  quota_warning3 = messages=95%% quota-warning 95 %u
  quota_warning4 = messages=80%% quota-warning 80 %u
  setting_name = quota
}
postmaster_address = postmaster at unimore.it
quota_full_tempfail = yes
service anvil {
  client_limit = 199999
  process_limit = 1
}
service auth {
  client_limit = 21224
  unix_listener auth-userdb {
    mode = 0600
    user = vmail
  }
}
service imap-login {
  inet_listener imap {
    port = 143
  }
  inet_listener imaps {
    port = 993
  }
  process_limit = 10000
}
service imap {
  process_limit = 10000
}
service pop3-login {
  inet_listener pop3 {
    port = 110
  }
  inet_listener pop3s {
    port = 995
  }
}
service pop3 {
  process_limit = 1024
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
    group = vmail
    user = vmail
  }
  user = dovecot
}
ssl_ca = </etc/pki/tls/certs/ca_unimore_tcs.pem
ssl_cert = </etc/pki/tls/certs/mailposta1.pem
ssl_key = </etc/pki/tls/certs/mailposta1.key
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
verbose_ssl = yes
protocol lda {
  mail_plugins = $mail_plugins quota
}
protocol imap {
  mail_plugins = $mail_plugins
}
protocol pop3 {
  mail_plugins = $mail_plugins quota
  pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
  pop3_uidl_format = %08Xu%08Xv
}
-----------
Is anyone able to tell me what I have to check?

thanks in advance

Fabio Ferrari

On 28.6.2013, at 13.28, FABIO FERRARI <fabio.ferrari at unimore.it> wrote:
> I'm migrating a mail server from a centos 5 cluster architecture to a
> centos 6 cluster architecture. The new cluster involves faster machines
> then the old cluster, and a virtual machine.
> I use dovecot-2.0.9-5.el6.x86_64, while the old cluster uses
> dovecot-2.0.1-1_118.el5.
Since both are v2.0.x there aren't any big differences in how they work.
> Tha mail server uses mysql for the users database, and a local ldap for
> authentication.
..
> ./imaptest user=XXXXXXX pass=XXXXXXX secs=120 seed=123 mbox=./dovecot-crlf
Test if the difference is in authentication or non-authentication:

auth: imaptest user=XX pass=XX secs=120 seed=123 - select=0

non-auth: imaptest user=XX pass=XX secs=120 seed=123 logout=0

My guess is that it's authentication related.