dovecot - Dec 2011 - Corrupted index cache file and high CPU usage

I suspect, that dovecot v2.0.16 under Linux (tested on Gentoo 3.0.4
Hardened, Gentoo 3.0.6, Gentoo 3.1.4) causes 100% cpu utilization when
index.cache is broken and doveadm is lanched. Dovecot uses LDAP as the
userdb and provides IMAP and LDA. Because each user has a quota, the
doveadm's expunge functionality is used to get rid of old trashed mail.

The mail log:
Dec 12 07:38:59 www dovecot: imap(user): Error: Corrupted index cache file
/var/data/mail/domain.tld/user/mail/dovecot.index.cache: invalid record size

Doveadm is launched as:
doveadm -D expunge -A mailbox Trash savedbefore 1w

Mail is stored on ext4 and flush will use all of the CPU capacity like this:
load average: 4.00, 4.01, 4.05

ps:
root     26401 98.2  0.0      0     0 ?        R    Dec20 1369:49
[flush-253:1]

lsblk:
vg_data-mail (dm-1)       253:1    0   500M  0 /var/data/mail

dovecot -n:
# 2.0.16: /etc/dovecot/dovecot.conf
# OS: Linux 3.0.4-hardened-r5 x86_64 Gentoo Base System release 2.0.3
auth_debug = yes
auth_debug_passwords = yes
auth_verbose = yes
hostname = mail.domain.tld
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *
mail_debug = yes
mail_gid = vmail
mail_home = /var/data/mail/domain.tld/%n
mail_location = maildir:~/mail
mail_plugins = " mail_log notify zlib quota"
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date ihave
passdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
plugin {
  autocreate = Sent
  autocreate2 = Trash
  autocreate3 = Spam
  autosubscribe = Sent
  autosubscribe2 = Trash
  autosubscribe3 = Spam
  fts = squat
  fts_squat = partial=4 full=10
  quota = maildir:User quota
  quota_rule = *:storage=100M
  quota_rule2 = Trash:ignore
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = ~/.dovecot.sieve
  sieve_dir = ~/sieve
  sieve_quota_max_scripts = 5
  sieve_quota_max_storage = 5M
}
postmaster_address = postmaster at domain.tld
protocols = imap sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
  unix_listener auth-userdb {
    group = vmail
    mode = 0600
    user = vmail
  }
}
service quota-warning {
  executable = script /etc/dovecot/quota-warning.sh
  unix_listener quota-warning {
    user = vmail
  }
  user = vmail
}
ssl_cert = </etc/postfix/mail.domain.tld-cert.pem
ssl_key = </etc/postfix/mail.domain.tld-key.pem

userdb {
  args = /etc/dovecot/dovecot-ldap.conf.ext
  driver = ldap
}
verbose_proctitle = yes
verbose_ssl = yes
protocol lda {
  info_log_path   log_path   mail_plugins = " mail_log notify zlib quota
sieve"
}
protocol imap {
  mail_plugins = " mail_log notify zlib quota autocreate fts fts_squat
imap_zlib imap_quota"
}

dovecot-ldap.conf.ext:
hosts = localhost
dn = user
dnpass = pass
base = base
auth_bind = yes
user_attrs
homeDirectory=home,uidNumber=uid,gidNumber=gid,mailboxQuota=quota_rule=*:bytes=%$
user_filter = (&(objectClass=inetOrgPerson)(uid=%u))
pass_attrs = uid=user
pass_filter = (&(objectClass=inetOrgPerson)(uid=%u))
iterate_attrs = uid=user
iterate_filter = (objectClass=inetOrgPerson)

I suspect, that this is something to do with Dovecot, because after
deleting the dovecot.index.cache file, everything went back to normal. When
this happens, I cannot unmount the drive nor a system reboot works.

Thank you !

On 21.12.2011, at 18.38, hydra wrote:
> I suspect, that dovecot v2.0.16 under Linux (tested on Gentoo 3.0.4
> Hardened, Gentoo 3.0.6, Gentoo 3.1.4) causes 100% cpu utilization when
> index.cache is broken and doveadm is lanched. Dovecot uses LDAP as the
> userdb and provides IMAP and LDA. Because each user has a quota, the
> doveadm's expunge functionality is used to get rid of old trashed mail.
> 
> The mail log:
> Dec 12 07:38:59 www dovecot: imap(user): Error: Corrupted index cache file
> /var/data/mail/domain.tld/user/mail/dovecot.index.cache: invalid record
size
OK..
> Mail is stored on ext4 and flush will use all of the CPU capacity like
this:
> load average: 4.00, 4.01, 4.05
> 
> ps:
> root     26401 98.2  0.0      0     0 ?        R    Dec20 1369:49
> [flush-253:1]
That's a kernel process..
> I suspect, that this is something to do with Dovecot, because after
> deleting the dovecot.index.cache file, everything went back to normal. When
> this happens, I cannot unmount the drive nor a system reboot works.
That's a kernel bug..

I think you're thinking it the wrong way: Dovecot isn't causing your
system to break. Your system is causing Dovecot to break. Faulty hardware or
faulty kernel.