dovecot - Apr 2014 - Segmentation fault running doveadm index (lucene) on a big mailbox

Hi,

on a server running dovecot 2.2.12 I have a user with a quite big
mailbox (~37000 Mails in the INBOX).

I tried to enable full text search using the fts_lucene backend (dovecot
2.2.12).


However, the `doveadm -v index -u <username> INBOX` always runs into a
segmentation fault after mail number 24200:

I tried to get a backtrace using gdb:

doveadm(<username>): Info: INBOX: Caching mails seq=1..37578
24200/37578
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff6612151 in lucene_index_build_more () from
/usr/lib64/dovecot/lib21_fts_lucene_plugin.so
(gdb) bt
#0  0x00007ffff6612151 in lucene_index_build_more () from
/usr/lib64/dovecot/lib21_fts_lucene_plugin.so
#1  0x00007ffff6610b01 in ?? () from
/usr/lib64/dovecot/lib21_fts_lucene_plugin.so
#2  0x00007ffff6820d5b in fts_build_mail () from
/usr/lib64/dovecot/lib20_fts_plugin.so
#3  0x00007ffff6825b90 in ?? () from /usr/lib64/dovecot/lib20_fts_plugin.so
#4  0x000000000041650a in ?? ()
#5  0x00000000004123a7 in _start ()


`ulimit` reports "unlimited", so ulimits shouldn't be the culprit.
Is
there another limit in dovecot?

Unfortunately, I can't "resume" the process, the indexing always
starts
again from the beginning.

Florian



doveconf -n:

# 2.2.12: /etc/dovecot/dovecot.conf
# OS: Linux 3.14.0-gentoo x86_64 Gentoo Base System release 2.2 ext4
auth_mechanisms = plain login cram-md5 ntlm
mail_gid = 900
mail_location = maildir:/home/vmail/%u
mail_plugins = acl quota fts fts_lucene
mail_uid = 900
maildir_stat_dirs = yes
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope
encoded-character vacation subaddress comparator-i;ascii-numeric
relational regex imap4flags copy include variables body enotify
environment mailbox date ihave
namespace {
  hidden = no
  list = children
  location = maildir:/home/vmail/%%u/:INDEX=/home/vmail/%%u/
  prefix = shared/%%u/
  separator = /
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  list = yes
  location   mailbox Drafts {
    special_use = \Drafts
  }
  mailbox Junk {
    special_use = \Junk
  }
  mailbox Sent {
    special_use = \Sent
  }
  mailbox "Sent Messages" {
    special_use = \Sent
  }
  mailbox Trash {
    special_use = \Trash
  }
  prefix   separator = /
  subscriptions = yes
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  acl = vfile
  acl_shared_dict = file:/home/vmail/shared-mailboxes
  fts = lucene
  fts_lucene = whitespace_chars=@.
  quota = maildir
  quota_rule = *:storage=1G
  quota_rule2 = Trash:storage=+100M
  quota_warning = storage=95%% quota-warning 95 %u
  quota_warning2 = storage=80%% quota-warning 80 %u
  sieve = /home/vmail/%u/.dovecot.sieve
  sieve_dir = /home/vmail/%u/sieve
}
postmaster_address = postmaster@<domain>
protocols = imap pop3 lmtp sieve sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0666
    user = postfix
  }
  unix_listener auth-master {
    group = vmail
    mode = 0600
    user = vmail
  }
}
service imap-postlogin {
  executable = script-login /usr/local/bin/imap-postlogin.sh
  user = $default_internal_user
}
service imap {
  executable = imap imap-postlogin
}
service managesieve-login {
  inet_listener sieve {
    port = 4190
  }
}
service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  unix_listener quota-warning {
    user = vmail
  }
  user = dovecot
}
ssl = required
ssl_cert = </etc/ssl/<domain>/chain.crt
ssl_key = </etc/ssl/<domain>/server.key
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocol lda {
  auth_socket_path = /var/run/dovecot/auth-master
  mail_plugins = quota sieve acl
}
protocol imap {
  mail_plugins = acl quota quota imap_quota imap_acl
}

On 04/15/2014 04:35 PM, Florian Klink wrote:
> Hi,
> 
> on a server running dovecot 2.2.12 I have a user with a quite big
> mailbox (~37000 Mails in the INBOX).
> 
> I tried to enable full text search using the fts_lucene backend (dovecot
> 2.2.12).
This patch (to be included in 2.2.13) addresses a different Lucene error, but
might mitigate your issue as well.
http://hg.dovecot.org/dovecot-2.2/rev/d63b209737be

If the issue remains and if possible, install dovecot dbg packages to get
symbols and run a "bt full" instead of "bt".

br,
Teemu Huovila