dovecot - Apr 2014 - Migration to Dovecot 2.2.12 - How to trigger full site indexing

Hi list,

We are in the process of migrating our old (dovecot 1.2.4 based) mail 
system to the new one:

- Centos 6 x86_64
- Dovecot 2.2.12
- Users in an openldap directory
- clucene FTS

I'm trying to trigger a full indexing for all the user and all their 
mailboxes (avoiding them to individually trigger it using a search 
within their MUA). I'm trying to use the "doveadm index" command.

First question, is there some sort of 'wildcard' option for the name of 
the mailbox to index, as I dont know all the mailboxes's names the users 
have created (beside the regular Sent Inbox Trash and Drafts ?

Second question I've tried:
# doveadm -D index -A Sent
which gave me:
doveadm(root): Error: User listing returned failure
doveadm: Error: Failed to iterate through some users

Can the '-A' option work with ldap backend ?

Thanks a lot and have a nice day,
kfx

---------------------------------------------
# 2.2.12: /usr/local/etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-431.11.2.el6.x86_64 x86_64 CentOS release 6.5 (Final) 
ext4
auth_cache_size = 10 M
auth_debug = yes
auth_socket_path = /var/run/dovecot/auth-userdb
auth_verbose = yes
auth_verbose_passwords = sha1
base_dir = /var/run/dovecot/
debug_log_path = /var/log/dovecot-debug.log
default_vsz_limit = 2 G
hostname = mydomain.com
info_log_path = /var/log/dovecot.log
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
listen = *
log_path = /var/log/dovecot-error.log
login_greeting = Welcome
login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c
mail_debug = yes
mail_gid = vmail
mail_location = maildir:/var/vmail/%Ln
mail_plugins = fts fts_lucene quota stats
mail_uid = vmail
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope 
encoded-character vacation subaddress comparator-i;ascii-numeric 
relational regex imap4flags copy include variables body enotify 
environment mailbox date ihave
namespace {
   inbox = yes
   location = maildir:/var/vmail/%Ln
   mailbox Drafts {
     auto = subscribe
     special_use = \Drafts
   }
   mailbox Sent {
     auto = subscribe
     special_use = \Sent
   }
   mailbox Trash {
     auto = subscribe
     special_use = \Trash
   }
   prefix    type = private
}
passdb {
   args = /usr/local/etc/dovecot/deny-users
   deny = yes
   driver = passwd-file
}
passdb {
   args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
   driver = ldap
}
plugin {
   fts = lucene
   fts_autoindex = yes
   fts_autoindex_max_recent = 1000
   fts_lucene = whitespace_chars=@
   quota = maildir:User quota
   quota_rule = *:storage=20GB
   quota_rule2 = Trash:storage=+200M
   quota_warning = storage=95%% quota-warning 95 %u
   quota_warning2 = storage=80%% quota-warning 80 %u
   sieve = /var/sieve-scripts/%Ln.sieve
   sieve_dir = /var/vmail/%Ln/home/sieve
   stats_command_min_time = 1 mins
   stats_domain_min_time = 12 hours
   stats_ip_min_time = 12 hours
   stats_memory_limit = 16 M
   stats_refresh = 30 secs
   stats_session_min_time = 15 mins
   stats_track_cmds = yes
   stats_user_min_time = 1 hours
}
postmaster_address = postmaster at mydomain.com
protocols = imap pop3 sieve sieve
service auth-worker {
   user = root
}
service auth {
   unix_listener /var/spool/postfix/private/auth {
     mode = 0777
     user = vmail
   }
}
service imap-login {
   inet_listener imaps {
     port = 993
     ssl = yes
   }
   process_min_avail = 5
}
service managesieve-login {
   inet_listener sieve {
     port = 4190
   }
}
service pop3-login {
   inet_listener pop3s {
     port = 995
     ssl = yes
   }
}
service quota-warning {
   executable = script /usr/local/bin/quota-warning.sh
   user = nobody
}
service stats {
   fifo_listener stats-mail {
     mode = 0600
     user = vmail
   }
}
ssl_cert = </etc/pki/tls/certs/mail.mydomain.com.pem
ssl_key = </etc/pki/tls/private/mail.mydomain.com.key
userdb {
   args = uid=12347 gid=12347 home=/var/vmail/%Ln/home allow_all_users=yes
   driver = static
}
verbose_proctitle = yes
protocol lda {
   mail_plugins = fts fts_lucene quota stats sieve virtual
}
protocol imap {
   mail_plugins = fts fts_lucene quota stats imap_quota virtual imap_stats
}

On Wednesday, April 9, 2014 12:57:13 PM CEST, kadafax at gmail.com
wrote:
> Hi list,
>
> We are in the process of migrating our old (dovecot 1.2.4 
> based) mail system to the new one:
>
> - Centos 6 x86_64
> - Dovecot 2.2.12
> - Users in an openldap directory
> - clucene FTS
>
> I'm trying to trigger a full indexing for all the user and all 
> their mailboxes (avoiding them to individually trigger it using 
> a search within their MUA). I'm trying to use the "doveadm 
> index" command.
>
> First question, is there some sort of 'wildcard' option for the 
> name of the mailbox to index, as I dont know all the mailboxes's 
> names the users have created (beside the regular Sent Inbox 
> Trash and Drafts ?
I'm using a totally different set of things, but at least dovecot and 
clucene are agreeable.. ;-)

You should have a fts plugin to doveadm that should solve this issue:

doveadm fts rescan
doveadm fts optimize

That's what I use to trigger rebuilds.
> Second question I've tried:
> # doveadm -D index -A Sent
> which gave me:
> doveadm(root): Error: User listing returned failure
> doveadm: Error: Failed to iterate through some users
>
> Can the '-A' option work with ldap backend ?
http://wiki2.dovecot.org/AuthDatabase/LDAP/Userdb

# For using doveadm -A:
iterate_attrs = uid=user
iterate_filter = (objectClass=posixAccount)

Are you defining iterate_* for your userdb?

Regards,
Ben