dovecot - Dec 2019 - Dovecot 2.3.8 - How to force index creation for user/mailbox?

Neither of dovecot.index files are created (.cache, .log, .thread) - these
files are created when I open the folder with IMAP client. When mailbox is
not used for long time, the indexing takes very long time. So I'm looking
for any solution (preferably with doveadm) to manually reindex folders in
background.

wt., 3 gru 2019 o 20:23 Aki Tuomi <aki.tuomi at open-xchange.com>
napisa?(a):
> Wait, did you mean 'dovecot.index' file? That file is only created
> occasionally, the actual index is kept in dovecot.index.log.
>
> Aki
>
> > On 03/12/2019 21:20 Tomek Lutelmowski via dovecot <dovecot at
dovecot.org>
> wrote:
> >
> >
> > Thanks, but both commands didn't create dovecot.index. Only when I
open
> folder with IMAP client, the dovecot.index is created.
> >
> >
> > wt., 3 gru 2019 o 18:16 Aki Tuomi <aki.tuomi at
open-xchange.com>
> napisa?(a):
> > >
> > >  > On 03/12/2019 18:30 Tomek Lutelmowski via dovecot <
> dovecot at dovecot.org> wrote:
> > >  >
> > >  >
> > >  > Hello,
> > >  >
> > >  > I'm using maildirs, and I have problems with big
mailboxes (~ 20000
> emails, 30-50GB) - sometimes access to such mailbox takes very long (~10
> minutes), I would like to be able to create index for mailbox in
> background, but I couldn't find any way to do it. So far I tried:
> > >  >
> > >  > doveadm -D -vvv index -u user 'Folder'
> > >  >
> > >  > But even if this folder have no dovecot.index* files - I
always get:
> > >  >
> > >  > doveadm(user): Info: Wersje robocze: Cache is already up to
date
> > >  >
> > >  > It seems that doveadm doesn't actually checks if the
index is
> present. The index is created only when I log with IMAP client and check
> the folder.
> > >  >
> > >  > So i tried:
> > >  > doveadm mailbox cache purge -u user 'Folder'
> > >  >
> > >  > No joy either:
> > >  > doveadm(user): Error: Cache is unusable
> > >  >
> > >  > And command below doesn't actually do anything to
index:
> > >  > doveadm -vvv force-resync -u user 'Folder'
> > >  >
> > >  > Is there any way to use doveadm tool to create/fix index?
Shouldn't
> the index be updated on each email arrival?
> > >  >
> > >  > Cheers,
> > >  > TomekL
> > >
> > >  Try first 'doveadm fetch -u victim body ALL'. If this
does not
> produce any useful results, try 'doveadm force-resync -u victim
"*"'.
> > >
> > >  Aki
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20191203/be0f4ba4/attachment.html>

Hi,

then please post doveconf -n

Sami

> On 3 Dec 2019, at 21.31, Tomek Lutelmowski via dovecot <dovecot at
dovecot.org> wrote:
> 
> Neither of dovecot.index files are created (.cache, .log, .thread) - these
files are created when I open the folder with IMAP client. When mailbox is not
used for long time, the indexing takes very long time. So I'm looking for
any solution (preferably with doveadm) to manually reindex folders in
background.
> 
> wt., 3 gru 2019 o 20:23 Aki Tuomi <aki.tuomi at open-xchange.com
<mailto:aki.tuomi at open-xchange.com>> napisa?(a):
> Wait, did you mean 'dovecot.index' file? That file is only created
occasionally, the actual index is kept in dovecot.index.log.
> 
> Aki
> 
> > On 03/12/2019 21:20 Tomek Lutelmowski via dovecot <dovecot at
dovecot.org <mailto:dovecot at dovecot.org>> wrote:
> > 
> > 
> > Thanks, but both commands didn't create dovecot.index. Only when I
open folder with IMAP client, the dovecot.index is created.
> > 
> > 
> > wt., 3 gru 2019 o 18:16 Aki Tuomi <aki.tuomi at open-xchange.com
<mailto:aki.tuomi at open-xchange.com>> napisa?(a):
> > > 
> > >  > On 03/12/2019 18:30 Tomek Lutelmowski via dovecot
<dovecot at dovecot.org <mailto:dovecot at dovecot.org>> wrote:
> > >  > 
> > >  > 
> > >  > Hello,
> > >  > 
> > >  > I'm using maildirs, and I have problems with big
mailboxes (~ 20000 emails, 30-50GB) - sometimes access to such mailbox takes
very long (~10 minutes), I would like to be able to create index for mailbox in
background, but I couldn't find any way to do it. So far I tried:
> > >  > 
> > >  > doveadm -D -vvv index -u user 'Folder' 
> > >  > 
> > >  > But even if this folder have no dovecot.index* files - I
always get:
> > >  > 
> > >  > doveadm(user): Info: Wersje robocze: Cache is already up to
date
> > >  > 
> > >  > It seems that doveadm doesn't actually checks if the
index is present. The index is created only when I log with IMAP client and
check the folder.
> > >  > 
> > >  > So i tried:
> > >  > doveadm mailbox cache purge -u user 'Folder' 
> > >  > 
> > >  > No joy either:
> > >  > doveadm(user): Error: Cache is unusable
> > >  > 
> > >  > And command below doesn't actually do anything to
index:
> > >  > doveadm -vvv force-resync -u user 'Folder'
> > >  > 
> > >  > Is there any way to use doveadm tool to create/fix index?
Shouldn't the index be updated on each email arrival?
> > >  > 
> > >  > Cheers,
> > >  > TomekL
> > >  
> > >  Try first 'doveadm fetch -u victim body ALL'. If this
does not produce any useful results, try 'doveadm force-resync -u victim
"*"'.
> > >  
> > >  Aki
> > >
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20191204/f5510a9f/attachment.html>

Here is doveconf -n output:

# 2.3.8 (9df20d2db): /etc/dovecot/dovecot.conf
# Pigeonhole version 0.5.8 (b7b03ba2)
# OS: Linux 5.4.1-gentoo x86_64 Gentoo Base System release 2.6
# Hostname: mail.xxxx.com
auth_cache_negative_ttl = 0
auth_cache_size = 10 M
auth_cache_ttl = 4 hours
auth_default_realm = xxxx.com
auth_master_user_separator = *
auth_verbose = yes
auth_verbose_passwords = plain
default_vsz_limit = 1 G
disable_plaintext_auth = no
listen = *
login_greeting = IMAP ready.
mail_gid = mail
mail_location = maildir:~
mail_plugins = zlib fts fts_lucene virtual acl
mailbox_idle_check_interval = 3 secs
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags
copy include variables body enotify environment mailbox date index ihave
duplicate mime foreverypart extracttext
namespace {
  list = children
  location = maildir:/e/xxxx.com/mail-shared:INDEXPVT=~/Maildir/public/
  mailbox poczta {
    auto = no
  }
  prefix = Wsp?lne/
  separator = /
  subscriptions = no
  type = shared
}
namespace inbox {
  inbox = yes
  list = yes
  location   mailbox "Elementy usuni?te" {
    auto = subscribe
    special_use = \Trash
  }
  mailbox "Elementy wys?ane" {
    auto = subscribe
    special_use = \Sent
  }
  mailbox "Wersje robocze" {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Wiadomo?ci-?mieci {
    auto = subscribe
    special_use = \Junk
  }
  prefix   separator = /
  type = private
}
passdb {
  args = /etc/dovecot/dovecot-sql-master.conf.ext
  driver = sql
  master = yes
  pass = yes
}
passdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
plugin {
  acl = vfile
  fts = lucene
  fts_autoindex = yes
  fts_lucene = whitespace_chars="@.-_()[]{}<>/\\+"
  mail_log_fields = uid box msgid from subject size
  sieve = file:~/sieve;active=~/sieve/sieve.active
  sieve_before = /etc/dovecot/sieve/global.sieve
  sieve_global = /etc/dovecot/sieve
  sieve_vacation_dont_check_recipient = yes
  sieve_vacation_use_original_recipient = yes
  zlib_save = xz
  zlib_save_level = 9
}
pop3_uidl_format = %v.%u
protocols = imap lmtp sieve
service auth {
  unix_listener /var/spool/postfix/private/auth {
    group = postfix
    mode = 0660
    user = postfix
  }
}
service imap-login {
  process_min_avail = 10
  service_count = 1
  unix_listener imap {
    mode = 0666
  }
}
service imap-postlogin {
  executable = script-login /usr/local/bin/postlogin.sh
  user = $default_internal_user
}
service imap {
  client_limit = 1
  executable = imap imap-postlogin
}
service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    group = postfix
    mode = 0600
    user = postfix
  }
  vsz_limit = 4 G
}
service managesieve-login {
  inet_listener sieve {
    address = 127.0.0.1
    port = 4190
  }
}
ssl_ca = </e/xxxx.com/cert/CA.pem
ssl_cert = </e/ xxxx.com/cert/mail-chain.crt
ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL
ssl_dh = # hidden, use -P to show it
ssl_key = # hidden, use -P to show it
ssl_require_crl = no
userdb {
  driver = prefetch
}
userdb {
  args = /etc/dovecot/dovecot-sql.conf.ext
  driver = sql
}
protocol lmtp {
  mail_plugins = zlib fts fts_lucene virtual acl sieve virtual
}
protocol lda {
  mail_plugins = zlib fts fts_lucene virtual acl sieve
}
protocol imap {
  mail_plugins = zlib fts fts_lucene virtual acl imap_acl
}

?r., 4 gru 2019 o 07:34 Sami Ketola <sami.ketola at dovecot.fi>
napisa?(a):
> Hi,
>
> then please post doveconf -n
>
> Sami
>
>
> On 3 Dec 2019, at 21.31, Tomek Lutelmowski via dovecot <
> dovecot at dovecot.org> wrote:
>
> Neither of dovecot.index files are created (.cache, .log, .thread) - these
> files are created when I open the folder with IMAP client. When mailbox is
> not used for long time, the indexing takes very long time. So I'm
looking
> for any solution (preferably with doveadm) to manually reindex folders in
> background.
>
> wt., 3 gru 2019 o 20:23 Aki Tuomi <aki.tuomi at open-xchange.com>
napisa?(a):
>
>> Wait, did you mean 'dovecot.index' file? That file is only
created
>> occasionally, the actual index is kept in dovecot.index.log.
>>
>> Aki
>>
>> > On 03/12/2019 21:20 Tomek Lutelmowski via dovecot <dovecot at
dovecot.org>
>> wrote:
>> >
>> >
>> > Thanks, but both commands didn't create dovecot.index. Only
when I open
>> folder with IMAP client, the dovecot.index is created.
>> >
>> >
>> > wt., 3 gru 2019 o 18:16 Aki Tuomi <aki.tuomi at
open-xchange.com>
>> napisa?(a):
>> > >
>> > >  > On 03/12/2019 18:30 Tomek Lutelmowski via dovecot <
>> dovecot at dovecot.org> wrote:
>> > >  >
>> > >  >
>> > >  > Hello,
>> > >  >
>> > >  > I'm using maildirs, and I have problems with big
mailboxes (~
>> 20000 emails, 30-50GB) - sometimes access to such mailbox takes very
long
>> (~10 minutes), I would like to be able to create index for mailbox in
>> background, but I couldn't find any way to do it. So far I tried:
>> > >  >
>> > >  > doveadm -D -vvv index -u user 'Folder'
>> > >  >
>> > >  > But even if this folder have no dovecot.index* files -
I always
>> get:
>> > >  >
>> > >  > doveadm(user): Info: Wersje robocze: Cache is already
up to date
>> > >  >
>> > >  > It seems that doveadm doesn't actually checks if
the index is
>> present. The index is created only when I log with IMAP client and
check
>> the folder.
>> > >  >
>> > >  > So i tried:
>> > >  > doveadm mailbox cache purge -u user 'Folder'
>> > >  >
>> > >  > No joy either:
>> > >  > doveadm(user): Error: Cache is unusable
>> > >  >
>> > >  > And command below doesn't actually do anything to
index:
>> > >  > doveadm -vvv force-resync -u user 'Folder'
>> > >  >
>> > >  > Is there any way to use doveadm tool to create/fix
index?
>> Shouldn't the index be updated on each email arrival?
>> > >  >
>> > >  > Cheers,
>> > >  > TomekL
>> > >
>> > >  Try first 'doveadm fetch -u victim body ALL'. If
this does not
>> produce any useful results, try 'doveadm force-resync -u victim
"*"'.
>> > >
>> > >  Aki
>> > >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://dovecot.org/pipermail/dovecot/attachments/20191204/13d80e79/attachment.html>