Aki Tuomi
2018-May-21 10:38 UTC
Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
can you try turning on pluign { fts_enforced = yes } and repeat your test?
Aki
On 21.05.2018 13:31, kadafax at gmail.com wrote:> Le 21/05/2018 ? 11:46, Aki Tuomi a ?crit?:
>> That message is coming from the temporary raw user, not from the actual
>> user.
>>
>> How did you verify that the FTS indexes are not created or updated
>> automatically?
> I send an email containing a specific strings ('thahB8ea' in that
case)
> and search agains it:
> # doveadm search -u username mailbox INBOX body thahB8ea
> --> No result
> (same with integrated search on roundcube)
>
> Then I force a rescan, followed by the same search:
> # doveadm? fts rescan -u username
> # doveadm search -u username mailbox INBOX body thahB8ea
> (Long waiting time)
> --> e09cce0283e8695ab760000002deed92 29048
>
> So I conclude that autoindexing is not working.
>
> Thanks,
> kfx
>
>> Aki
>>
>>
>> On 04.05.2018 12:47, kadafax at gmail.com wrote:
>>> ?Bump. Can someone from dovecot dev team please respond something
to my
>>> problem ? I've provided all the possible information, and if
not please
>>> tell me.
>>> Or do you only provide support for your own FTS solution bundled
with
>>> the paid version of dovecot-pro ? If so, you should clearly state
it in
>>> the wiki.
>>>
>>> Thanks
>>>
>>> Le 19/04/2018 ? 10:30, kadafax at gmail.com a ?crit?:
>>>> I think I'm progressing: if I create a brand new user, send
her an
>>>> email, it will not be indexed right away. I need to doveadm
rescan /
>>>> index the Inbox before.
>>>> At the end of this email, a dovecot's debug log snipset
(grep'ed on this
>>>> user). In it I can read: "Debug: fts: Indexes disabled for
namespace ''
>>>> Can this be a problem ?
>>>>
>>>> Thanks
>>>>
>>>> doveconf -n output:
>>>> # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf
>>>> # Pigeonhole version 0.4.22 (22940fb7)
>>>> # OS: Linux 3.10.0-693.17.1.el7.x86_64 x86_64 CentOS Linux
release
>>>> 7.4.1708 (Core)? ext4
>>>> # Hostname: mailServer.myDomain
>>>> auth_master_user_separator = *
>>>> auth_socket_path = /var/run/dovecot/auth-userdb
>>>> auth_username_chars >>>>
abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@*
>>>> auth_verbose = yes
>>>> base_dir = /var/run/dovecot/
>>>> debug_log_path = /var/log/dovecot-debug.log
>>>> default_client_limit = 5120
>>>> default_process_limit = 1024
>>>> default_vsz_limit = 2 G
>>>> hostname = myDomain
>>>> info_log_path = /var/log/dovecot.log
>>>> lda_mailbox_autocreate = yes
>>>> lda_mailbox_autosubscribe = yes
>>>> listen = *
>>>> log_path = /var/log/dovecot-error.log
>>>> login_greeting = Welcome
>>>> login_log_format_elements = user=<%u> method=%m rip=%r
lip=%l mpid=%e %c
>>>> mail_gid = vmail
>>>> mail_location = maildir:/var/vmail/%Ln
>>>> mail_plugins = " quota stats fts fts_solr"
>>>> mail_uid = vmail
>>>> managesieve_notify_capability = mailto
>>>> managesieve_sieve_capability = fileinto reject envelope
>>>> encoded-character vacation subaddress
comparator-i;ascii-numeric
>>>> relational regex imap4flags copy include variables body enotify
>>>> environment mailbox date index ihave duplicate mime
foreverypart extracttext
>>>> namespace {
>>>> ? inbox = yes
>>>> ? location = maildir:/var/vmail/%Ln
>>>> ? mailbox Drafts {
>>>> ??? auto = subscribe
>>>> ??? special_use = \Drafts
>>>> ? }
>>>> ? mailbox Sent {
>>>> ??? auto = subscribe
>>>> ??? special_use = \Sent
>>>> ? }
>>>> ? mailbox "Sent Messages" {
>>>> ??? special_use = \Sent
>>>> ? }
>>>> ? mailbox Trash {
>>>> ??? auto = subscribe
>>>> ??? special_use = \Trash
>>>> ? }
>>>> ? prefix >>>> ? type = private
>>>> }
>>>> passdb {
>>>> ? args = /usr/local/etc/dovecot/deny-users
>>>> ? deny = yes
>>>> ? driver = passwd-file
>>>> }
>>>> passdb {
>>>> ? args = /usr/local/etc/dovecot/master-users
>>>> ? driver = passwd-file
>>>> ? master = yes
>>>> }
>>>> passdb {
>>>> ? args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext
>>>> ? driver = ldap
>>>> }
>>>> plugin {
>>>> ? fts = solr
>>>> ? fts_autoindex = yes
>>>> ? fts_solr = url=http://localhost:8983/solr/dovecot/
break-imap-search
>>>> ? quota = maildir:User quota
>>>> ? quota_rule = *:storage=20GB
>>>> ? quota_rule2 = Trash:storage=+200M
>>>> ? quota_warning = storage=95%% quota-warning 95 %u
>>>> ? quota_warning2 = storage=80%% quota-warning 80 %u
>>>> ? sieve = /var/sieve-scripts/%Ln.sieve
>>>> ? sieve_dir = /var/vmail/%Ln/home/sieve
>>>> ? stats_command_min_time = 1 mins
>>>> ? stats_domain_min_time = 12 hours
>>>> ? stats_ip_min_time = 12 hours
>>>> ? stats_memory_limit = 16 M
>>>> ? stats_refresh = 30 secs
>>>> ? stats_session_min_time = 15 mins
>>>> ? stats_track_cmds = yes
>>>> ? stats_user_min_time = 1 hours
>>>> }
>>>> postmaster_address = postmaster at myDomain
>>>> protocols = imap pop3 sieve sieve
>>>> service auth-worker {
>>>> ? user = root
>>>> }
>>>> service auth {
>>>> ? unix_listener /var/spool/postfix/private/auth {
>>>> ??? mode = 0777
>>>> ??? user = vmail
>>>> ? }
>>>> }
>>>> service imap-login {
>>>> ? inet_listener imaps {
>>>> ??? port = 993
>>>> ??? ssl = yes
>>>> ? }
>>>> ? process_min_avail = 10
>>>> }
>>>> service imap {
>>>> ? process_limit = 2128
>>>> }
>>>> service managesieve-login {
>>>> ? inet_listener sieve {
>>>> ??? port = 4190
>>>> ? }
>>>> }
>>>> service pop3-login {
>>>> ? inet_listener pop3s {
>>>> ??? port = 995
>>>> ??? ssl = yes
>>>> ? }
>>>> }
>>>> service quota-warning {
>>>> ? executable = script /usr/local/bin/quota-warning.sh
>>>> ? user = nobody
>>>> }
>>>> service stats {
>>>> ? fifo_listener stats-mail {
>>>> ??? mode = 0600
>>>> ??? user = vmail
>>>> ? }
>>>> }
>>>> ssl_cert =
</etc/letsencrypt/live/mailServer.myDomain/fullchain.pem
>>>> ssl_key =? # hidden, use -P to show it
>>>> userdb {
>>>> ? args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext
>>>> ? driver = ldap
>>>> ? override_fields = uid=vmail gid=vmail
home=/var/vmail/%Ln/home
>>>> }
>>>> verbose_proctitle = yes
>>>> protocol lda {
>>>> ? mail_plugins = " quota stats fts fts_solr sieve
virtual"
>>>> }
>>>> protocol imap {
>>>> ? mail_max_userip_connections = 20
>>>> ? mail_plugins = " quota stats fts fts_solr imap_quota
virtual imap_stats"
>>>> }
>>>> protocol sieve {
>>>> ? mail_max_userip_connections = 20
>>>> }
>>>>
>>>> DEBUG LOG:
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Loading
modules from
>>>> directory: /usr/local/lib/dovecot
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_sieve_plugin.so
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: auth USER
input:
>>>> NewUser at myDomain uid=992 gid=989
home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Effective
uid=992,
>>>> gid=989, home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota root:
name=User
>>>> quota backend=maildir args>>>> Apr 19 10:23:26
lda(NewUser at myDomain): Debug: Quota rule: root=User
>>>> quota mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota rule:
root=User
>>>> quota mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota warning:
>>>> bytes=20401094656 (95%) messages=0 reverse=no
command=quota-warning 95
>>>> NewUser at myDomain
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota warning:
>>>> bytes=17179869184 (80%) messages=0 reverse=no
command=quota-warning 80
>>>> NewUser at myDomain
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota grace:
root=User
>>>> quota bytes=2147483648 (10%)
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Namespace :
type=private,
>>>> prefix=, sep=, inbox=yes, hidden=no, list=yes,
subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:26
lda(NewUser at myDomain): Debug: quota: quota_over_flag
>>>> check: quota_over_script unset - skipping
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota root:
name=User
>>>> quota backend=maildir args>>>> Apr 19 10:23:26
lda(NewUser at myDomain): Debug: Quota rule: root=User
>>>> quota mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota rule:
root=User
>>>> quota mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota warning:
>>>> bytes=20401094656 (95%) messages=0 reverse=no
command=quota-warning 95
>>>> raw mail user
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota warning:
>>>> bytes=17179869184 (80%) messages=0 reverse=no
command=quota-warning 80
>>>> raw mail user
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Quota grace:
root=User
>>>> quota bytes=2147483648 (10%)
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: none: root=,
index=,
>>>> indexpvt=, control=, inbox=, alt>>>> Apr 19
10:23:26 lda(NewUser at myDomain): Debug: fts: Indexes disabled for
>>>> namespace ''
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: quota:
quota_over_flag
>>>> check: quota_over_script unset - skipping
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Destination
address:
>>>> NewUser at myDomain (source: -a parameter)
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve:
Pigeonhole version
>>>> 0.4.22 (22940fb7) initializing
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve:
include:
>>>> sieve_global is not set; it is currently not possible to
include
>>>> `:global' scripts.
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve: file
storage:
>>>> Storage path `/var/vmail/NewUser/home/sieve' not found
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve: file
storage: Using
>>>> Sieve script path: /var/sieve-scripts/NewUser.sieve
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve: file
storage:
>>>> Storage path `/var/sieve-scripts/NewUser.sieve' not found
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve:
storage: No default
>>>> script location configured
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve: User
has no
>>>> personal script
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: sieve: No
scripts to
>>>> execute: reverting to default delivery.
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: INBOX: Mailbox
opened
>>>> because: lib-lda delivery
>>>> Apr 19 10:23:26 lda(NewUser at myDomain): Debug: Mailbox stdin:
Opened mail
>>>> UID=1 because: copying
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
auth USER
>>>> input: NewUser at myDomain uid=992 gid=989
home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Effective
>>>> uid=992, gid=989, home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Quota root:
>>>> name=User quota backend=maildir args>>>> Apr 19
10:23:26 indexer-worker(NewUser at myDomain): Debug: Quota rule:
>>>> root=User quota mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Quota rule:
>>>> root=User quota mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Quota warning:
>>>> bytes=20401094656 (95%) messages=0 reverse=no
command=quota-warning 95
>>>> NewUser at myDomain
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Quota warning:
>>>> bytes=17179869184 (80%) messages=0 reverse=no
command=quota-warning 80
>>>> NewUser at myDomain
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Quota grace:
>>>> root=User quota bytes=2147483648 (10%)
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Namespace :
>>>> type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes,
>>>> subscriptions=yes location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:26
indexer-worker(NewUser at myDomain): Debug: quota:
>>>> quota_over_flag check: quota_over_script unset - skipping
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
INBOX: Mailbox
>>>> opened because: indexing
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Mailbox INBOX:
>>>> Opened mail UID=6 because: prefetch
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Mailbox INBOX:
>>>> Opened mail UID=6 because: access
>>>> Apr 19 10:23:26 indexer-worker(NewUser at myDomain): Debug:
Mailbox INBOX:
>>>> Opened mail UID=6 because: fts indexing
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:50
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:50 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:50 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:50
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:50 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:52
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:52 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:52 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:52
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:52 imap(NewUser): Debug: INBOX: Mailbox opened
because: STATUS
>>>> Apr 19 10:23:52 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:53
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:53 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:53
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:53 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=3
>>>> because: prefetch
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=3
>>>> because: access
>>>> Apr 19 10:23:53 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=3
>>>> because: MIME part
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:54
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:54 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:54
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:54 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=4
>>>> because: prefetch
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=4
>>>> because: access
>>>> Apr 19 10:23:54 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=4
>>>> because: MIME part
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:55
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:55 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:55
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:55 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=5
>>>> because: prefetch
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=5
>>>> because: access
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=5
>>>> because: MIME part
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:55
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:55 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:55
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:55 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=6
>>>> because: prefetch
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=6
>>>> because: access
>>>> Apr 19 10:23:55 imap(NewUser): Debug: Mailbox INBOX: Opened
mail UID=6
>>>> because: MIME part
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:57
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:57 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:57 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:57
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:57 imap(NewUser): Debug: INBOX: Mailbox opened
because: STATUS
>>>> Apr 19 10:23:57 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:23:59
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:23:59 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:23:59 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:23:59
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:23:59 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Loading modules from
directory:
>>>> /usr/local/lib/dovecot
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib10_quota_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib11_imap_quota_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_fts_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib20_virtual_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib21_fts_solr_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib90_stats_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Module loaded:
>>>> /usr/local/lib/dovecot/lib95_imap_stats_plugin.so
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Effective uid=992,
gid=989,
>>>> home=/var/vmail/NewUser/home
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Quota root: name=User
quota
>>>> backend=maildir args>>>> Apr 19 10:24:07
imap(NewUser): Debug: Quota rule: root=User quota
>>>> mailbox=* bytes=21474836480 messages=0
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Quota rule: root=User
quota
>>>> mailbox=Trash bytes=+209715200 messages=0
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Quota warning:
bytes=20401094656
>>>> (95%) messages=0 reverse=no command=quota-warning 95 NewUser
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Quota warning:
bytes=17179869184
>>>> (80%) messages=0 reverse=no command=quota-warning 80 NewUser
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Quota grace: root=User
quota
>>>> bytes=2147483648 (10%)
>>>> Apr 19 10:24:07 imap(NewUser): Debug: Namespace : type=private,
prefix=,
>>>> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
>>>> location=maildir:/var/vmail/NewUser
>>>> Apr 19 10:24:07 imap(NewUser): Debug: maildir++:
>>>> root=/var/vmail/NewUser, index=, indexpvt=, control=,
>>>> inbox=/var/vmail/NewUser, alt>>>> Apr 19 10:24:07
imap(NewUser): Debug: quota: quota_over_flag check:
>>>> quota_over_script unset - skipping
>>>> Apr 19 10:24:07 imap(NewUser): Debug: INBOX: Mailbox opened
because: STATUS
>>>> Apr 19 10:24:07 imap(NewUser): Debug: INBOX: Mailbox opened
because: SELECT
>>>>
>>>>
>
kadafax at gmail.com
2018-May-21 11:11 UTC
Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
Le 21/05/2018 ? 12:38, Aki Tuomi a ?crit?:> can you try turning on pluign { fts_enforced = yes } and repeat your test?Same (wrong) result: 1. Send an email with "too6Ouka" in the body 2. Search against "too6Ouka": # doveadm search -u username mailbox INBOX body too6Ouka --> No result 3. Force re-index: # doveadm fts rescan -u username 4. Search again against "too6Ouka": # doveadm search -u username mailbox INBOX body too6Ouka --> e09cce0283e8695ab760000002deed92 29055 Don't know if relevant, but on a side note, if I send a second message with "too6Ouka" in the body, followed by: # doveadm -v index -u username Inbox --> doveadm(username): Info: INBOX: Cache is already up to date And a search against the pattern immediately return only one result instead of two. # doveconf -n # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.22 (22940fb7) # OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 CentOS Linux release 7.5.1804 (Core)? ext4 # Hostname: mailserver.domain auth_master_user_separator = * auth_socket_path = /var/run/dovecot/auth-userdb auth_username_chars abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* auth_verbose = yes base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot-debug.log default_client_limit = 5120 default_process_limit = 1024 default_vsz_limit = 2 G hostname = domain info_log_path = /var/log/dovecot.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot-error.log login_greeting = Welcome to dovecot on mydomain login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_gid = vmail mail_location = maildir:/var/vmail/%Ln mail_plugins = " quota stats fts fts_solr" mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { ? inbox = yes ? location = maildir:/var/vmail/%Ln ? mailbox Drafts { ??? auto = subscribe ??? special_use = \Drafts ? } ? mailbox Sent { ??? auto = subscribe ??? special_use = \Sent ? } ? mailbox "Sent Messages" { ??? special_use = \Sent ? } ? mailbox Trash { ??? auto = subscribe ??? special_use = \Trash ? } ? prefix ? type = private } passdb { ? args = /usr/local/etc/dovecot/deny-users ? deny = yes ? driver = passwd-file } passdb { ? args = /usr/local/etc/dovecot/master-users ? driver = passwd-file ? master = yes } passdb { ? args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } plugin { ? fts = solr ? fts_autoindex = yes ? fts_enforced = yes ? fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search ? quota = maildir:User quota ? quota_rule = *:storage=20GB ? quota_rule2 = Trash:storage=+200M ? quota_warning = storage=95%% quota-warning 95 %u ? quota_warning2 = storage=80%% quota-warning 80 %u ? sieve = /var/sieve-scripts/%Ln.sieve ? sieve_dir = /var/vmail/%Ln/home/sieve ? stats_command_min_time = 1 mins ? stats_domain_min_time = 12 hours ? stats_ip_min_time = 12 hours ? stats_memory_limit = 16 M ? stats_refresh = 30 secs ? stats_session_min_time = 15 mins ? stats_track_cmds = yes ? stats_user_min_time = 1 hours } postmaster_address = postmaster at domain protocols = imap pop3 sieve sieve service auth-worker { ? user = root } service auth { ? unix_listener /var/spool/postfix/private/auth { ??? mode = 0777 ??? user = vmail ? } } service imap-login { ? inet_listener imaps { ??? port = 993 ??? ssl = yes ? } ? process_min_avail = 10 } service imap { ? process_limit = 2128 } service managesieve-login { ? inet_listener sieve { ??? port = 4190 ? } } service pop3-login { ? inet_listener pop3s { ??? port = 995 ??? ssl = yes ? } } service quota-warning { ? executable = script /usr/local/bin/quota-warning.sh ? user = nobody } service stats { ? fifo_listener stats-mail { ??? mode = 0600 ??? user = vmail ? } } ssl_cert = </etc/letsencrypt/live/mailserver.domain/fullchain.pem ssl_key =? # hidden, use -P to show it userdb { ? args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext ? driver = ldap ? override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home } verbose_proctitle = yes protocol lda { ? mail_plugins = " quota stats fts fts_solr sieve virtual" } protocol imap { ? mail_max_userip_connections = 20 ? mail_plugins = " quota stats fts fts_solr imap_quota virtual imap_stats" } protocol sieve { ? mail_max_userip_connections = 20 }> > Aki > > > On 21.05.2018 13:31, kadafax at gmail.com wrote: >> Le 21/05/2018 ? 11:46, Aki Tuomi a ?crit?: >>> That message is coming from the temporary raw user, not from the actual >>> user. >>> >>> How did you verify that the FTS indexes are not created or updated >>> automatically? >> I send an email containing a specific strings ('thahB8ea' in that case) >> and search agains it: >> # doveadm search -u username mailbox INBOX body thahB8ea >> --> No result >> (same with integrated search on roundcube) >> >> Then I force a rescan, followed by the same search: >> # doveadm? fts rescan -u username >> # doveadm search -u username mailbox INBOX body thahB8ea >> (Long waiting time) >> --> e09cce0283e8695ab760000002deed92 29048 >> >> So I conclude that autoindexing is not working. >> >> Thanks, >> kfx >>
kfx
2018-May-24 12:22 UTC
Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
After seeing that dovecot 2.2.36 was out and that:> - fts: Fixes to indexing mails via virtual mailboxes. > - fts: If mails contained NUL characters, the text around it wasn't > indexed.I've upgraded but the problem still persist :( On 21/05/2018 13:11, kadafax at gmail.com wrote:> Le 21/05/2018 ? 12:38, Aki Tuomi a ?crit?: >> can you try turning on pluign { fts_enforced = yes } and repeat your test? > > Same (wrong) result: > 1. Send an email with "too6Ouka" in the body > > 2. Search against "too6Ouka": > # doveadm search -u username mailbox INBOX body too6Ouka > --> No result > > 3. Force re-index: > # doveadm fts rescan -u username > > 4. Search again against "too6Ouka": > # doveadm search -u username mailbox INBOX body too6Ouka > --> e09cce0283e8695ab760000002deed92 29055 > > Don't know if relevant, but on a side note, if I send a second message > with "too6Ouka" in the body, followed by: > # doveadm -v index -u username Inbox > --> doveadm(username): Info: INBOX: Cache is already up to date > And a search against the pattern immediately return only one result > instead of two. > > # doveconf -n > # 2.2.34 (874deae): /usr/local/etc/dovecot/dovecot.conf > # Pigeonhole version 0.4.22 (22940fb7) > # OS: Linux 3.10.0-862.2.3.el7.x86_64 x86_64 CentOS Linux release > 7.5.1804 (Core)? ext4 > # Hostname: mailserver.domain > auth_master_user_separator = * > auth_socket_path = /var/run/dovecot/auth-userdb > auth_username_chars > abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@* > auth_verbose = yes > base_dir = /var/run/dovecot/ > debug_log_path = /var/log/dovecot-debug.log > default_client_limit = 5120 > default_process_limit = 1024 > default_vsz_limit = 2 G > hostname = domain > info_log_path = /var/log/dovecot.log > lda_mailbox_autocreate = yes > lda_mailbox_autosubscribe = yes > listen = * > log_path = /var/log/dovecot-error.log > login_greeting = Welcome to dovecot on mydomain > login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c > mail_gid = vmail > mail_location = maildir:/var/vmail/%Ln > mail_plugins = " quota stats fts fts_solr" > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope > encoded-character vacation subaddress comparator-i;ascii-numeric > relational regex imap4flags copy include variables body enotify > environment mailbox date index ihave duplicate mime foreverypart extracttext > namespace { > ? inbox = yes > ? location = maildir:/var/vmail/%Ln > ? mailbox Drafts { > ??? auto = subscribe > ??? special_use = \Drafts > ? } > ? mailbox Sent { > ??? auto = subscribe > ??? special_use = \Sent > ? } > ? mailbox "Sent Messages" { > ??? special_use = \Sent > ? } > ? mailbox Trash { > ??? auto = subscribe > ??? special_use = \Trash > ? } > ? prefix > ? type = private > } > passdb { > ? args = /usr/local/etc/dovecot/deny-users > ? deny = yes > ? driver = passwd-file > } > passdb { > ? args = /usr/local/etc/dovecot/master-users > ? driver = passwd-file > ? master = yes > } > passdb { > ? args = /usr/local/etc/dovecot/dovecot-ldap.conf.ext > ? driver = ldap > } > plugin { > ? fts = solr > ? fts_autoindex = yes > ? fts_enforced = yes > ? fts_solr = url=http://localhost:8983/solr/dovecot/ break-imap-search > ? quota = maildir:User quota > ? quota_rule = *:storage=20GB > ? quota_rule2 = Trash:storage=+200M > ? quota_warning = storage=95%% quota-warning 95 %u > ? quota_warning2 = storage=80%% quota-warning 80 %u > ? sieve = /var/sieve-scripts/%Ln.sieve > ? sieve_dir = /var/vmail/%Ln/home/sieve > ? stats_command_min_time = 1 mins > ? stats_domain_min_time = 12 hours > ? stats_ip_min_time = 12 hours > ? stats_memory_limit = 16 M > ? stats_refresh = 30 secs > ? stats_session_min_time = 15 mins > ? stats_track_cmds = yes > ? stats_user_min_time = 1 hours > } > postmaster_address = postmaster at domain > protocols = imap pop3 sieve sieve > service auth-worker { > ? user = root > } > service auth { > ? unix_listener /var/spool/postfix/private/auth { > ??? mode = 0777 > ??? user = vmail > ? } > } > service imap-login { > ? inet_listener imaps { > ??? port = 993 > ??? ssl = yes > ? } > ? process_min_avail = 10 > } > service imap { > ? process_limit = 2128 > } > service managesieve-login { > ? inet_listener sieve { > ??? port = 4190 > ? } > } > service pop3-login { > ? inet_listener pop3s { > ??? port = 995 > ??? ssl = yes > ? } > } > service quota-warning { > ? executable = script /usr/local/bin/quota-warning.sh > ? user = nobody > } > service stats { > ? fifo_listener stats-mail { > ??? mode = 0600 > ??? user = vmail > ? } > } > ssl_cert = </etc/letsencrypt/live/mailserver.domain/fullchain.pem > ssl_key =? # hidden, use -P to show it > userdb { > ? args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf.ext > ? driver = ldap > ? override_fields = uid=vmail gid=vmail home=/var/vmail/%Ln/home > } > verbose_proctitle = yes > protocol lda { > ? mail_plugins = " quota stats fts fts_solr sieve virtual" > } > protocol imap { > ? mail_max_userip_connections = 20 > ? mail_plugins = " quota stats fts fts_solr imap_quota virtual imap_stats" > } > protocol sieve { > ? mail_max_userip_connections = 20 > } > > > >> >> Aki >> >> >> On 21.05.2018 13:31, kadafax at gmail.com wrote: >>> Le 21/05/2018 ? 11:46, Aki Tuomi a ?crit?: >>>> That message is coming from the temporary raw user, not from the actual >>>> user. >>>> >>>> How did you verify that the FTS indexes are not created or updated >>>> automatically? >>> I send an email containing a specific strings ('thahB8ea' in that case) >>> and search agains it: >>> # doveadm search -u username mailbox INBOX body thahB8ea >>> --> No result >>> (same with integrated search on roundcube) >>> >>> Then I force a rescan, followed by the same search: >>> # doveadm? fts rescan -u username >>> # doveadm search -u username mailbox INBOX body thahB8ea >>> (Long waiting time) >>> --> e09cce0283e8695ab760000002deed92 29048 >>> >>> So I conclude that autoindexing is not working. >>> >>> Thanks, >>> kfx >>>
Timo Sirainen
2018-May-24 13:33 UTC
Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
On 21 May 2018, at 14.11, kadafax at gmail.com wrote:> > Le 21/05/2018 ? 12:38, Aki Tuomi a ?crit : >> can you try turning on pluign { fts_enforced = yes } and repeat your test? > > Same (wrong) result: > 1. Send an email with "too6Ouka" in the body > > 2. Search against "too6Ouka": > # doveadm search -u username mailbox INBOX body too6Ouka > --> No result > > 3. Force re-index: > # doveadm fts rescan -u username > > 4. Search again against "too6Ouka": > # doveadm search -u username mailbox INBOX body too6Ouka > --> e09cce0283e8695ab760000002deed92 29055 > > Don't know if relevant, but on a side note, if I send a second message > with "too6Ouka" in the body, followed by: > # doveadm -v index -u username Inbox > --> doveadm(username): Info: INBOX: Cache is already up to date > And a search against the pattern immediately return only one result > instead of two.So it looks like after "doveadm fts rescan" you can do initial indexing successfully. Afterwards the index isn't updated at all, unless you rebuild it entirely. fts_autoindex=yes appears to work as expected, because after fts rescan you did a search which automatically updated the index. So this doesn't have anything to do with fts_autoindex, just that your index updates don't seem to work at all. What does it show in the header if you do: doveadm dump /var/vmail/username/dovecot.index Especially what is in the "fts" header vs. "next uid" header? Does the UID in "fts" header keep changing every time you save a new mail? I suppose it will. You could also monitor (e.g. tcpdump/wireshark) the network traffic between Dovecot <-> Solr what happens when a new mail arrives. I suspect Dovecot sends it to Solr, which for whatever reason just ignores the update.
Possibly Parallel Threads
- Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
- Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
- Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
- Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"
- Bug: Dovecot index loosing sync with FTS despite "fts_autoindex = yes"