I am running ISPConfig 3 on Debian and have managed to install the
dovecot-solr and dovecot-fts plugins.
I have solr running undet tomcat at http://localhost:8880 but the
indexing is not working.
I am using Dovecot 2.17
I do not understand namespaces and why fts_solr needs them, I just want
to index the entire users Maildir.
I am trying to index a users mailbox but am getting the following error:
(changed user name)
doveadm fts rescan -u user at user.com inbox
doveadm(user at user.com): Error: Namespace prefix not found: inbox
running that in debig mode gives:
doveadm(root): Debug: Loading modules from directory:
/usr/lib/dovecot/modules
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib20_fts_plugin.so
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/lib21_fts_solr_plugin.so
doveadm(root): Debug: Loading modules from directory:
/usr/lib/dovecot/modules/doveadm
doveadm(root): Debug: Skipping module doveadm_acl_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined
symbol: acl_user_module (this is usually intentional, so just ignore
this message)
doveadm(root): Debug: Skipping module doveadm_expire_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so:
undefined symbol: expire_set_deinit (this is usually intentional, so
just ignore this message)
doveadm(root): Debug: Skipping module doveadm_quota_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so:
undefined symbol: quota_user_module (this is usually intentional, so
just ignore this message)
doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because
dlopen() failed:
/usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined
symbol: i_stream_create_deflate (this is usually intentional, so just
ignore this message)
doveadm(root): Debug: Module loaded:
/usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so
doveadm(user at user.com): Debug: Added userdb setting:
mail=maildir:/var/vmail/user.com/user/Maildir
doveadm(user at user.com): Debug: Added userdb setting:
plugin/quota_rule=*:storage=0B
doveadm(user at user.com): Debug: Added userdb setting:
plugin/sieve=/var/vmail/user.com/user/.sieve
doveadm(user at user.com): Debug: Effective uid=5000, gid=5000,
home=/var/vmail/user.com/user
doveadm(user at user.com): Debug: Namespace inbox: type=private, prefix=,
sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
location=maildir:/var/vmail/user.com/user/Maildir
doveadm(user at user.com): Debug: maildir++:
root=/var/vmail/user.com/user/Maildir, index=, control=,
inbox=/var/vmail/user.com/user/Maildir, altdoveadm(user at user.com): Debug:
fts: No fts setting - plugin disabled
doveadm(user at user.com): Error: Namespace prefix not found: inbox
in conf.d/10-mail.conf I have this namespace defined:
namespace inbox {
# Namespace type: private, shared or public
#type = private
# Hierarchy separator to use. You should use the same separator for
all
# namespaces or some clients get confused. '/' is usually a good one.
# The default however depends on the underlying mail storage format.
#separator
# Prefix required to access this namespace. This needs to be different
for
# all namespaces. For example "Public/".
#prefix
# Physical location of the mailbox. This is in same format as
# mail_location, which is also the default for it.
#location
# There can be only one INBOX, and this setting defines which
namespace
# has it.
inbox = yes
# If namespace is hidden, it's not advertised to clients via NAMESPACE
# extension. You'll most likely also want to set list=no. This is
mostly
# useful when converting from another server with different namespaces
which
# you want to deprecate but still keep working. For example you can
create
# hidden namespaces with prefixes "~/mail/", "~%u/mail/"
and "mail/".
#hidden = no
protocol imap {
plugin {
fts = solr
fts_solr = break-imap-search url=http://localhost:8880/solr/
}
}
protocol pop3 {
plugin {
fts = solr
fts_solr = break-imap-search url=http://localhost:8880/solr/
}
# Show the mailboxes under this namespace with LIST command. This
makes the
# namespace visible for clients that don't support NAMESPACE
extension.
# "children" value lists child mailboxes, but hides the namespace
prefix.
#list = yes
# Namespace handles its own subscriptions. If set to "no", the
parent
# namespace handles them (empty prefix should always have this as
"yes")
#subscriptions = yes
}
I had to put this in dovecot.conf in order for the plugin to be enabled:
mail_plugins = fts fts_solr
solr is reachable at localhost:8880/solr and appears to be working.
Please help, any suggestions are welcome
Thanks
> On June 23, 2016 at 9:06 PM Jeff Gamsby <jeffgamsby at merlock.com> wrote: > > > I am running ISPConfig 3 on Debian and have managed to install the > dovecot-solr and dovecot-fts plugins. > > I have solr running undet tomcat at http://localhost:8880 but the > indexing is not working. > > I am using Dovecot 2.17 > > I do not understand namespaces and why fts_solr needs them, I just want > to index the entire users Maildir. > > I am trying to index a users mailbox but am getting the following error: > > (changed user name) > doveadm fts rescan -u user at user.com inbox > doveadm(user at user.com): Error: Namespace prefix not found: inbox > > running that in debig mode gives: > > doveadm(root): Debug: Loading modules from directory: > /usr/lib/dovecot/modules > doveadm(root): Debug: Module loaded: > /usr/lib/dovecot/modules/lib20_fts_plugin.so > doveadm(root): Debug: Module loaded: > /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so > doveadm(root): Debug: Loading modules from directory: > /usr/lib/dovecot/modules/doveadm > doveadm(root): Debug: Skipping module doveadm_acl_plugin, because > dlopen() failed: > /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined > symbol: acl_user_module (this is usually intentional, so just ignore > this message) > doveadm(root): Debug: Skipping module doveadm_expire_plugin, because > dlopen() failed: > /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: > undefined symbol: expire_set_deinit (this is usually intentional, so > just ignore this message) > doveadm(root): Debug: Skipping module doveadm_quota_plugin, because > dlopen() failed: > /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: > undefined symbol: quota_user_module (this is usually intentional, so > just ignore this message) > doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because > dlopen() failed: > /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined > symbol: i_stream_create_deflate (this is usually intentional, so just > ignore this message) > doveadm(root): Debug: Module loaded: > /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so > doveadm(user at user.com): Debug: Added userdb setting: > mail=maildir:/var/vmail/user.com/user/Maildir > doveadm(user at user.com): Debug: Added userdb setting: > plugin/quota_rule=*:storage=0B > doveadm(user at user.com): Debug: Added userdb setting: > plugin/sieve=/var/vmail/user.com/user/.sieve > doveadm(user at user.com): Debug: Effective uid=5000, gid=5000, > home=/var/vmail/user.com/user > doveadm(user at user.com): Debug: Namespace inbox: type=private, prefix=, > sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes > location=maildir:/var/vmail/user.com/user/Maildir > doveadm(user at user.com): Debug: maildir++: > root=/var/vmail/user.com/user/Maildir, index=, control=, > inbox=/var/vmail/user.com/user/Maildir, alt> doveadm(user at user.com): Debug: fts: No fts setting - plugin disabled > doveadm(user at user.com): Error: Namespace prefix not found: inbox > > in conf.d/10-mail.conf I have this namespace defined: > > namespace inbox { > # Namespace type: private, shared or public > #type = private > > # Hierarchy separator to use. You should use the same separator for > all > # namespaces or some clients get confused. '/' is usually a good one. > # The default however depends on the underlying mail storage format. > #separator > > # Prefix required to access this namespace. This needs to be different > for > # all namespaces. For example "Public/". > #prefix > > # Physical location of the mailbox. This is in same format as > # mail_location, which is also the default for it. > #location > > # There can be only one INBOX, and this setting defines which > namespace > # has it. > inbox = yes > > # If namespace is hidden, it's not advertised to clients via NAMESPACE > # extension. You'll most likely also want to set list=no. This is > mostly > # useful when converting from another server with different namespaces > which > # you want to deprecate but still keep working. For example you can > create > # hidden namespaces with prefixes "~/mail/", "~%u/mail/" and "mail/". > #hidden = no > protocol imap { > plugin { > fts = solr > fts_solr = break-imap-search url=http://localhost:8880/solr/ > } > } > protocol pop3 { > plugin { > fts = solr > fts_solr = break-imap-search url=http://localhost:8880/solr/ > } > > # Show the mailboxes under this namespace with LIST command. This > makes the > # namespace visible for clients that don't support NAMESPACE > extension. > # "children" value lists child mailboxes, but hides the namespace > prefix. > #list = yes > > # Namespace handles its own subscriptions. If set to "no", the parent > # namespace handles them (empty prefix should always have this as > "yes") > #subscriptions = yes > } > > > I had to put this in dovecot.conf in order for the plugin to be enabled: > > mail_plugins = fts fts_solr > > solr is reachable at localhost:8880/solr and appears to be working. > > Please help, any suggestions are welcome > > ThanksCan you please send doveconf -n? --- Aki Tuomi
> On Jun 24, 2016, at 2:06 AM, Jeff Gamsby <jeffgamsby at merlock.com> wrote: > > doveadm(user at user.com): Debug: fts: No fts setting - plugin disabledIsn't it very clear here? ---- Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ Time zone: GMT+8 (China/Beijing).
> On Jun 24, 2016, at 2:06 AM, Jeff Gamsby <jeffgamsby at merlock.com> wrote: > > protocol imap { > plugin { > fts = solr > fts_solr = break-imap-search url=http://localhost:8880/solr/ > } > } > protocol pop3 { > plugin { > fts = solr > fts_solr = break-imap-search url=http://localhost:8880/solr/ > }Don't place `plugin {}` setting in other config block. It should be: protocol imap {...} protocol pop3 {...} plugin { ... } ---- Zhang Huangbin, founder of iRedMail project: http://www.iredmail.org/ Time zone: GMT+8 (China/Beijing).
On 2016-06-23 11:24, aki.tuomi at dovecot.fi wrote:>> On June 23, 2016 at 9:06 PM Jeff Gamsby <jeffgamsby at merlock.com> >> wrote: >> >> >> I am running ISPConfig 3 on Debian and have managed to install the >> dovecot-solr and dovecot-fts plugins. >> >> I have solr running undet tomcat at http://localhost:8880 but the >> indexing is not working. >> >> I am using Dovecot 2.17 >> >> I do not understand namespaces and why fts_solr needs them, I just >> want >> to index the entire users Maildir. >> >> I am trying to index a users mailbox but am getting the following >> error: >> >> (changed user name) >> doveadm fts rescan -u user at user.com inbox >> doveadm(user at user.com): Error: Namespace prefix not found: inbox >> >> running that in debig mode gives: >> >> doveadm(root): Debug: Loading modules from directory: >> /usr/lib/dovecot/modules >> doveadm(root): Debug: Module loaded: >> /usr/lib/dovecot/modules/lib20_fts_plugin.so >> doveadm(root): Debug: Module loaded: >> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so >> doveadm(root): Debug: Loading modules from directory: >> /usr/lib/dovecot/modules/doveadm >> doveadm(root): Debug: Skipping module doveadm_acl_plugin, because >> dlopen() failed: >> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: >> undefined >> symbol: acl_user_module (this is usually intentional, so just ignore >> this message) >> doveadm(root): Debug: Skipping module doveadm_expire_plugin, because >> dlopen() failed: >> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: >> undefined symbol: expire_set_deinit (this is usually intentional, so >> just ignore this message) >> doveadm(root): Debug: Skipping module doveadm_quota_plugin, because >> dlopen() failed: >> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so: >> undefined symbol: quota_user_module (this is usually intentional, so >> just ignore this message) >> doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because >> dlopen() failed: >> /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: >> undefined >> symbol: i_stream_create_deflate (this is usually intentional, so just >> ignore this message) >> doveadm(root): Debug: Module loaded: >> /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so >> doveadm(user at user.com): Debug: Added userdb setting: >> mail=maildir:/var/vmail/user.com/user/Maildir >> doveadm(user at user.com): Debug: Added userdb setting: >> plugin/quota_rule=*:storage=0B >> doveadm(user at user.com): Debug: Added userdb setting: >> plugin/sieve=/var/vmail/user.com/user/.sieve >> doveadm(user at user.com): Debug: Effective uid=5000, gid=5000, >> home=/var/vmail/user.com/user >> doveadm(user at user.com): Debug: Namespace inbox: type=private, prefix=, >> sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes >> location=maildir:/var/vmail/user.com/user/Maildir >> doveadm(user at user.com): Debug: maildir++: >> root=/var/vmail/user.com/user/Maildir, index=, control=, >> inbox=/var/vmail/user.com/user/Maildir, alt>> doveadm(user at user.com): Debug: fts: No fts setting - plugin disabled >> doveadm(user at user.com): Error: Namespace prefix not found: inbox >> >> in conf.d/10-mail.conf I have this namespace defined: >> >> namespace inbox { >> # Namespace type: private, shared or public >> #type = private >> >> # Hierarchy separator to use. You should use the same separator for >> all >> # namespaces or some clients get confused. '/' is usually a good >> one. >> # The default however depends on the underlying mail storage >> format. >> #separator >> >> # Prefix required to access this namespace. This needs to be >> different >> for >> # all namespaces. For example "Public/". >> #prefix >> >> # Physical location of the mailbox. This is in same format as >> # mail_location, which is also the default for it. >> #location >> >> # There can be only one INBOX, and this setting defines which >> namespace >> # has it. >> inbox = yes >> >> # If namespace is hidden, it's not advertised to clients via >> NAMESPACE >> # extension. You'll most likely also want to set list=no. This is >> mostly >> # useful when converting from another server with different >> namespaces >> which >> # you want to deprecate but still keep working. For example you can >> create >> # hidden namespaces with prefixes "~/mail/", "~%u/mail/" and >> "mail/". >> #hidden = no >> protocol imap { >> plugin { >> fts = solr >> fts_solr = break-imap-search url=http://localhost:8880/solr/ >> } >> } >> protocol pop3 { >> plugin { >> fts = solr >> fts_solr = break-imap-search url=http://localhost:8880/solr/ >> } >> >> # Show the mailboxes under this namespace with LIST command. This >> makes the >> # namespace visible for clients that don't support NAMESPACE >> extension. >> # "children" value lists child mailboxes, but hides the namespace >> prefix. >> #list = yes >> >> # Namespace handles its own subscriptions. If set to "no", the >> parent >> # namespace handles them (empty prefix should always have this as >> "yes") >> #subscriptions = yes >> } >> >> >> I had to put this in dovecot.conf in order for the plugin to be >> enabled: >> >> mail_plugins = fts fts_solr >> >> solr is reachable at localhost:8880/solr and appears to be working. >> >> Please help, any suggestions are welcome >> >> Thanks > > Can you please send doveconf -n? > --- > Aki TuomiI managed to get fts_slor working and now I can index mailboxes, but I am getting a solr error and cannot use the indexes that were created I get: Error: fts_solr: Lookup failed: Internal Server Error dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1 auth_mechanisms = plain login default_vsz_limit = 2 G disable_plaintext_auth = no lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = *,[::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_plugins = fts fts_solr mail_privileged_group = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location prefix } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { fts = solr fts_autoindex = yes fts_solr = break-imap-search url=http://localhost:8880/solr/ quota = dict:user::file:/var/vmail/%d/%n/.quotausage sieve = ~/.dovecot.sieve sieve_after = /etc/sieve/after sieve_before = /etc/sieve/before sieve_default = /var/vmail/sieve/default.sieve sieve_dir = ~/sieve } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = root } service imap-login { client_limit = 1000 process_limit = 500 } service imap { vsz_limit = 2 G } ssl_cert = </etc/postfix/smtpd.cert ssl_key = </etc/postfix/smtpd.key userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { auth_socket_path = /var/run/dovecot/auth-master info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log mail_plugins = fts fts_solr autocreate sieve quota postmaster_address = postmaster at user.com } And this shows that indexes are correct: doveadm -Dv index -u jeff at user.com testing doveadm(jeff at user.com): Info: testing: Cache is already up to date using solr-schema.xml from dovecot-2.2.13 sources in /etc/solr/conf/schema.xml dovecot-sql.conf # http://wiki.dovecot.org/AuthDatabase/SQL # # CREATE TABLE users ( # userid VARCHAR(128) NOT NULL, # password VARCHAR(64) NOT NULL, # home VARCHAR(255) NOT NULL, # uid INTEGER NOT NULL, # gid INTEGER NOT NULL, # active CHAR(1) DEFAULT 'Y' NOT NULL # ); driver = mysql connect = host=localhost dbname=dbispconfig user=ispconfig password=xxxxxxxxxxxxxxxxxxxxx default_pass_scheme = CRYPT password_query = SELECT password FROM mail_user WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' AND server_id = '1' user_query = SELECT email as user, maildir as home, CONCAT('maildir:', maildir, '/Maildir') as mail, uid, gid, CONCAT('*:storage=', quota, 'B') AS quota_rule, CONCAT(maildir, '/.sieve') as sieve FROM mail_user WHERE (login = '%u' OR email = '%u') AND `disable%Ls` = 'n' AND server_id = '1' # The iterate_query is required for the doveadm command only and works only on dovecot 2 servers. # Do not enable it on Dovecot 1.x servers iterate_query = SELECT email as user FROM mail_user
Possibly Parallel Threads
- fts solr ignores new mailboxes until fts rescan
- Fwd: Problem with solr working, but not indexing
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed)
- v2.3.11.3 solr plugin search via MUA fails to match accented ascii characters; cmd line exec of `doveadm fts lookup` PANICs (assertion failed) [proposed patch]
- fts_solr and connection via https://