I am testing the solr FTS following the guide here:
http://things.m31.ch/?p=379
Now I am having problem when I try to test:
# doveadm -v? index -u fail2ban at mydomain.com Inbox
doveadm(fail2ban at mydomain.com): Error: fts_solr: Invalid XML input at
1:0: not well-formed (invalid token) (near: {
? "responseHeader":{
??? "status":0,
??? "QTime":0,
??? "params":{
????? "q":"box:8864fa1d51ea1d5a7b12000096a1aaf8 AND user:fa)
doveadm(fail2ban at mydomain.com): Error: Mailbox INBOX: Status lookup
failed: Internal error occurred. Refer to server log for more
information. [2017-11-30 13:05:24]
My system is CentOS 7, Dovecot 3.2.4 (installed from source with solr
support), Java OpenJDK-1.8.0, Solr v7.1.0.
Could some one help me on this please?
Here is detailed debug out put:
[root at mail ~]# doveadm -vD? index -u fail2ban at mydomain.com Inbox
Debug: Loading modules from directory: /usr/lib64/dovecot
Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib20_fts_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/lib21_fts_solr_plugin.so
Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm
Debug: Skipping module doveadm_acl_plugin, because dlopen() failed:
/usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined
symbol: acl_user_module (this is usually intentional, so just ignore
this message)
Debug: Skipping module doveadm_expire_plugin, because dlopen() failed:
/usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined
symbol: expire_set_deinit (this is usually intentional, so just ignore
this message)
Debug: Module loaded:
/usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so
Debug: Module loaded:
/usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so
Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so
Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen()
failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so:
undefined symbol: mail_crypt_box_get_pvt_digests (this is usually
intentional, so just ignore this message)
doveadm(fail2ban at mydomain.com): Debug: Added userdb setting:
mail=maildir:/home/vmail/mydomain.com/fail2ban/
doveadm(fail2ban at mydomain.com): Debug: Added userdb setting:
plugin/quota_rule=*:bytes=0
doveadm(fail2ban at mydomain.com): Debug: Effective uid=5000, gid=5000,
home=/home/vmail/mydomain.com/fail2ban/
doveadm(fail2ban at mydomain.com): Debug: Quota root: name=User quota
backend=maildir argsdoveadm(fail2ban at mydomain.com): Debug: Quota rule:
root=User quota
mailbox=* bytes=0 messages=0
doveadm(fail2ban at mydomain.com): Debug: Quota rule: root=User quota
mailbox=Trash bytes=+104857600 messages=0
doveadm(fail2ban at mydomain.com): Debug: Quota warning: bytes=0 (90%)
messages=0 reverse=no command=quota-warning 90 fail2ban at mydomain.com
mydomain.com
doveadm(fail2ban at mydomain.com): Debug: Quota warning: bytes=0 (80%)
messages=0 reverse=no command=quota-warning 80 fail2ban at mydomain.com
mydomain.com
doveadm(fail2ban at mydomain.com): Debug: Quota grace: root=User quota
bytes=0 (10%)
doveadm(fail2ban at mydomain.com): Debug: Namespace inbox: type=private,
prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes
location=maildir:/home/vmail/mydomain.com/fail2ban/
doveadm(fail2ban at mydomain.com): Debug: maildir++:
root=/home/vmail/mydomain.com/fail2ban, index=, indexpvt=, control=,
inbox=/home/vmail/mydomain.com/fail2ban, altdoveadm(fail2ban at mydomain.com):
Debug: quota: quota_over_flag check:
quota_over_script unset - skipping
doveadm(fail2ban at mydomain.com): Debug: INBOX: Mailbox opened because: index
doveadm(fail2ban at mydomain.com): Debug: http-client: host 127.0.0.1: Host
created
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Peer created
doveadm(fail2ban at mydomain.com): Debug: http-client: queue
http://127.0.0.1:8983: Setting up connection to 127.0.0.1:8983 (1
requests pending)
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Linked queue http://127.0.0.1:8983 (1 queues linked)
doveadm(fail2ban at mydomain.com): Debug: http-client: queue
http://127.0.0.1:8983: Started new connection to 127.0.0.1:8983
doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]:
Submitted
doveadm(fail2ban at mydomain.com): Debug: http-client: Waiting for 1
requests to finish
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Creating 1 new connections to handle requests (already 0 usable,
connecting to 0, closing 0)
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Making new connection 1 of 1
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: HTTP connection created (1 parallel connections exist)
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Connected
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Ready for requests
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Successfully connected (connections=1)
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Using 1 idle connections to handle 1 requests (1 total connections ready)
doveadm(fail2ban at mydomain.com): Debug: http-client: queue
http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed request
[Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Claimed request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]
doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]:
Sent header
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
No more requests to service for this peer (1 connections exist)
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Got 200 response for request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]
(took 3 ms + 2 ms in queue)
doveadm(fail2ban at mydomain.com): Error: fts_solr: Invalid XML input at
1:0: not well-formed (invalid token) (near: {
? "responseHeader":{
??? "status":0,
??? "QTime":1,
??? "params":{
????? "q":"box:8864fa1d51ea1d5a7b12000096a1aaf8 AND user:fa)
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Response payload stream destroyed (0 ms after initial response)
doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]:
Finished
doveadm(fail2ban at mydomain.com): Debug: http-client: queue
http://127.0.0.1:8983: Dropping request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]
doveadm(fail2ban at mydomain.com): Debug: http-client: host 127.0.0.1: Host
is idle (timeout = 100 msecs)
doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET
http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban
at mydomain.com]:
Free (requests left=1)
doveadm(fail2ban at mydomain.com): Debug: http-client: All requests finished
doveadm(fail2ban at mydomain.com): Error: Mailbox INBOX: Status lookup
failed: Internal error occurred. Refer to server log for more
information. [2017-11-30 13:13:57]
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Peer close
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Peer disconnect
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Peer closed
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Connection disconnect
doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983
[0]: Connection destroy
doveadm(fail2ban at mydomain.com): Debug: http-client: peer 127.0.0.1:8983:
Peer destroy
doveadm(fail2ban at mydomain.com): Debug: http-client: host 127.0.0.1: Host
destroy
doveadm(fail2ban at mydomain.com): Debug: http-client: queue
http://127.0.0.1:8983: Destroy
Am 30. November 2017 22:14:59 MEZ schrieb Gao <gao at pztop.com>:>I am testing the solr FTS following the guide here: >http://things.m31.ch/?p=379 > >Now I am having problem when I try to test: > ># doveadm -v? index -u fail2ban at mydomain.com Inbox >doveadm(fail2ban at mydomain.com): Error: fts_solr: Invalid XML input at >1:0: not well-formed (invalid token) (near: { > ? "responseHeader":{ > ??? "status":0, > ??? "QTime":0, > ??? "params":{ > ????? "q":"box:8864fa1d51ea1d5a7b12000096a1aaf8 AND user:fa) >doveadm(fail2ban at mydomain.com): Error: Mailbox INBOX: Status lookup >failed: Internal error occurred. Refer to server log for more >information. [2017-11-30 13:05:24] > >My system is CentOS 7, Dovecot 3.2.4 (installed from source with solr >support), Java OpenJDK-1.8.0, Solr v7.1.0.The link you posted is for solr 6.5.0 and you are using solr 7.1.0 and solr needs to be configured to accept XML queries instead of json as the default starting with version 7. Search the list archives there have been postings regarding solr 7 and dovecot in the past.>Could some one help me on this please? > >Here is detailed debug out put: > >[root at mail ~]# doveadm -vD? index -u fail2ban at mydomain.com Inbox >Debug: Loading modules from directory: /usr/lib64/dovecot >Debug: Module loaded: /usr/lib64/dovecot/lib10_quota_plugin.so >Debug: Module loaded: /usr/lib64/dovecot/lib20_fts_plugin.so >Debug: Module loaded: /usr/lib64/dovecot/lib21_fts_solr_plugin.so >Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm >Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: >/usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so: undefined >symbol: acl_user_module (this is usually intentional, so just ignore >this message) >Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: >/usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined >symbol: expire_set_deinit (this is usually intentional, so just ignore >this message) >Debug: Module loaded: >/usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so >Debug: Module loaded: >/usr/lib64/dovecot/doveadm/lib10_doveadm_sieve_plugin.so >Debug: Module loaded: >/usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so >Debug: Skipping module doveadm_mail_crypt_plugin, because dlopen() >failed: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so: >undefined symbol: mail_crypt_box_get_pvt_digests (this is usually >intentional, so just ignore this message) >doveadm(fail2ban at mydomain.com): Debug: Added userdb setting: >mail=maildir:/home/vmail/mydomain.com/fail2ban/ >doveadm(fail2ban at mydomain.com): Debug: Added userdb setting: >plugin/quota_rule=*:bytes=0 >doveadm(fail2ban at mydomain.com): Debug: Effective uid=5000, gid=5000, >home=/home/vmail/mydomain.com/fail2ban/ >doveadm(fail2ban at mydomain.com): Debug: Quota root: name=User quota >backend=maildir args>doveadm(fail2ban at mydomain.com): Debug: Quota rule: root=User quota >mailbox=* bytes=0 messages=0 >doveadm(fail2ban at mydomain.com): Debug: Quota rule: root=User quota >mailbox=Trash bytes=+104857600 messages=0 >doveadm(fail2ban at mydomain.com): Debug: Quota warning: bytes=0 (90%) >messages=0 reverse=no command=quota-warning 90 fail2ban at mydomain.com >mydomain.com >doveadm(fail2ban at mydomain.com): Debug: Quota warning: bytes=0 (80%) >messages=0 reverse=no command=quota-warning 80 fail2ban at mydomain.com >mydomain.com >doveadm(fail2ban at mydomain.com): Debug: Quota grace: root=User quota >bytes=0 (10%) >doveadm(fail2ban at mydomain.com): Debug: Namespace inbox: type=private, >prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes >location=maildir:/home/vmail/mydomain.com/fail2ban/ >doveadm(fail2ban at mydomain.com): Debug: maildir++: >root=/home/vmail/mydomain.com/fail2ban, index=, indexpvt=, control=, >inbox=/home/vmail/mydomain.com/fail2ban, alt>doveadm(fail2ban at mydomain.com): Debug: quota: quota_over_flag check: >quota_over_script unset - skipping >doveadm(fail2ban at mydomain.com): Debug: INBOX: Mailbox opened because: >index >doveadm(fail2ban at mydomain.com): Debug: http-client: host 127.0.0.1: >Host >created >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Peer created >doveadm(fail2ban at mydomain.com): Debug: http-client: queue >http://127.0.0.1:8983: Setting up connection to 127.0.0.1:8983 (1 >requests pending) >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Linked queue http://127.0.0.1:8983 (1 queues linked) >doveadm(fail2ban at mydomain.com): Debug: http-client: queue >http://127.0.0.1:8983: Started new connection to 127.0.0.1:8983 >doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com]: > >Submitted >doveadm(fail2ban at mydomain.com): Debug: http-client: Waiting for 1 >requests to finish >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Creating 1 new connections to handle requests (already 0 usable, >connecting to 0, closing 0) >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Making new connection 1 of 1 >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: HTTP connection created (1 parallel connections exist) >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Connected >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Ready for requests >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Successfully connected (connections=1) >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Using 1 idle connections to handle 1 requests (1 total connections >ready) >doveadm(fail2ban at mydomain.com): Debug: http-client: queue >http://127.0.0.1:8983: Connection to peer 127.0.0.1:8983 claimed >request >[Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com] > > >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Claimed request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com] >doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com]: > >Sent header >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >No more requests to service for this peer (1 connections exist) >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Got 200 response for request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com] > >(took 3 ms + 2 ms in queue) >doveadm(fail2ban at mydomain.com): Error: fts_solr: Invalid XML input at >1:0: not well-formed (invalid token) (near: { > ? "responseHeader":{ > ??? "status":0, > ??? "QTime":1, > ??? "params":{ > ????? "q":"box:8864fa1d51ea1d5a7b12000096a1aaf8 AND user:fa) >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Response payload stream destroyed (0 ms after initial response) >doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com]: > >Finished >doveadm(fail2ban at mydomain.com): Debug: http-client: queue >http://127.0.0.1:8983: Dropping request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com] >doveadm(fail2ban at mydomain.com): Debug: http-client: host 127.0.0.1: >Host >is idle (timeout = 100 msecs) >doveadm(fail2ban at mydomain.com): Debug: http-client: request [Req1: GET >http://127.0.0.1:8983/solr/dovecot/select?fl=uid&rows=1&sort=uid+desc&q=box:8864fa1d51ea1d5a7b12000096a1aaf8+AND+user:fail2ban at mydomain.com]: > >Free (requests left=1) >doveadm(fail2ban at mydomain.com): Debug: http-client: All requests >finished >doveadm(fail2ban at mydomain.com): Error: Mailbox INBOX: Status lookup >failed: Internal error occurred. Refer to server log for more >information. [2017-11-30 13:13:57] >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Peer close >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Peer disconnect >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Peer closed >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Connection disconnect >doveadm(fail2ban at mydomain.com): Debug: http-client: conn 127.0.0.1:8983 > >[0]: Connection destroy >doveadm(fail2ban at mydomain.com): Debug: http-client: peer >127.0.0.1:8983: >Peer destroy >doveadm(fail2ban at mydomain.com): Debug: http-client: host 127.0.0.1: >Host >destroy >doveadm(fail2ban at mydomain.com): Debug: http-client: queue >http://127.0.0.1:8983: Destroy-- Christian Kivalo
Thanks a lot for the help. You are right about the JSON as the default in solr v7.1.0. I now switched it back to XML and now it works. # doveadm -v index -u fail2ban at mydomain.com Inbox doveadm(fail2ban at mydomain.com): Info: INBOX: Caching mails seq=1..388 388/388 Gao On 2017-11-30 02:00 PM, Christian Kivalo wrote:> solr 7 and dovecot