On 3 Apr 2019, at 20.30, Joan Moreau via dovecot <dovecot at dovecot.org> wrote:> doveadm search -u jom at grosjo.net mailbox inbox text milan > output > > doveadm(jom at grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox OR uid:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan OR uid:milan ) > > 1 - The query is wrongThat's because fts_backend_xapian_lookup() isn't anywhere close to being correct. Try to copy the logic based on solr_add_definite_query_args().
For this first point, the problem is that dovecot core sends TWICE the request and "Inbox" appears in the list of arguments ! (inbox shall serve to select teh right mailbox, never sent to the backend) And even if this would be solved, the dovecot core loops *after* the backend hs returneds the results # doveadm search -u jom at grosjo.net mailbox inbox text milan doveadm(jom at grosjo.net): Info: Get last UID of INBOX = 315526 doveadm(jom at grosjo.net): Info: Get last UID of INBOX = 315526 doveadm(jom at grosjo.net): Info: Query: FLAG=AND doveadm(jom at grosjo.net): Info: Query(1): add term(wilcard) : inbox doveadm(jom at grosjo.net): Info: Query(2): add term(wilcard) : milan doveadm(jom at grosjo.net): Info: Testing if wildcard doveadm(jom at grosjo.net): Info: Query: set GLOBAL (no specified header) doveadm(jom at grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan ) DOVEADM(JOM at GROSJO.NET): INFO: QUERY: 2 RESULTS IN 1 MS // THIS IS WHEN BACKEND HAS FOUND RESULTS AND STOPPED d82b4b0f550d38593644000095331209 847 d82b4b0f550d38593644000095331209 1569 d82b4b0f550d38593644000095331209 2260 d82b4b0f550d38593644000095331209 2575 d82b4b0f550d38593644000095331209 2811 d82b4b0f550d38593644000095331209 2885 d82b4b0f550d38593644000095331209 3038 D82B4B0F550D38593644000095331209 3121 -> LOOPING FOREVER On 2019-04-21 09:57, Timo Sirainen via dovecot wrote:> On 3 Apr 2019, at 20.30, Joan Moreau via dovecot <dovecot at dovecot.org> wrote: > >> doveadm search -u jom at grosjo.net mailbox inbox text milan >> output >> >> doveadm(jom at grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox OR uid:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan OR uid:milan ) >> >> 1 - The query is wrong > > That's because fts_backend_xapian_lookup() isn't anywhere close to being correct. Try to copy the logic based on solr_add_definite_query_args().-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190421/453f773a/attachment.html>
Inbox appears in the list of arguments, because fts_backend_xapian_lookup() is parsing the search args wrong. Not sure about the other issue.> On 21 Apr 2019, at 19.31, Joan Moreau <jom at grosjo.net> wrote: > > For this first point, the problem is that dovecot core sends TWICE the request and "Inbox" appears in the list of arguments ! (inbox shall serve to select teh right mailbox, never sent to the backend) > > And even if this would be solved, the dovecot core loops *after* the backend hs returneds the results > > > > # doveadm search -u jom at grosjo.net mailbox inbox text milan > doveadm(jom at grosjo.net): Info: Get last UID of INBOX = 315526 > doveadm(jom at grosjo.net): Info: Get last UID of INBOX = 315526 > doveadm(jom at grosjo.net): Info: Query: FLAG=AND > doveadm(jom at grosjo.net): Info: Query(1): add term(wilcard) : inbox > doveadm(jom at grosjo.net): Info: Query(2): add term(wilcard) : milan > doveadm(jom at grosjo.net): Info: Testing if wildcard > doveadm(jom at grosjo.net): Info: Query: set GLOBAL (no specified header) > doveadm(jom at grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan ) > doveadm(jom at grosjo.net): Info: Query: 2 results in 1 ms // THIS IS WHEN BACKEND HAS FOUND RESULTS AND STOPPED > d82b4b0f550d38593644000095331209 847 > d82b4b0f550d38593644000095331209 1569 > d82b4b0f550d38593644000095331209 2260 > d82b4b0f550d38593644000095331209 2575 > d82b4b0f550d38593644000095331209 2811 > d82b4b0f550d38593644000095331209 2885 > d82b4b0f550d38593644000095331209 3038 > d82b4b0f550d38593644000095331209 3121 -> LOOPING FOREVER > > > > > > > On 2019-04-21 09:57, Timo Sirainen via dovecot wrote: > >> On 3 Apr 2019, at 20.30, Joan Moreau via dovecot <dovecot at dovecot.org <mailto:dovecot at dovecot.org>> wrote: >>> >>> doveadm search -u jom at grosjo.net <mailto:jom at grosjo.net> mailbox inbox text milan >>> output >>> >>> doveadm(jom at grosjo.net <mailto:jom at grosjo.net>): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox OR uid:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan OR uid:milan ) >>> >>> 1 - The query is wrong >> >> That's because fts_backend_xapian_lookup() isn't anywhere close to being correct. Try to copy the logic based on solr_add_definite_query_args(). >> >>-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190421/dc2c2987/attachment.html>
Antoher example so you understand how may understand the bug in dovecote core : # doveadm search -u jom at grosjo.net mailbox SENT text milan doveadm(jom at grosjo.net): Info: Get last UID of Sent = 61707 -> CORRECTLY ASSIGNED THE PROPER MAILBOX TO THE BACK END doveadm(jom at grosjo.net): Info: Get last UID of Sent = 61707 doveadm(jom at grosjo.net): Info: Query: FLAG=AND doveadm(jom at grosjo.net): Info: Query(1): add term(wilcard) : Sent -> WHY IS "SENT" AMONG THE SERACH PARAMETERS ??? doveadm(jom at grosjo.net): Info: Query(2): add term(wilcard) : milan doveadm(jom at grosjo.net): Info: Testing if wildcard doveadm(jom at grosjo.net): Info: Query: set GLOBAL (no specified header) doveadm(jom at grosjo.net): Info: Query : ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan ) AND ( bcc:sent OR body:sent OR cc:sent OR from:sent OR message-id:sent OR subject:sent OR to:sent ) doveadm(jom at grosjo.net): Info: Query: 7 results in 71 ms (AND SAME LOOP) In this example, the "Sent" shall *never* be passed as argument to the backend (xapian, solr or any other), only the mailbox reference. However, it appears in the search parameters On 2019-04-21 10:31, Joan Moreau via dovecot wrote:> For this first point, the problem is that dovecot core sends TWICE the request and "Inbox" appears in the list of arguments ! (inbox shall serve to select teh right mailbox, never sent to the backend) > > And even if this would be solved, the dovecot core loops *after* the backend hs returneds the results > > # doveadm search -u jom at grosjo.net mailbox inbox text milan > doveadm(jom at grosjo.net): Info: Get last UID of INBOX = 315526 > doveadm(jom at grosjo.net): Info: Get last UID of INBOX = 315526 > doveadm(jom at grosjo.net): Info: Query: FLAG=AND > doveadm(jom at grosjo.net): Info: Query(1): add term(wilcard) : inbox > doveadm(jom at grosjo.net): Info: Query(2): add term(wilcard) : milan > doveadm(jom at grosjo.net): Info: Testing if wildcard > doveadm(jom at grosjo.net): Info: Query: set GLOBAL (no specified header) > doveadm(jom at grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan ) > DOVEADM(JOM at GROSJO.NET): INFO: QUERY: 2 RESULTS IN 1 MS // THIS IS WHEN BACKEND HAS FOUND RESULTS AND STOPPED > d82b4b0f550d38593644000095331209 847 > d82b4b0f550d38593644000095331209 1569 > d82b4b0f550d38593644000095331209 2260 > d82b4b0f550d38593644000095331209 2575 > d82b4b0f550d38593644000095331209 2811 > d82b4b0f550d38593644000095331209 2885 > d82b4b0f550d38593644000095331209 3038 > D82B4B0F550D38593644000095331209 3121 -> LOOPING FOREVER > > On 2019-04-21 09:57, Timo Sirainen via dovecot wrote: > On 3 Apr 2019, at 20.30, Joan Moreau via dovecot <dovecot at dovecot.org> wrote: doveadm search -u jom at grosjo.net mailbox inbox text milan > output > > doveadm(jom at grosjo.net): Info: Query : ( bcc:inbox OR body:inbox OR cc:inbox OR from:inbox OR message-id:inbox OR subject:inbox OR to:inbox OR uid:inbox ) AND ( bcc:milan OR body:milan OR cc:milan OR from:milan OR message-id:milan OR subject:milan OR to:milan OR uid:milan ) > > 1 - The query is wrong > That's because fts_backend_xapian_lookup() isn't anywhere close to being correct. Try to copy the logic based on solr_add_definite_query_args().-------------- next part -------------- An HTML attachment was scrubbed... URL: <https://dovecot.org/pipermail/dovecot/attachments/20190421/c73795e0/attachment-0001.html>