Robert L Mathews
2015-May-08 20:34 UTC
Full text search indexes not used for header/body OR queries?
I've noticed that when using Lucene full text search, most queries use the indexes and/or header cache and are fast: . SEARCH BODY test . OK Search completed (0.001 secs). . SEARCH SUBJECT test . OK Search completed (0.053 secs). . SEARCH BODY test SUBJECT test . OK Search completed (0.002 secs). . SEARCH OR SUBJECT test FROM test . OK Search completed (0.093 secs). But an OR query that mixes headers and body does not use the available FTS indexes for the BODY part and is slow: . SEARCH OR BODY test SUBJECT test * OK Searched 62% of the mailbox, ETA 0:05 * OK Searched 70% of the mailbox, ETA 0:04 . OK Search completed (15.147 secs). Is this the expected behavior? Since the FTS code can handle an AND of header and body searches, I'm surprised it doesn't do the same for an OR. I noticed this while tracking down poor performance in Thunderbird, which issues searches like this: UID SEARCH RETURN (ALL) (OR FROM "Evelyn" (OR SUBJECT "Evelyn" (OR TO "Evelyn" (OR CC "Evelyn" BODY "Evelyn")))) NOT DELETED These are slow even with FTS enabled because of this behavior. I'm using Dovecot 2.1.7 from Debian wheezy. (I know this is outdated; however, I've examined the 2.1.x and 2.2.x changelogs and found no mention of it.) -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/
Robert L Mathews
2015-May-09 04:47 UTC
Full text search indexes not used for header/body OR queries?
As a followup to my own message: On 5/8/15 1:34 PM, Robert L Mathews wrote:> I've noticed that when using Lucene full text search, most queries use > the indexes and/or header cache and are fast [...] But an OR query that > mixes headers and body does not use the available > FTS indexes for the BODY part and is slow:This turned out to be my own fault because of a foolish mistake I made when testing. Dovecot actually works fine on all the search queries I mentioned, even in version 2.1.7. My apologies for the noise on the list. (My mistake was that when switching from Squat to Lucene, I didn't remove a local patch that prevented FTS from being used for header searches, because I thought the patch was only affecting Squat. That patch was to workaround what I reported in <http://www.dovecot.org/list/dovecot/2014-May/096360.html>. But the patch also affected Lucene.) -- Robert L Mathews, Tiger Technologies, http://www.tigertech.net/