Xapian discuss - Sep 2018 - xapian parser bug?

Today we noticed that keywords can't be searched as prefixed terms. Or
that's what it looks like anyway. I tested and, or, and not.

╰─% NOTMUCH_DEBUG_QUERY=y notmuch search 'subject:"and"'
Query string is:
subject:"and"
notmuch search: A Xapian exception occurred
A Xapian exception occurred parsing query: Syntax: <expression> AND
<expression>
Query string was: subject:"and"

╰─% NOTMUCH_DEBUG_QUERY=y notmuch search 'subject:"or"' 
Query string is:
subject:"or"
notmuch search: A Xapian exception occurred
A Xapian exception occurred parsing query: Syntax: <expression> OR
<expression>
Query string was: subject:"or"

╰─% NOTMUCH_DEBUG_QUERY=y notmuch search 'subject:"not"'
Query string is:
subject:"not"
notmuch search: A Xapian exception occurred
A Xapian exception occurred parsing query: Syntax: <expression> NOT
<expression>
Query string was: subject:"not"

Interestingly, putting space around the operator seems to be a
workaround. Something about turning on phrase parsing maybe?

╰─% NOTMUCH_DEBUG_QUERY=y notmuch count 'subject:" not "'
Query string is:
subject:" not "
Exclude query is:
Query((((Kspam OR Kdeleted) OR Kmuted) OR Kbad-address))
Final query is:
Query(((Tmail AND 0 * XSUBJECTnot at 1) AND_NOT (((Kspam OR Kdeleted) OR Kmuted)
OR Kbad-address)))
9927

David — this isn't the behaviour I see what QueryParser alone, unless
you're driving it in a way I don't expect. In python:
>>> import xapian
>>> qp = xapian.QueryParser()
>>> qp.add_prefix('subject', 'S')
>>> str(qp.parse_query('subject:"and"',
qp.FLAG_DEFAULT|qp.FLAG_BOOLEAN_ANY_CASE))
'Query(Sand at 1)'
>>> str(qp.parse_query('subject:"or"',
qp.FLAG_DEFAULT|qp.FLAG_BOOLEAN_ANY_CASE))
'Query(Sor at 1)'
>>> str(qp.parse_query('subject:"not"',
qp.FLAG_DEFAULT|qp.FLAG_BOOLEAN_ANY_CASE))
'Query(Snot at 1)'
>>> str(qp.parse_query('subject:" not "',
qp.FLAG_DEFAULT|qp.FLAG_BOOLEAN_ANY_CASE))
'Query(Snot at 1)'

Note that I'm using 1.4.7, and from your output I believe you're not
(the * in the query description I believe doesn't happen in those situations
any more).

J
> On 29 Sep 2018, at 23:09, David Bremner <david at tethera.net> wrote:
> 
> 
> Today we noticed that keywords can't be searched as prefixed terms. Or
> that's what it looks like anyway. I tested and, or, and not.
> 
> ╰─% NOTMUCH_DEBUG_QUERY=y notmuch search 'subject:"and"'
> Query string is:
> subject:"and"
> notmuch search: A Xapian exception occurred
> A Xapian exception occurred parsing query: Syntax: <expression> AND
<expression>
> Query string was: subject:"and"
> 
> ╰─% NOTMUCH_DEBUG_QUERY=y notmuch search 'subject:"or"' 
> Query string is:
> subject:"or"
> notmuch search: A Xapian exception occurred
> A Xapian exception occurred parsing query: Syntax: <expression> OR
<expression>
> Query string was: subject:"or"
> 
> ╰─% NOTMUCH_DEBUG_QUERY=y notmuch search 'subject:"not"'
> Query string is:
> subject:"not"
> notmuch search: A Xapian exception occurred
> A Xapian exception occurred parsing query: Syntax: <expression> NOT
<expression>
> Query string was: subject:"not"
> 
> Interestingly, putting space around the operator seems to be a
> workaround. Something about turning on phrase parsing maybe?
> 
> ╰─% NOTMUCH_DEBUG_QUERY=y notmuch count 'subject:" not "'
> Query string is:
> subject:" not "
> Exclude query is:
> Query((((Kspam OR Kdeleted) OR Kmuted) OR Kbad-address))
> Final query is:
> Query(((Tmail AND 0 * XSUBJECTnot at 1) AND_NOT (((Kspam OR Kdeleted) OR
Kmuted) OR Kbad-address)))
> 9927
> 
-- 
 James Aylett
 devfort.com — spacelog.org — tartarus.org/james/

On Sun, Sep 30, 2018 at 09:50:30AM +0100, James Aylett
wrote:
> Note that I'm using 1.4.7, and from your output I believe you're
not
> (the * in the query description I believe doesn't happen in those
> situations any more).
1.4.4 and later eliminate redundant 0 scaling factors, but this one
isn't actually redundant:
> > Query(((Tmail AND 0 * XSUBJECTnot at 1) AND_NOT (((Kspam OR Kdeleted)
OR Kmuted) OR Kbad-address)))
If it was on the right-hand side of AND_NOT it would be eliminated
(because the right-hand side doesn't contribute any weight anyway).

FWIW, I also couldn't reproduce this (I tried with quest and 1.4.7):

$ quest -psubject:S -fdefault,boolean_any_case 'subject:"and"'
Parsed Query: Query(Sand at 1)

Cheers,
    Olly