Jim Lynch
2012-Jun-04 16:31 UTC
[Xapian-discuss] Search not finding queries with stop words.
I have a search in perl that looks a bit like: my $qp = new Search::Xapian::QueryParser(); $qp->set_stemmer(new Search::Xapian::Stem("english")); $qp->set_stemming_strategy(STEM_SOME); $qp->set_default_op($defaultop); ... my $par = $qp->parse_query($query); my $enq = $xDatabase->enquire( $par ); and in the db create script: my $stopper = Search::Xapian::SimpleStopper->new(); foreach my $word (@ar) { $stopper->add($word); } ... my $doc = Search::Xapian::Document->new(); my $indexer = Search::Xapian::TermGenerator->new(); my $stemmer = Search::Xapian::Stem->new('english'); $doc->set_data($jsonText); $indexer->set_stemmer($stemmer); $indexer->set_stopper($stopper); $indexer->set_document($doc); $indexer->index_text($docBody); $indexer->increase_termpos(); $indexer->index_text($subject); ... (other index_text and add_value calls) $xdb->add_document($doc); If I look for something like index of elements, I get no results even though that phrase exists (no, I don't do a phrase search, just those three words separated by spaces). However if I leave off the "of" I get the expected number of hits. I'm guessing I have to remove the stop words from the query? I default to OP_OR so I would think it would work. Can anyone shed some light on this? Thanks, Jim.
Olly Betts
2012-Jun-05 01:16 UTC
[Xapian-discuss] Search not finding queries with stop words.
On Mon, Jun 04, 2012 at 12:31:05PM -0400, Jim Lynch wrote:> If I look for something like index of elements, I get no results even > though that phrase exists (no, I don't do a phrase search, just those > three words separated by spaces). However if I leave off the "of" I get > the expected number of hits. I'm guessing I have to remove the stop > words from the query? I default to OP_OR so I would think it would > work. Can anyone shed some light on this?Yes, you need to set the same stopper at query time too. There are some helpful tips for debugging this sort of situation here: http://trac.xapian.org/wiki/FAQ/NoMatches Cheers, Olly