Ferret talk - Oct 2006 - Problems with stop word analysis and queries

I have a problem, and I think it''s because stop word analysis
isn''t
happenning in the queries.  I think it''s the way ferret is doing things
that''s causing this bug.  Let''s say I''m searching
across documents with
a title.  And I have a document with a title of "Bash Guide for 
Beginners".  If a user types in the query:

Bash Guide for Beginners

No quotes.  I get no hits.  But if I drop the "for" ferret finds it. 
Then say I type a quoted query like "Bash Guide for Beginners" ferret 
finds it.  So I tried all of this from a rails app, and I thought maybe 
acts_as_ferret was doing something with the query to cause the "for" 
word to stay in the query.  But, I then opened up a script/console and 
fetched the ferret index directly like:

findex = MyModel.ferret_index
findex.search("Bash Guide for Beginners")

And nothing came up.  I tried all three queries, full title no quotes, 
full title minus "for", and quoted query with the same results as
above.
So now I think it might be a problem with ferret.  My theory is maybe 
the stop word analysis isn''t taking place when I submit queries. 
That''s
my theory at least.  Any ideas?

Charlie

-- 
Posted via http://www.ruby-forum.com/.

On 10/17/06, Charlie Hubbard <charlie.hubbard at gmail.com>
wrote:
>
> I have a problem, and I think it''s because stop word analysis
isn''t
> happenning in the queries.  I think it''s the way ferret is doing
things
> that''s causing this bug.  Let''s say I''m
searching across documents with
> a title.  And I have a document with a title of "Bash Guide for
> Beginners".  If a user types in the query:
>
> Bash Guide for Beginners
>
> No quotes.  I get no hits.  But if I drop the "for" ferret finds
it.
> Then say I type a quoted query like "Bash Guide for Beginners"
ferret
> finds it.  So I tried all of this from a rails app, and I thought maybe
> acts_as_ferret was doing something with the query to cause the
"for"
> word to stay in the query.  But, I then opened up a script/console and
> fetched the ferret index directly like:
>
> findex = MyModel.ferret_index
> findex.search("Bash Guide for Beginners")
>
> And nothing came up.  I tried all three queries, full title no quotes,
> full title minus "for", and quoted query with the same results as
above.
> So now I think it might be a problem with ferret.  My theory is maybe
> the stop word analysis isn''t taking place when I submit queries. 
That''s
> my theory at least.  Any ideas?
>
> Charlie
Hi Charlie,
I''m afraid I can''t reproduce the problem here. So unless
someone else
can help you I see you have two options. Try reproducing the problem
with a small script like this:

    require ''rubygems''
    require ''ferret''

    index = Ferret::I.new(:or_default => false)
    index << "Bash Guide for Beginners"
    puts index.search(''Bash Guide for Beginners'')
    index.close

Or you could send me the index off-list and I''ll have a look at it
here.

Cheers,
Dave

Ok thanks for looking into.  I''m going to try and reproduce it with 
plain ferret, if not I''ll send you the index.

Charlie

David Balmain wrote:
> On 10/17/06, Charlie Hubbard <charlie.hubbard at gmail.com> wrote:
>> Then say I type a quoted query like "Bash Guide for
Beginners" ferret
>> So now I think it might be a problem with ferret.  My theory is maybe
>> the stop word analysis isn''t taking place when I submit
queries.  That''s
>> my theory at least.  Any ideas?
>>
>> Charlie
> 
> Hi Charlie,
> I''m afraid I can''t reproduce the problem here. So unless
someone else
> can help you I see you have two options. Try reproducing the problem
> with a small script like this:
> 
>     require ''rubygems''
>     require ''ferret''
> 
>     index = Ferret::I.new(:or_default => false)
>     index << "Bash Guide for Beginners"
>     puts index.search(''Bash Guide for Beginners'')
>     index.close
> 
> Or you could send me the index off-list and I''ll have a look at it
here.
> 
> Cheers,
> Dave

-- 
Posted via http://www.ruby-forum.com/.