Ferret talk - Jul 2007 - Inconsistent results when using wild card queries

We get some unexpected results when using wild card queries. We''re
using
aaf and Ferret 0.11.4

For exampel, when seraching on a part of a collegues name (kristofer)
and limiting it to a specific source_id:

Query: source_id:25 AND kri*
Result: 2 documents. None of them containg the word kristofer, but other
matching words, as "kring" and "kring?" (swedish)

Query: source_id:25 AND kris*
Result: 0 documents.

Query: source_id:25 AND krist*
Result: 12 document. Works as expected.

The index contains in total about 200 000 documents and I''ve tried
rebuilding and optimizing with no result.

Has anyone else experienced something similar? Any ideas how to fix it?

Thanks!

/David Wennergren

-- 
Posted via http://www.ruby-forum.com/.

David Wennergren <david.wennergren at gmail.com> writes:

Hi,
> Has anyone else experienced something similar? Any ideas how to fix it?
Unfortunatly i''ve also experienced that kind of weirdness. And most of
the time it as to do with accentuation.
i''m unable to match a single ? if I search for *?* (while it works
with wordwith?)
If i search for e it highlights single e, but it doesn''t for single
a...

Sorry to say that, but at the moment I''m considering using another
search enigne. (since I also have very weird unresolved issues with
highlighting)
I''m looking at xapian at the moment.

--

On Thu, Jul 05, 2007 at 11:05:36AM +0200, syrius.ml at no-log.org
wrote:
> David Wennergren <david.wennergren at gmail.com> writes:
> 
> Hi,
> 
> > Has anyone else experienced something similar? Any ideas how to fix
it?
> 
> Unfortunatly i''ve also experienced that kind of weirdness. And
most of
> the time it as to do with accentuation.
> i''m unable to match a single ? if I search for *?* (while it works
> with wordwith?)
I don''t know if this is acceptable for you in terms of result
exactness,
but you might consider replacing accentuated chars with their
ascii-counterparts during analysis.
> If i search for e it highlights single e, but it doesn''t for
single
> a...
wild guess - maybe this is because a is a stopword and e isn''t?
In general highlighting ''e'' works, as does highlighting
''a'', as long as
you use an analyzer with empty stopword list:

require ''ferret''
include Ferret
i = I.new :analyzer => Analysis::StandardAnalyzer.new([])
i << ''A tree in the woods''
i << ''Some sentence with e''
i.highlight ''a'', 0, :field => :id
# => ["<b>A</b> tree in the woods"]
i.highlight ''e'', 1, :field => :id
# => ["Some sentence with <b>e</b>"]


Jens

-- 
Jens Kr?mer
webit! Gesellschaft f?r neue Medien mbH
Schnorrstra?e 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
kraemer at webit.de | www.webit.de
 
Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

Jens Kraemer <kraemer at webit.de> writes:
>> > Has anyone else experienced something similar? Any ideas how to
fix it?
>> 
>> Unfortunatly i''ve also experienced that kind of weirdness. And
most of
>> the time it as to do with accentuation.
>> i''m unable to match a single ? if I search for *?* (while it
works
>> with wordwith?)
>
> I don''t know if this is acceptable for you in terms of result
exactness,
> but you might consider replacing accentuated chars with their
> ascii-counterparts during analysis.
Thanks for your quick answers Jens.
It could be acceptable, but the highlighting problems I''ve discovered
are stopping me from doing any further development.
Unfortunatly I don''t have time to fix them myself and Dave seems very
busy. :(

sorry if it sounds like whinging :)

Cheers

--

On Thu, Jul 05, 2007 at 12:19:52PM +0200, syrius.ml at no-log.org
wrote:
> Jens Kraemer <kraemer at webit.de> writes:
> 
> >> > Has anyone else experienced something similar? Any ideas how
to fix it?
> >> 
> >> Unfortunatly i''ve also experienced that kind of
weirdness. And most of
> >> the time it as to do with accentuation.
> >> i''m unable to match a single ? if I search for *?* (while
it works
> >> with wordwith?)
> >
> > I don''t know if this is acceptable for you in terms of result
exactness,
> > but you might consider replacing accentuated chars with their
> > ascii-counterparts during analysis.
> 
> Thanks for your quick answers Jens.
> It could be acceptable, but the highlighting problems I''ve
discovered
> are stopping me from doing any further development.
> Unfortunatly I don''t have time to fix them myself and Dave seems
very
> busy. :(
if you really like to switch, did you consider acts_as_solr? it''s API
is
much like aaf''s.


Jens

-- 
Jens Kr?mer
webit! Gesellschaft f?r neue Medien mbH
Schnorrstra?e 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
kraemer at webit.de | www.webit.de
 
Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa

Jens Kraemer <kraemer at webit.de> writes:
> if you really like to switch, did you consider acts_as_solr? it''s
API is
> much like aaf''s.
I certainly would if I was ok to use java. :) (but i''m not)
at the moment, I''m considering hyperestraier and xapian.
If there were a python api + rails plugin (and also as much features
as ferret) that would be perfect :)
I haven''t really looked/tested yet :)

--

On Thu, Jul 05, 2007 at 04:55:59PM +0200, syrius.ml at no-log.org
wrote:
> Jens Kraemer <kraemer at webit.de> writes:
> 
> > if you really like to switch, did you consider acts_as_solr?
it''s API is
> > much like aaf''s.
> 
> I certainly would if I was ok to use java. :) (but i''m not)
afair you need no Java skills to get Solr running, however you''ll need
some spare server resources, that''s for sure ;-)
> at the moment, I''m considering hyperestraier and xapian.
> If there were a python api + rails plugin (and also as much features
> as ferret) that would be perfect :)
Solr has an http interface, so talking to it from python would be no big
deal.

Otherwise you could now, possibly being the first user of xapian in a rails app,
start your very own acts_as_xapian ;-)


Jens


-- 
Jens Kr?mer
webit! Gesellschaft f?r neue Medien mbH
Schnorrstra?e 76 | 01069 Dresden
Telefon +49 351 46766-0 | Telefax +49 351 46766-66
kraemer at webit.de | www.webit.de
 
Amtsgericht Dresden | HRB 15422
GF Sven Haubold, Hagen Malessa