I noticed that if I have a field that contains something like "Institute for medicine", that if I search using nay of these queries: for *for* for~ Nothing shows up. If I search for either of the other two words, though, that term would show up in the result set. Does this indicate that short words like "for" are not indexed? Thanks! Jen
What analyzer are you using? On Dec 28, 2005, at 7:13 PM, jennyw wrote:> I noticed that if I have a field that contains something like > "Institute > for medicine", that if I search using nay of these queries: > > for > *for* > for~ > > Nothing shows up. If I search for either of the other two words, > though, > that term would show up in the result set. Does this indicate that > short > words like "for" are not indexed?Jen - what analyzer are you using? If you''re using the default, it is the StandardAnalyzer, which removes these stop words during tokenization: ENGLISH_STOP_WORDS = [ "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "s", "such", "t", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", "with" ] Off the cuff, you should be able to adjust this to not remove any stop words by using: :analyzer => StandardAnalyzer.new([]) if you''re using the Index class Ferret provides. Erik
Erik Hatcher wrote:> :analyzer => StandardAnalyzer.new([]) >I am having a similar problem, and i''ve tried implementing your suggestion like this: acts_as_ferret :analyzer => Ferret::Analysis::StandardAnalyzer.new([]) in my indexed classes. I have rebuilt the indexes, but still can''t get some of these short words to return results. I''ve also found that words which have hyphens in them don''t work. Is there something else necessary in order to get this working in an active record class? Thanks, Cam -- Posted via http://www.ruby-forum.com/.
On 8/24/06, Cameron Hickey <rails at thepattern.net> wrote:> Erik Hatcher wrote: > > > :analyzer => StandardAnalyzer.new([]) > > > > > I am having a similar problem, and i''ve tried implementing your > suggestion like this: > > acts_as_ferret :analyzer => > Ferret::Analysis::StandardAnalyzer.new([]) > > in my indexed classes. > > I have rebuilt the indexes, but still can''t get some of these short > words to return results. > > I''ve also found that words which have hyphens in them don''t work. > > Is there something else necessary in order to get this working in an > active record class? > > Thanks, > > CamHi Cam, Get Ferret 0.9.6. This was a bug which should now be fixed. Better yet, wait until acts_as_ferret works with Ferret 0.10. Jens Kraemer is already working on it and I have few bugs to work out. As for hyphens not working, it sounds like the same analyzer is not being used for the queries but you''d have to check with one of the acts_as_ferret developers that you are using it correctly. Cheers, Dave
David Balmain wrote:> Get Ferret 0.9.6. This was a bug which should now be fixed. Better > yet, wait until acts_as_ferret works with Ferret 0.10. Jens Kraemer is > already working on it and I have few bugs to work out. As for hyphens > not working, it sounds like the same analyzer is not being used for > the queries but you''d have to check with one of the acts_as_ferret > developers that you are using it correctly.I am currently using 0.9.6 after reading about the update somewhere else. I didn''t have luck switching the analyzer, so i tried the alternate solution suggested elsewhere on this forum, which was just stripping the STOP words out of the original query. This actually seems to work somewhat well, but doesn''t solve the hypen problem. Can you clarify what you mean about using the same analyzer for queries? Is there any reason that hyphenated terms would not be getting indexed and searched properly by default? Thanks Cam -- Posted via http://www.ruby-forum.com/.
On 8/25/06, Cameron Hickey <rails at thepattern.net> wrote:> David Balmain wrote: > > Get Ferret 0.9.6. This was a bug which should now be fixed. Better > > yet, wait until acts_as_ferret works with Ferret 0.10. Jens Kraemer is > > already working on it and I have few bugs to work out. As for hyphens > > not working, it sounds like the same analyzer is not being used for > > the queries but you''d have to check with one of the acts_as_ferret > > developers that you are using it correctly. > > > I am currently using 0.9.6 after reading about the update somewhere > else. I didn''t have luck switching the analyzer, so i tried the > alternate solution suggested elsewhere on this forum, which was just > stripping the STOP words out of the original query. This actually seems > to work somewhat well, but doesn''t solve the hypen problem. > > Can you clarify what you mean about using the same analyzer for queries? > Is there any reason that hyphenated terms would not be getting indexed > and searched properly by default?This was in fact a bug. Thanks Cameron. It is fixed in subversion now. Cheers, Dave