Ferret talk - Apr 2006 - Creating my own analyzer

I created this analyzer:

class DescriptionAnalyzer < Ferret::Analysis::Analyzer
  def token_stream(field, string)
    if field == "code"
      return CodeTokenStream.new(string)
    else
      return Ferret::Analysis::Analyzer.new.token_stream(field,string)
    end
  end
end

and created an IndexWriter with it:

Ferret::Index::IndexWriter.new(get_index_path, 
                               :create => true,
                               :analyzer => DescriptionAnalyzer.new)

get_index_path is a method to calculate the path to use.

The problem is that my token_stream method is never called.

I''m using ferret 0.9.1 from rubygems, ruby is:

ruby 1.8.4 (2005-12-24) [i486-linux]

I''m running this in a rails app, but the script that creates the index
only uses ActiveRecord to iterate all the records and index them.

Any ideas?

Greetings,

Pedro.

Sorry Pedro, you can''t create your own analyzer in 0.9.1. If you want
to check out the latest version from subversion, this should work.

Cheers,
Dave

On 4/21/06, Pedro C?rte-Real <Pedro.CorteReal at iantt.pt>
wrote:
> I created this analyzer:
>
> class DescriptionAnalyzer < Ferret::Analysis::Analyzer
>   def token_stream(field, string)
>     if field == "code"
>       return CodeTokenStream.new(string)
>     else
>       return Ferret::Analysis::Analyzer.new.token_stream(field,string)
>     end
>   end
> end
>
> and created an IndexWriter with it:
>
> Ferret::Index::IndexWriter.new(get_index_path,
>                                :create => true,
>                                :analyzer => DescriptionAnalyzer.new)
>
> get_index_path is a method to calculate the path to use.
>
> The problem is that my token_stream method is never called.
>
> I''m using ferret 0.9.1 from rubygems, ruby is:
>
> ruby 1.8.4 (2005-12-24) [i486-linux]
>
> I''m running this in a rails app, but the script that creates the
index
> only uses ActiveRecord to iterate all the records and index them.
>
> Any ideas?
>
> Greetings,
>
> Pedro.
>
> _______________________________________________
> Ferret-talk mailing list
> Ferret-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ferret-talk
>