There is nothing fancy about the CJKAnalyzer.... it chunks characters
into pairs. So the phrase ??? would be tokenized into two
tokens [??] [??].
Erik
On Feb 23, 2006, at 1:54 AM, David Balmain wrote:
> Hi Jerry,
> Basically you''ll have to write an analyzer that matches Chinese
tokens
> (words). If you can write a regular expression in Ruby that matches
> Chinese tokens then it''s very simple to write an Analyzer for
Ferret.
> I haven''t looked at teh CJKAnalyzer in Lucene but I can''t
imagine it
> would be too hard to port to Ruby.
>
> Cheers,
> Dave
>
> On 2/23/06, Jerry Liu <bigliu at gmail.com> wrote:
>> I need decide on if our site will go with Java or Ruby on Rails. The
>> major factor is that does Farret support Lucene''s
ChineseAnalyzer or
>> CJKAnalyzer or not.
>>
>> Can anyboby shine some lights on Farret''s Chinese search
support?
>>
>> Really appreciate.
>>
>> --
>> Posted via http://www.ruby-forum.com/.
>> _______________________________________________
>> Ferret-talk mailing list
>> Ferret-talk at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/ferret-talk
>>
>
> _______________________________________________
> Ferret-talk mailing list
> Ferret-talk at rubyforge.org
> http://rubyforge.org/mailman/listinfo/ferret-talk