search for: tinysegmenter

...on mentions switching to a language-specific segmentation algorithm; since a particular algorithm isn't specified I'm guessing that the evaluation and selection of an algorithm would necessarily be part of the project? I've worked with Japanese text mostly in Python and so I've used TinySegmenter in Python<https://code.google.com/p/mhagiwara/source/browse/trunk/nltk/jpbook/tinysegmenter.py> when I needed segmentation; I've run across a couple of other options that were available in C++ but I haven't really had occasion to use them. So basically, I've dealt with Japanese se...