Displaying 1 result from an estimated 1 matches for "file_to_tokenize".
2006 Apr 12
1
testing tokenizers
Hi,
is there a way to test tokenizers? I mean, I want to give input stream
and see the output tokens.
AND is there a way to see an indexed document''s index tokens? Which
words in the document are used to index this document?
Thanks in advance
Onur