tfpt review "/shelveset:TokenizerFixes;REDMOND\tomat" Refactors string terminators in Ruby tokenizer a bit. Removes unnecesary calls to newtok() method that allocated a new StringBuilder per each token. Fixes handling of \e in strings. Removes uses of obsolete method TryStatementBuilder::SkipIf. Removes TokenizerBuffer''s dependency on SourceUnitReader. Moves the responsibility for line mapping to the tokenizer. Tomas -------------- next part -------------- A non-text attachment was scrubbed... Name: TokenizerFixes.diff Type: application/octet-stream Size: 33701 bytes Desc: TokenizerFixes.diff URL: <http://rubyforge.org/pipermail/ironruby-core/attachments/20080715/9ef98b60/attachment-0001.obj>