Rails core - Mar 2012 - Does Ruby 1.9 support Unicode normalization yet?

In the process of upgrading from 1.8 to 1.9 we are getting a lot of
warnings about "Ruby 1.9 doesn''t support Unicode normalization
yet".
However the commit that added those lines is from 2008 and just
mentions "Ruby 1.9 compat: no Unicode normalization support yet"
without any references. Does anyone know whether this is still true
for ruby 1.9 and for which minor versions of 1.9?

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Tue, Mar 13, 2012 at 06:59:17PM -0700, Sam Grönblom
wrote:
> In the process of upgrading from 1.8 to 1.9 we are getting a lot of
> warnings about "Ruby 1.9 doesn''t support Unicode
normalization yet".
> However the commit that added those lines is from 2008 and just
> mentions "Ruby 1.9 compat: no Unicode normalization support yet"
> without any references. Does anyone know whether this is still true
> for ruby 1.9 and for which minor versions of 1.9?
Ruby doesn''t support unicode normalization natively, but you can use
this gem:

  https://github.com/knu/ruby-unf

Hope that helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/

So using Iconv for transliteration is not working properly in 1.9 or?

On Wednesday, March 14, 2012 11:47:18 AM UTC+9, Aaron Patterson
wrote:
>
> On Tue, Mar 13, 2012 at 06:59:17PM -0700, Sam Grönblom wrote:
> > In the process of upgrading from 1.8 to 1.9 we are getting a lot of
> > warnings about "Ruby 1.9 doesn''t support Unicode
normalization yet".
> > However the commit that added those lines is from 2008 and just
> > mentions "Ruby 1.9 compat: no Unicode normalization support
yet"
> > without any references. Does anyone know whether this is still true
> > for ruby 1.9 and for which minor versions of 1.9?
>
> Ruby doesn''t support unicode normalization natively, but you can
use
> this gem:
>
>   https://github.com/knu/ruby-unf
>
> Hope that helps.
>
> -- 
> Aaron Patterson
> http://tenderlovemaking.com/
>
>
-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/rubyonrails-core/-/T-dScIrCvuoJ.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Tue, Mar 13, 2012 at 09:10:34PM -0700, Sam Grönblom
wrote:
> So using Iconv for transliteration is not working properly in 1.9 or?
You can still use it, but AFAIK iconv support is going to be dropped.
The encoding system in 1.9 will let you convert between encodings just
fine, but it doesn''t do transliteration.

Probably the best path would either be to extract the iconv extension
from Ruby and provide it as a gem, or to use an existing gem like this
one:

  https://github.com/rsl/stringex

I know it''s not the ideal answer, but I hope it helps.

-- 
Aaron Patterson
http://tenderlovemaking.com/

2012/3/14 Sam Grönblom <sgronblo@gmail.com>:
> So using Iconv for transliteration is not working properly in 1.9 or?
iconv''s //TRANSLIT is not Unicode normalization.

Moreover iconv translit works only with glibc and GNU libiconv,
and the actual behavior is not documented.

-- 
NARUSE, Yui  <naruse@airemix.jp>

-- 
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Core" group.
To post to this group, send email to rubyonrails-core@googlegroups.com.
To unsubscribe from this group, send email to
rubyonrails-core+unsubscribe@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/rubyonrails-core?hl=en.

On Thu, Mar 15, 2012 at 04:47:21AM +0900, NARUSE, Yui
wrote:
> 2012/3/14 Sam Grönblom <sgronblo@gmail.com>:
> > So using Iconv for transliteration is not working properly in 1.9 or?
> 
> iconv''s //TRANSLIT is not Unicode normalization.
> 
> Moreover iconv translit works only with glibc and GNU libiconv,
> and the actual behavior is not documented.
Ah.  Thanks for the clarification!

-- 
Aaron Patterson
http://tenderlovemaking.com/