Rails - Mar 2008 - Converting data to utf8 to load into an ActiveRecord?

Hi,
I''ve got a text file of data that I need to parse and loaded into my
Rails
app. I''ve created a migration script the loads and parses the file, but
some
of the data contains non-utf8 strings, causing an exception.

I''ve found two Windows only ways to convert the strings to utf8, but I
would
like to deploy my app on heroku. Any suggestions for a non OS specific way
to convert?

This is my current implementation using chilkat:

require ''db/chilkat''

class AddDataSources < ActiveRecord::Migration
  def self.encode(str)
    strObj = Chilkat::CkString.new()
    strObj.appendAnsi(str)
    strObj.urlDecode("utf-8")
  end

  def self.up
    DataSource.delete_all
    data = IO.readlines(''db/data/data.txt'')
    data.each { |line|
      cells = line.split('','')
      DataSource.create(:code => cells[0],
                        :authors => encode(cells[1]))
    }
  end

  def self.down
    DataSource.delete_all
  end
end


-- 
Henry
http://www.henrywagner.org/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

2008/3/20, Henry Wagner
<hjw3001-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> Hi,
>
> I''ve got a text file of data that I need to parse and loaded into
my Rails
> app. I''ve created a migration script the loads and parses the
file, but some
> of the data contains non-utf8 strings, causing an exception.
>
> I''ve found two Windows only ways to convert the strings to utf8,
but I would
> like to deploy my app on heroku. Any suggestions for a non OS specific way
> to convert?
Iconv?

    irb(main):001:0> require "iconv"
    => true
    irb(main):002:0> Iconv.conv("utf-8", "windows-1252",
"\200")
    => "€"

Stefan

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---

Thanks, that works.

On Thu, Mar 20, 2008 at 12:12 PM, Stefan Lang <
perfectly.normal.hacker-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
>
> 2008/3/20, Henry Wagner
<hjw3001-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>:
> > Hi,
> >
> > I''ve got a text file of data that I need to parse and loaded
into my
> Rails
> > app. I''ve created a migration script the loads and parses the
file, but
> some
> > of the data contains non-utf8 strings, causing an exception.
> >
> > I''ve found two Windows only ways to convert the strings to
utf8, but I
> would
> > like to deploy my app on heroku. Any suggestions for a non OS specific
> way
> > to convert?
>
> Iconv?
>
>    irb(main):001:0> require "iconv"
>    => true
>    irb(main):002:0> Iconv.conv("utf-8",
"windows-1252", "\200")
>    => "€"
>
> Stefan
>
> >
>

-- 
Henry
http://www.henrywagner.org/

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Ruby on Rails: Talk" group.
To post to this group, send email to
rubyonrails-talk-/JYPxA39Uh5TLH3MbocFFw@public.gmane.org
To unsubscribe from this group, send email to
rubyonrails-talk-unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/rubyonrails-talk?hl=en
-~----------~----~----~----~------~----~------~--~---