I am a very amateur Rubyist who, amongst other things, likes to use a simple Rails app to query my company''s MySQL config database. The server I now use to do this has got 1.9.1 and Rails 2.3.3. I''ve now hit the ''problems'' related to 1.9 and string encoding, which means that when Rails try to display, say, E acute characters, it throws an invalid byte sequence, namely ArgumentError (invalid byte sequence in UTF-8): Given that I only access the MySQL database over a private network and with a read-only account, is there some simple and easy way to suppress this issue? Without being an expert in this area (obviously) I guess that either I can try to "tell" Ruby to treat the MySQL data as an encoding other than UTF-8 (I guess US-ASCII but it could be trial and error to work out what), and/or I could add some rescue code to find (and ignore) bad byte sequences. I''ve tried to find recipes for both the above, but quickly get lost in the subtleties of it all! Any and all help appreciated. Many thanks in advance. -- Posted via http://www.ruby-forum.com/.
On Nov 2, 5:31 pm, Toby Rodwell <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> wrote:> I am a very amateur Rubyist who, amongst other things, likes to use a > simple Rails app to query my company''s MySQL config database. The > server I now use to do this has got 1.9.1 and Rails 2.3.3. I''ve now hit > the ''problems'' related to 1.9 and string encoding, which means that when > Rails try to display, say, E acute characters, it throws an invalid byte > sequence, namely > ArgumentError (invalid byte sequence in UTF-8): > > Given that I only access the MySQL database over a private network and > with a read-only account, is there some simple and easy way to suppress > this issue? Without being an expert in this area (obviously) I guess > that either I can try to "tell" Ruby to treat the MySQL data as an > encoding other than UTF-8 (I guess US-ASCII but it could be trial and > error to work out what), and/or I could add some rescue code to find > (and ignore) bad byte sequences. I''ve tried to find recipes for both > the above, but quickly get lost in the subtleties of it all! Any and > all help appreciated. Many thanks in advance.I''d check with the whoever admins the MySQL DB to find out what character set it''s actually using. I think you can then tell the adapter to translate. Best guess is either US ASCII, or (more likely) Windows-1252 pretending to be ASCII. --Matt Jones
Matt Jones wrote:> On Nov 2, 5:31�pm, Toby Rodwell <rails-mailing-l...-ARtvInVfO7ksV2N9l4h3zg@public.gmane.org> > wrote: >> this issue? �Without being an expert in this area (obviously) I guess >> that either I can try to "tell" Ruby to treat the MySQL data as an >> encoding other than UTF-8 (I guess US-ASCII �but it could be trial and >> error to work out what), and/or I could add some rescue code to find >> (and ignore) bad byte sequences. �I''ve tried to find recipes for both >> the above, but quickly get lost in the subtleties of it all! �Any and >> all help appreciated. �Many thanks in advance. > > I''d check with the whoever admins the MySQL DB to find out what > character set it''s actually using. I think you can then tell the > adapter to translate. Best guess is either US ASCII, or (more likely) > Windows-1252 pretending to be ASCII. > > --Matt JonesMany thanks for the reply Matt. I used the console to determine that the db is serving up ASCII-8BIT>>e = Equipment.find(:first, :conditions => [''id = ?'', 1234]) >> e.name.encoding=> #<Encoding:ASCII-8BIT> I then set the encoding in /config/database.yml to ''ascii'' which although it can''t display special characters, at least it shows the page with "?" in place of the accented charaters. I tried setting encoding to "ascii-8bit" and varieties of this, but each time Rails complained - so if anyone can tell me how to indicate ASCII-8BIT I''d be grateful. -- Posted via http://www.ruby-forum.com/.
Toby Rodwell wrote: [...]> Many thanks for the reply Matt. I used the console to determine that > the db is serving up ASCII-8BIT > >>>e = Equipment.find(:first, :conditions => [''id = ?'', 1234]) >>> e.name.encoding > => #<Encoding:ASCII-8BIT> > > I then set the encoding in /config/database.yml to ''ascii'' which > although it can''t display special characters, at least it shows the page > with "?" in place of the accented charaters. I tried setting encoding > to "ascii-8bit" and varieties of this, but each time Rails complained - > so if anyone can tell me how to indicate ASCII-8BIT I''d be grateful.This doesn''t solve your immediate problem, but...if your host locks the DB in ASCII 8-bit and you can''t change it, then find a new host. That encoding is inappropriate for real work. :) Best, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
On Nov 3, 10:09 pm, Marnen Laibow-Koser <rails-mailing-l...@andreas- s.net> wrote:> Toby Rodwell wrote: > > [...] > > > Many thanks for the reply Matt. I used the console to determine that > > the db is serving up ASCII-8BIT > > >>>e = Equipment.find(:first, :conditions => [''id = ?'', 1234]) > >>> e.name.encoding > > => #<Encoding:ASCII-8BIT> > > > I then set the encoding in /config/database.yml to ''ascii'' which > > although it can''t display special characters, at least it shows the page > > with "?" in place of the accented charaters. I tried setting encoding > > to "ascii-8bit" and varieties of this, but each time Rails complained - > > so if anyone can tell me how to indicate ASCII-8BIT I''d be grateful. > > This doesn''t solve your immediate problem, but...if your host locks the > DB in ASCII 8-bit and you can''t change it, then find a new host. That > encoding is inappropriate for real work. :) >This is not about the database itself this is to do with the interaction between the mysql driver and the new string encoding schemes - strings in ruby 1.9 are encoding aware and from what I gather the mysql driver creates strings with the ascii-8bit encoding regardless of their actual encoding (my very vague understanding is that ascii-8bit is sort of pseudo encoding that doesn''t actually mean ascii - it just means raw bytes) There is quite a lot of discussion on lighthouse here: https://rails.lighthouseapp.com/projects/8994/tickets/2476-ascii-8bit-encoding-of-query-results-in-rails-232-and-ruby-191#ticket-2476-2 although no clear resolution that I could see. May provide some help to Toby. One way out would be to fall back to ruby 1.8.x, where these problems do not exist because strings are just dumb collections of bytes. Fred> Best, > -- > Marnen Laibow-Koserhttp://www.marnen.org > mar...-sbuyVjPbboAdnm+yROfE0A@public.gmane.org > -- > Posted viahttp://www.ruby-forum.com/.
Frederick Cheung wrote: [...]> This is not about the database itself this is to do with the > interaction between the mysql driver and the new string encoding > schemes - strings in ruby 1.9 are encoding aware and from what I > gather the mysql driver creates strings with the ascii-8bit encoding > regardless of their actual encodingOh, I didn''t know that since I don''t use 1.9 yet.> FredBest, -- Marnen Laibow-Koser http://www.marnen.org marnen-sbuyVjPbboAdnm+yROfE0A@public.gmane.org -- Posted via http://www.ruby-forum.com/.
Seemingly Similar Threads
- has_many through , or habtm , using form
- Authlogic Password confirmation is too short Error. NEED HELP.
- acts_as_list / acts_as_tree / acts_as_nested_set - which one
- <b> tag in HTML 5 (was Re: Re: assert_select for <p><b>text</b>value</p>)
- default value if textfield is empty