I have my database entirely in utf8. When I set my document''s content type to utf-8, the DB content comes out with the wrong encoding. When I don''t specify an encoding, the DB content comes out fine but my localized text comes out funky. I''ve checked and the encoding is all utf-8. What might be going on? ___________________ Ben Jackson Diretor de Desenvolvimento ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org http://www.incomumdesign.com
Hi Ben,
While I personally haven''t setup unicode on a rails app, there is
some info about possible needed steps in the wiki. I suppose you have
had a look at this wiki article?
http://wiki.rubyonrails.com/rails/show/HowToUseUnicodeStrings
Best,
Tim
On 6/1/05, Ben Jackson
<ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org>
wrote:> I have my database entirely in utf8. When I set my document''s
content
> type to utf-8, the DB content comes out with the wrong encoding. When I
> don''t specify an encoding, the DB content comes out fine but my
> localized text comes out funky. I''ve checked and the encoding is
all
> utf-8. What might be going on?
> ___________________
> Ben Jackson
> Diretor de Desenvolvimento
>
> ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org
> http://www.incomumdesign.com
>
> _______________________________________________
> Rails mailing list
> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
> http://lists.rubyonrails.org/mailman/listinfo/rails
>
Have you configured your database connection to let it know that you are sending data in UTF-8? If not, the database may be helpfully trying to convert your already encoded data for you. We are using MySQL 4.1 as our data store with all data in UTF-8. Our environment.rb file contains the following line to set the database client encoding: ActiveRecord::Base.connection.update "SET NAMES ''utf8''" There may be a better way of doing this, but this has been working for us. -Bob. On Jun 1, 2005, at 4:21 PM, Ben Jackson wrote:> I have my database entirely in utf8. When I set my document''s > content type to utf-8, the DB content comes out with the wrong > encoding. When I don''t specify an encoding, the DB content comes > out fine but my localized text comes out funky. I''ve checked and > the encoding is all utf-8. What might be going on? > ___________________ > Ben Jackson > Diretor de Desenvolvimento > > ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org > http://www.incomumdesign.com > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
I added the appropriate line in my environment. Also altered the app controller like in the article mentioned in the previous post. Still no luck. Any ideas? - Ben On 1 Jun 2005, at 20:51, Robert Cottrell wrote:> Have you configured your database connection to let it know that you > are sending data in UTF-8? If not, the database may be helpfully > trying to convert your already encoded data for you. > > We are using MySQL 4.1 as our data store with all data in UTF-8. Our > environment.rb file contains the following line to set the database > client encoding: > > ActiveRecord::Base.connection.update "SET NAMES ''utf8''" > > There may be a better way of doing this, but this has been working for > us. > > -Bob. > > > On Jun 1, 2005, at 4:21 PM, Ben Jackson wrote: > >> I have my database entirely in utf8. When I set my document''s content >> type to utf-8, the DB content comes out with the wrong encoding. When >> I don''t specify an encoding, the DB content comes out fine but my >> localized text comes out funky. I''ve checked and the encoding is all >> utf-8. What might be going on? >> ___________________ >> Ben Jackson >> Diretor de Desenvolvimento >> >> ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org >> http://www.incomumdesign.com >> >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Ben Jackson wrote:> I added the appropriate line in my environment. Also altered the app > controller like in the article mentioned in the previous post. Still no > luck. Any ideas?Have you tried sending the correctz headers? Setting the doctype in the html document alone is not enough. Look at the 3rd listing in my article here: http://manuals.rubyonrails.com/read/chapter/88#page219 the lines: @charset = ''utf-8'' @headers["Content-Type"] = "text/html; charset=#{@charset}" Sascha
I have set the headers in the application controller and the issue persists. What''s the next step? - B On 2 Jun 2005, at 08:29, Sascha Ebach wrote:> Ben Jackson wrote: >> I added the appropriate line in my environment. Also altered the app >> controller like in the article mentioned in the previous post. Still >> no luck. Any ideas? > > Have you tried sending the correctz headers? Setting the doctype in > the html document alone is not enough. Look at the 3rd listing in my > article here: > > http://manuals.rubyonrails.com/read/chapter/88#page219 > > the lines: > > @charset = ''utf-8'' > @headers["Content-Type"] = "text/html; charset=#{@charset}" > > > Sascha > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
On 6/2/05, Ben Jackson <ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org> wrote:> I have set the headers in the application controller and the issue > persists. What''s the next step? > - BDid you try changing the shebang on appropriate files (dispatch.fcgi, script/server, etc) #!/usr/local/bin/ruby -Ku Jason> On 2 Jun 2005, at 08:29, Sascha Ebach wrote: > > > Ben Jackson wrote: > >> I added the appropriate line in my environment. Also altered the app > >> controller like in the article mentioned in the previous post. Still > >> no luck. Any ideas? > > > > Have you tried sending the correctz headers? Setting the doctype in > > the html document alone is not enough. Look at the 3rd listing in my > > article here: > > > > http://manuals.rubyonrails.com/read/chapter/88#page219 > > > > the lines: > > > > @charset = ''utf-8'' > > @headers["Content-Type"] = "text/html; charset=#{@charset}" > > > > > > Sascha > > _______________________________________________ > > Rails mailing list > > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > > http://lists.rubyonrails.org/mailman/listinfo/rails > > > > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
Just tried that and it got no results. I changed all three dispatch scripts and script/server. - B On 2 Jun 2005, at 10:12, Jason Foreman wrote:> On 6/2/05, Ben Jackson <ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org> wrote: >> I have set the headers in the application controller and the issue >> persists. What''s the next step? >> - B > > Did you try changing the shebang on appropriate files (dispatch.fcgi, > script/server, etc) > > #!/usr/local/bin/ruby -Ku > > > Jason > > >> On 2 Jun 2005, at 08:29, Sascha Ebach wrote: >> >>> Ben Jackson wrote: >>>> I added the appropriate line in my environment. Also altered the app >>>> controller like in the article mentioned in the previous post. Still >>>> no luck. Any ideas? >>> >>> Have you tried sending the correctz headers? Setting the doctype in >>> the html document alone is not enough. Look at the 3rd listing in my >>> article here: >>> >>> http://manuals.rubyonrails.com/read/chapter/88#page219 >>> >>> the lines: >>> >>> @charset = ''utf-8'' >>> @headers["Content-Type"] = "text/html; charset=#{@charset}" >>> >>> >>> Sascha >>> _______________________________________________ >>> Rails mailing list >>> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >>> http://lists.rubyonrails.org/mailman/listinfo/rails >>> >> >> _______________________________________________ >> Rails mailing list >> Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org >> http://lists.rubyonrails.org/mailman/listinfo/rails >> > _______________________________________________ > Rails mailing list > Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org > http://lists.rubyonrails.org/mailman/listinfo/rails >
http://wiki.rubyonrails.com/rails/show/Internationalization> Fix up your mysql_adapter.rb to use this instead of the regular > Mysql::real_connect (line 33): > > conn = Mysql::real_connect(host, username, password, database, port, > socket) > conn.query("SET NAMES ''utf8'';") > > ConnectionAdapters::MysqlAdapter.new(conn, logger) > > SET NAMES ‘utf8’ is where the magic happens.To do this you need to check out the latest version of Rails: http://www.rubyonrails.org/show/EdgeRails And modify the file /vendor/rails/lib/active_record/connection_adapters/mysql_adapter.rb. In the latest checkout the line is 39, not 33. Thanks to all for your help in diagnosing this. Regards, Ben ___________________ Ben Jackson Diretor de Desenvolvimento ben-p14LI7ZcAE/pVLaUnt/cCQC/G2K4zDHf@public.gmane.org http://www.incomumdesign.comhttp://wiki.rubyonrails.com/rails/show/ Internationalization
Hello, Robert Cottrell <bob-9FctsW8Z9av3oGB3hsPCZA@public.gmane.org> wrote:> We are using MySQL 4.1 as our data store with all data in UTF-8. Our > environment.rb file contains the following line to set the database > client encoding: > > ActiveRecord::Base.connection.update "SET NAMES ''utf8''"I think that SET_CHARSET_NAME is better solution. But I need to use MySQL 4.1 and MySQL/Ruby (Ruby/MySQL not support SET_CHARSET_NAME). Especially, 0x5C (\) might be included in the second byte in Shift_JIS character set strings. Please note the difference between Mysql.quote and MySQL#quote. I''m not worry about SQL injection but syntax error occurs. def self.mysql_connection(config) # :nodoc: ... mysql = Mysql.init begin mysql.options(Mysql::SET_CHARSET_NAME, "sjis") rescue RuntimeError # end def quote_string(s) @connection.quote(s) end ---- Kazuhiro Yoshida moriq-g2rar0VqK2oAvxtiuMwx3w@public.gmane.org
On Jun 2, 2005, at 10:27 AM, Ben Jackson wrote:> http://wiki.rubyonrails.com/rails/show/Internationalization > > >> Fix up your mysql_adapter.rb to use this instead of the regular >> Mysql::real_connect (line 33): >> >> conn = Mysql::real_connect(host, username, password, database, >> port, socket) >> conn.query("SET NAMES ''utf8'';") >> >> ConnectionAdapters::MysqlAdapter.new(conn, logger) >> >> SET NAMES ‘utf8’ is where the magic happens. >> > > To do this you need to check out the latest version of Rails: > http://www.rubyonrails.org/show/EdgeRails > > And modify the file /vendor/rails/lib/active_record/ > connection_adapters/mysql_adapter.rb. In the latest checkout the > line is 39, not 33.I can''t use EdgeRails - I need to put this production. Is there a reason why EdgeRails is required? Any way to fix this on Gems Rails? Thanks, Brett _______________________________________________ Rails mailing list Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org http://lists.rubyonrails.org/mailman/listinfo/rails
Hi, At 06/03/05 20:30:59 (JST), Kazuhiro Yoshida <moriq-g2rar0VqK2oAvxtiuMwx3w@public.gmane.org> wrote:> I think that SET_CHARSET_NAME is better solution. > But I need to use MySQL 4.1 and MySQL/Ruby > (Ruby/MySQL not support SET_CHARSET_NAME).I submited a patch for this issue to trac. http://dev.rubyonrails.org/ticket/2975 Shugo
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Nov 21, 2005, at 9:43 AM, Shugo Maeda wrote:> At 06/03/05 20:30:59 (JST), > Kazuhiro Yoshida <moriq-g2rar0VqK2oAvxtiuMwx3w@public.gmane.org> wrote: >> I think that SET_CHARSET_NAME is better solution. >> But I need to use MySQL 4.1 and MySQL/Ruby >> (Ruby/MySQL not support SET_CHARSET_NAME). > > I submited a patch for this issue to trac. > http://dev.rubyonrails.org/ticket/2975Thank you; applied to trunk for 1.1 release. jeremy -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2 (Darwin) iD8DBQFDghliAQHALep9HFYRAq7eAKCzVcVuxLQXYCMSjrFcui27nM8qeACfY1PJ tkgJNoFF715q1xm6TNnG184=F1LR -----END PGP SIGNATURE-----