Hi All, I have a ruby script which picks up data from a text file and then adds the corresponding data into my postgresql database. However, it not picking up "diacritic" i.e. characters like "Á, é, ó" etc and throwing the following error. RuntimeError: ERROR C22021 Minvalid byte sequence for encoding "UTF8": 0xe9204d HThis error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding". Fwchar.c L1534 Rreport_invalid_encoding: SELECT * FROM "contacts" WHERE (preferred_given_names = E''ZahidT Miguel'' and surname = E''Baeza Tordesillas'' and (email_work = E''zahid-i8C1Mwm+lRU3uPMLIKxrzw@public.gmane.org'' OR email_work IS NULL)) LIMIT 1undefined local variable or method `some_string'' for #<DataImporter:0x55f0cbc @file="d:\\CGCtest.txt"> I''ve created the database with UTF-8 encoding. In fact I can manually add spanish characters directly into the database, however, not through the script and I want the script to do this for me. Any help would be much appreciated. Thanks, Zahid -- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
On Feb 22, 9:30 am, Zahid Karnekar <li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote:> Hi All, > > I have a ruby script which picks up data from a text file and then adds > the corresponding data into my postgresql database. However, it not > picking up "diacritic" i.e. characters like "Á, é, ó" etc and throwing > the following error. > > RuntimeError: ERROR C22021 Minvalid byte sequence for encoding > "UTF8": 0xe9204d HThis error can also happen if the byte sequence > does not match the encoding expected by the server, which is controlled > by "client_encoding". Fwchar.c L1534 Rreport_invalid_encoding: > SELECT * FROM "contacts" WHERE (preferred_given_names = E''ZahidT Miguel'' > and surname = E''Baeza Tordesillas'' and (email_work = E''za...@latech.edu'' > OR email_work IS NULL)) LIMIT 1undefined local variable or method > `some_string'' for #<DataImporter:0x55f0cbc @file="d:\\CGCtest.txt"> > > I''ve created the database with UTF-8 encoding. In fact I can manually > add spanish characters directly into the database, however, not through > the script and I want the script to do this for me. Any help would be > much appreciated. >What about the input file you are reading? Fred> Thanks, > Zahid > > -- > Posted viahttp://www.ruby-forum.com/.-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Hi Zahid, Make sure your database.yml file knows to talk to the database in UTF-8. Add the following to you database defined in there: encoding: utf8 That will ensure that the communication is in UTF-8. Now, Fredrick makes a good point. The input file you are reading from should also be in UTF-8 format and be displaying the characters properly. Double check that as well. Thanks, B. On Tue, Feb 22, 2011 at 3:48 AM, Frederick Cheung <frederick.cheung-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:> > > On Feb 22, 9:30 am, Zahid Karnekar <li...-fsXkhYbjdPsEEoCn2XhGlw@public.gmane.org> wrote: >> Hi All, >> >> I have a ruby script which picks up data from a text file and then adds >> the corresponding data into my postgresql database. However, it not >> picking up "diacritic" i.e. characters like "Á, é, ó" etc and throwing >> the following error. >> >> RuntimeError: ERROR C22021 Minvalid byte sequence for encoding >> "UTF8": 0xe9204d HThis error can also happen if the byte sequence >> does not match the encoding expected by the server, which is controlled >> by "client_encoding". Fwchar.c L1534 Rreport_invalid_encoding: >> SELECT * FROM "contacts" WHERE (preferred_given_names = E''ZahidT Miguel'' >> and surname = E''Baeza Tordesillas'' and (email_work = E''za...@latech.edu'' >> OR email_work IS NULL)) LIMIT 1undefined local variable or method >> `some_string'' for #<DataImporter:0x55f0cbc @file="d:\\CGCtest.txt"> >> >> I''ve created the database with UTF-8 encoding. In fact I can manually >> add spanish characters directly into the database, however, not through >> the script and I want the script to do this for me. Any help would be >> much appreciated. >> > > What about the input file you are reading? > > Fred >> Thanks, >> Zahid >> >> -- >> Posted viahttp://www.ruby-forum.com/. > > -- > You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. > To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org > To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. > For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en. > >-- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe@googlegroups.com. For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.
Hi Fred & Bryan, Thank you so very much for looking into my query and helping me resolve it. Your hint did the trick. The input file had an ANSI encoding instead of UTF-8. I just changed it and it worked. God bless you both :-) Thanks, Zahid Bryan Crossland wrote in post #983175:> Hi Zahid, > > Make sure your database.yml file knows to talk to the database in > UTF-8. Add the following to you database defined in there: > > encoding: utf8 > > That will ensure that the communication is in UTF-8. Now, Fredrick > makes a good point. The input file you are reading from should also be > in UTF-8 format and be displaying the characters properly. Double > check that as well. > > Thanks, > B. > > On Tue, Feb 22, 2011 at 3:48 AM, Frederick Cheung-- Posted via http://www.ruby-forum.com/. -- You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group. To post to this group, send email to rubyonrails-talk-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org To unsubscribe from this group, send email to rubyonrails-talk+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/Ez6ZCGd0@public.gmane.org For more options, visit this group at http://groups.google.com/group/rubyonrails-talk?hl=en.