R devel - Apr 2006 - UTF8 letters are imaged incorrectly (PR#8770)

Full_Name: Mikhail Bocharov
Version: 2.2.1
OS: Linux
Submission from: (NULL) (62.148.130.28)


Cyrillic UTF8 letters are imaged incorrectly

Sorry, but English is not my native language

My operating system is Gentoo-linux. Now i want get the gparhics with russian
title
for example: plot(1:20, type='l',
main='Утечка')
instead of russian letters in title i have this -
'5'd'V'i'\'Q , but must be
'утечка'


my locale:

$locale
LANG=ru_RU.UTF-8
LC_CTYPE="ru_RU.UTF-8"
LC_NUMERIC="ru_RU.UTF-8"
LC_TIME="ru_RU.UTF-8"
LC_COLLATE="ru_RU.UTF-8"
LC_MONETARY="ru_RU.UTF-8"
LC_MESSAGES="ru_RU.UTF-8"
LC_PAPER="ru_RU.UTF-8"
LC_NAME="ru_RU.UTF-8"
LC_ADDRESS="ru_RU.UTF-8"
LC_TELEPHONE="ru_RU.UTF-8"
LC_MEASUREMENT="ru_RU.UTF-8"
LC_IDENTIFICATION="ru_RU.UTF-8"
LC_ALL
PS 'утечка'
in english 'leakаge'

I am sorry, but why do you think this is a bug in R, when it is your X11 
display (I presume, you did not say this was the x11() device) that is not 
rendering the characters correctly?

We have seen this with UTF-8 locales such as Japanese, and it was an OS 
services problem, in that exactly the same code works correctly on some 
versions of the OS and not others. (Earlier this week I ran the compiled 
code for R 2.1.1 which I know worked correctly on FC3 on a fully updated 
machine, and it no longer renders Japanese chars correctly in UTF-8.  The 
R code is unchanged since it was first compiled, but the xorg RPMs have 
changed and hence the code it links to.)

We have traced the X11 calls, and they are being called with the correct 
UTF-8 character string.  Some UTF-8 locales work and some do not, and R 
does not know any difference between UTF-8 locales.

There's a comment at the top of src/modules/X11/rotated.c that you might 
like to uncomment and see if it helps.  That uses UTF-8-specific X11 calls 
that when this was written (for R 2.1.0) did not work correctly in the 
then XFree distribution.

There is nothing we can do to help unless you or someone else in a Russian 
UTF-8 locale can tell us a solution.  It's currently a mystery, but as no 
else has reported it, it seems that for most people in UTF-8 locales it 
does work.

We are well into the beta-test period for R 2.3.0, so please test that to 
see if the problem persists.  (I suspect it will, but no more work will be 
done on R 2.2.x.)


On Fri, 14 Apr 2006, bocharov_m at inbox.ru wrote:
> Full_Name: Mikhail Bocharov
> Version: 2.2.1
> OS: Linux
> Submission from: (NULL) (62.148.130.28)
>
>
> Cyrillic UTF8 letters are imaged incorrectly
>
> Sorry, but English is not my native language
>
> My operating system is Gentoo-linux. Now i want get the gparhics with
russian
> title
> for example: plot(1:20, type='l',
>
main='Утечка')
> instead of russian letters in title i have this -
'5'd'V'i'\'Q , but must be
>
'утечка'
>
>
> my locale:
>
> $locale
> LANG=ru_RU.UTF-8
> LC_CTYPE="ru_RU.UTF-8"
> LC_NUMERIC="ru_RU.UTF-8"
> LC_TIME="ru_RU.UTF-8"
> LC_COLLATE="ru_RU.UTF-8"
> LC_MONETARY="ru_RU.UTF-8"
> LC_MESSAGES="ru_RU.UTF-8"
> LC_PAPER="ru_RU.UTF-8"
> LC_NAME="ru_RU.UTF-8"
> LC_ADDRESS="ru_RU.UTF-8"
> LC_TELEPHONE="ru_RU.UTF-8"
> LC_MEASUREMENT="ru_RU.UTF-8"
> LC_IDENTIFICATION="ru_RU.UTF-8"
> LC_ALL>
> PS
'утечка' in
english 'leakаge'
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

try.
 $ xlsfonts -fn '*-koi8-r'
or
 $ xlsfonts -fn '*-iso8859-5'
Is there a font of iso8859-5 and koi8?
Please install it in search of a font if nothing comes out.

Next should use xfontsel to choose an appropriate font.

 $ xfontsel  -pattern "-*-*-*-*-*--24-*-*-*-*-*-iso8859-5"

1.Let's choose fmly from a bar
2.If the family is beautiful, does wgth include medium and boldface?
3.And please choose fndry.

UTF-8 environment of X11 is another name of most case en_US.UTF-8.
In UTF8 environment of X11, as for the appointment of rgstry-encdng, a
font of iso10646 will be given priority to at reference value degree.
# look in XLC_LOCALE

Please try to the following if you can choose a font.
> options(X11fonts=c("-cronyx-helvetica-%s-%s-*--%d-*-*-*-*-*-*-*",
+                             
"-adobe-symbol-medium-r-*-*-%d-*-*-*-*-*-*-*"))
> plot(1:20,main="russian string.")
Kiril font is not included in adobe of a default.
xlsfonts -lll -fn -adobe-helvetica-bold-o-normal--10-100-75-75-p-60-iso10646-1
However, there is not glif of the letter.

2006/4/14, bocharov_m at inbox.ru <bocharov_m at
inbox.ru>:
> Full_Name: Mikhail Bocharov
> Version: 2.2.1
> OS: Linux
> Submission from: (NULL) (62.148.130.28)
>
>
> Cyrillic UTF8 letters are imaged incorrectly
>
> Sorry, but English is not my native language
>
> My operating system is Gentoo-linux. Now i want get the gparhics with
russian
> title
> for example: plot(1:20, type='l',
>
main='&#1059;&#1090;&#1077;&#1095;&#1082;&#1072;')
> instead of russian letters in title i have this -
'5'd'V'i'\'Q , but must be
>
'&#1091;&#1090;&#1077;&#1095;&#1082;&#1072;'
>
>
> my locale:
>
> $locale
> LANG=ru_RU.UTF-8
> LC_CTYPE="ru_RU.UTF-8"
> LC_NUMERIC="ru_RU.UTF-8"
> LC_TIME="ru_RU.UTF-8"
> LC_COLLATE="ru_RU.UTF-8"
> LC_MONETARY="ru_RU.UTF-8"
> LC_MESSAGES="ru_RU.UTF-8"
> LC_PAPER="ru_RU.UTF-8"
> LC_NAME="ru_RU.UTF-8"
> LC_ADDRESS="ru_RU.UTF-8"
> LC_TELEPHONE="ru_RU.UTF-8"
> LC_MEASUREMENT="ru_RU.UTF-8"
> LC_IDENTIFICATION="ru_RU.UTF-8"
> LC_ALL>
> PS
'&#1091;&#1090;&#1077;&#1095;&#1082;&#1072;' in
english 'leak&#1072;ge'
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

--
EI-JI Nakama  <nakama at ki.rim.or.jp>
"\u4e2d\u9593\u6804\u6cbb"  <nakama at ki.rim.or.jp>

If an error is not given by a locale command, I think that local
setting of glibc does not have any problem.
 I think that there is not problem if  add encoding of koi8-r to
XLC_LOCALE file.

I am not readable, but probably the following URL is useful.
http://ru.gentoo-wiki.com/HOWTO_ru_RU.utf8_Gentoo_way

2006/4/15, Mike Bocharov <bocharov_m at inbox.ru>:
> Thank you for you answer, but your example not work for me...
> > Please try to the following if you can choose a font.
> > >
options(X11fonts=c("-cronyx-helvetica-%s-%s-*--%d-*-*-*-*-*-*-*",
> > +
> > "-adobe-symbol-medium-r-*-*-%d-*-*-*-*-*-*-*"))
> > > plot(1:20,main="russian string.")
>
> I not specialist in locales and fonts, maybe reason in my locale?
> In my linux locale is UTF-8:
> LANG=ru_RU.UTF-8
> LC_CTYPE="ru_RU.UTF-8"
> LC_NUMERIC="ru_RU.UTF-8"
> LC_TIME="ru_RU.UTF-8"
> LC_COLLATE="ru_RU.UTF-8"
> LC_MONETARY="ru_RU.UTF-8"
> LC_MESSAGES="ru_RU.UTF-8"
> LC_PAPER="ru_RU.UTF-8"
> LC_NAME="ru_RU.UTF-8"
> LC_ADDRESS="ru_RU.UTF-8"
> LC_TELEPHONE="ru_RU.UTF-8"
> LC_MEASUREMENT="ru_RU.UTF-8"
> LC_IDENTIFICATION="ru_RU.UTF-8"
> LC_ALL>
> ,but  fonts in xfontsel is koi8-r(in rgstry)
>

--
EI-JI Nakama  <nakama at ki.rim.or.jp>
"\u4e2d\u9593\u6804\u6cbb"  <nakama at ki.rim.or.jp>

This was not a bug.

2006/4/15, Mike Bocharov <bocharov_m at inbox.ru>:
> It work, thanks for your help!
>
> >  If an error is not given by a locale command, I think that local
> > setting of glibc does not have any problem.
> >  I think that there is not problem if  add encoding of koi8-r to
> > XLC_LOCALE file.
> >
> > I am not readable, but probably the following URL is useful.
> > http://ru.gentoo-wiki.com/HOWTO_ru_RU.utf8_Gentoo_way
>

--
EI-JI Nakama  <nakama at ki.rim.or.jp>
"\u4e2d\u9593\u6804\u6cbb"  <nakama at ki.rim.or.jp>

Thanks to Ei-ji Nakama's hints, I was able to confirm that on FC3 Linux:

1) in locale ru_RU.koi8r  Russian letters are rendered correctly if I use
"-cronyx-helvetica-%s-%s-*--%d-*-*-*-*-*-*-*" as the first element of
option "X11fonts".

2) This does not work in locale ru_RU.utf8, and the reason is that there 
is no XLC_LOCALE locale directory.  Adding a suitably configured directory 
made this work.

3) There _is_ an XLC_LOCALE locale directory ja_JA.utf8, and I was able to 
get Japanese text to display by using

"-*-mincho-%s-%s-*--%d-*-*-*-*-*-*-*"  as the first element of option 
"X11fonts".

Systems which have suitable .ttf fonts installed may work with the default 
setting.

I've added some notes on 1) and 3) to ?X11.

Thanks Ei-ji!

Brian Ripley

On Sat, 15 Apr 2006, Ei-ji Nakama wrote:
> This was not a bug.
>
> 2006/4/15, Mike Bocharov <bocharov_m at inbox.ru>:
>> It work, thanks for your help!
>>
>>>  If an error is not given by a locale command, I think that local
>>> setting of glibc does not have any problem.
>>>  I think that there is not problem if  add encoding of koi8-r to
>>> XLC_LOCALE file.
>>>
>>> I am not readable, but probably the following URL is useful.
>>> http://ru.gentoo-wiki.com/HOWTO_ru_RU.utf8_Gentoo_way
>>
>
>
> --
> EI-JI Nakama  <nakama at ki.rim.or.jp>
> "\u4e2d\u9593\u6804\u6cbb"  <nakama at ki.rim.or.jp>
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595