R devel - Apr 2020 - Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Hi

The R branch ...

https://svn.r-project.org/R/branches/R-symfam/

... is now set up so that it works "out of the box" on Fedora by
setting
the default to be 'symbolfamily=cairoSymbolFont(family, usePUA=FALSE)' 
when grSoftVersion()["pango"] is greater than "1.44".

This means that on Fedora 31 (at least on the Docker container I am 
testing on) "sans"->"NimubusSans" is used as the symbol
font by default
and R converts Adobe Symbol Encoding code points to non-PUA UTF8 code 
points.  This is not the prettiest result, but it is a lot better than 
the page full of missing glyphs that we had.

The default on less "bleeding edge" systems, e.g., my Ubuntu 18.04, 
remains 'symbolfamily="Symbol"'.

The default on other platforms is supposed to be the same as it was, but 
I need help to confirm that.  I have set up a github repo ...

https://github.com/pmur002/R-symfam-testing

... that describes how to test this on macOS and Windows if anyone has 
time to do so.

I will start trying to set up a Windows test unless someone beats me to it.

Paul

On 30/03/20 3:24 pm, Paul Murrell wrote:
> Hi
> 
> I have created an R branch that contains a potential fix ...
> 
> https://svn.r-project.org/R/branches/R-symfam/
> 
> This allows, for example, ...
> 
> cairo_pdf(symbolfamily="OpenSymbol")
> 
> ... to specify that the OpenSymbol family should be used as the
"symbol"
> font (e.g., for "plotmath") in R.
> 
> This is just a separate branch for now because, while I have tested it 
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows 
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at 
> this stage of the release process for R 4.0.0.
> 
> The attached file contains at least an outline of steps required to do a 
> minimal test if anyone wants to try the fix on Linux.
> 
> cc'ing Simon and Jeroen in case they are able to help with checking
that
> this builds and works on Mac and/or Windows.
> 
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is
opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
> 
> Paul
> 
> On 30/03/20 8:15 am, Paul Murrell wrote:
>> Hi
>>
>> Thanks for your input on this I?aki and Nicolas.
>>
>> I am starting testing an R fix for this problem today.
>>
>> As suggested, the plan is to allow the R user to specify a font family 
>> other than "symbol" for plotmath output (or, more generally,
in R
>> parlance, for 'font=5' or 'fontface=5') on a
Cairo-based graphics device.
>>
>> Paul
>>
>>
>> On 27/03/20 11:30 pm, I?aki Ucar wrote:
>>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>>> <nicolas.mailhot at laposte.net> wrote:
>>>>
>>>> <snip>
>>>>
>>>> R brought this all on itself by hardcoding a Windows-only
?Symbol? font
>>>> family name in its default conf. Linux systems are UTF-8 by
default for
>>>> ~20 years now, they don?t need the forcing of magic font
families to
>>>> handle symbols not present in the 8-bit legacy Windows
encodings.
>>>>
>>>> The actual effect of this conf is not the selection of font
files with
>>>> special and unusual symbols. It is to priorize fonts that match
the
>>>> "Symbol" magic name. And those fonts are few and
crumbling on Linux
>>>> systems, because no one has needed to bother with them since
Linux
>>>> switched to UTF-8 last millenium.
>>>>
>>>> Just stop using ?Symbol? in R and things will work a lot
better.
>>>> Alternatively, prepare to maintain the ?Symbol? aliasing stack
in
>>>> fontconfig (and fight with wine for it), because *no* *one*
*else*
>>>> *cares* about this legacy Windows-specific stuff.
>>>
>>> So, in the light of Nicolas' input (thanks!), I think that font
>>> selection should be fixed upstream in R. I'd be happy to put
all this
>>> together in R's bugzilla, but I don't have an account.
Could someone
>>> please invite me?
>>>
>>> I?aki
>>>
>>> ______________________________________________
>>> R-devel at r-project.org mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
> 
-- 
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
paul at stat.auckland.ac.nz
http://www.stat.auckland.ac.nz/~paul/

On Mon, 6 Apr 2020 at 04:59, Paul Murrell <paul at stat.auckland.ac.nz>
wrote:
>
> Hi
>
> The R branch ...
>
> https://svn.r-project.org/R/branches/R-symfam/
>
> ... is now set up so that it works "out of the box" on Fedora by
setting
> the default to be 'symbolfamily=cairoSymbolFont(family,
usePUA=FALSE)'
> when grSoftVersion()["pango"] is greater than "1.44".
That is awesome, thanks! Will you port this to the R-3-6-branch?

-- 
I?aki ?car

On 6/04/20 8:21 pm, I?aki Ucar wrote:
> On Mon, 6 Apr 2020 at 04:59, Paul Murrell <paul at
stat.auckland.ac.nz> wrote:
>>
>> Hi
>>
>> The R branch ...
>>
>> https://svn.r-project.org/R/branches/R-symfam/
>>
>> ... is now set up so that it works "out of the box" on Fedora
by setting
>> the default to be 'symbolfamily=cairoSymbolFont(family,
usePUA=FALSE)'
>> when grSoftVersion()["pango"] is greater than
"1.44".
> 
> That is awesome, thanks! Will you port this to the R-3-6-branch?
I'll be very pleased if I can get this merged in time for R 4.0.0.

I am not sure if there will be any further patch releases in the R 3.6 
series.  That would require wider discussion within R-core.
Paul
-- 
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
paul at stat.auckland.ac.nz
http://www.stat.auckland.ac.nz/~paul/

The R-symfam branch (r78176) is now working, for my basic tests, on ...

Ubuntu (pango < 1.44)
Ubuntu (no pango)
Fedora (pango > 1.44)
Windows

I need help to confirm that this builds on macOS and that the basic 
tests work ...

https://github.com/pmur002/R-symfam-testing

Brian has been helping with the build, but I am still looking for 
someone who can run the tests please.  Happy to be fed PDF files to 
scrutinize myself;  it's generating the PDF files on macOS that I need 
help with.

Paul

On 6/04/20 2:59 pm, Paul Murrell wrote:
> Hi
> 
> The R branch ...
> 
> https://svn.r-project.org/R/branches/R-symfam/
> 
> ... is now set up so that it works "out of the box" on Fedora by
setting
> the default to be 'symbolfamily=cairoSymbolFont(family,
usePUA=FALSE)'
> when grSoftVersion()["pango"] is greater than "1.44".
> 
> This means that on Fedora 31 (at least on the Docker container I am 
> testing on) "sans"->"NimubusSans" is used as the
symbol font by default
> and R converts Adobe Symbol Encoding code points to non-PUA UTF8 code 
> points.? This is not the prettiest result, but it is a lot better than 
> the page full of missing glyphs that we had.
> 
> The default on less "bleeding edge" systems, e.g., my Ubuntu
18.04,
> remains 'symbolfamily="Symbol"'.
> 
> The default on other platforms is supposed to be the same as it was, but 
> I need help to confirm that.? I have set up a github repo ...
> 
> https://github.com/pmur002/R-symfam-testing
> 
> ... that describes how to test this on macOS and Windows if anyone has 
> time to do so.
> 
> I will start trying to set up a Windows test unless someone beats me to it.
> 
> Paul
> 
> On 30/03/20 3:24 pm, Paul Murrell wrote:
>> Hi
>>
>> I have created an R branch that contains a potential fix ...
>>
>> https://svn.r-project.org/R/branches/R-symfam/
>>
>> This allows, for example, ...
>>
>> cairo_pdf(symbolfamily="OpenSymbol")
>>
>> ... to specify that the OpenSymbol family should be used as the 
>> "symbol" font (e.g., for "plotmath") in R.
>>
>> This is just a separate branch for now because, while I have tested it 
>> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows 
>> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel 
>> at this stage of the release process for R 4.0.0.
>>
>> The attached file contains at least an outline of steps required to do 
>> a minimal test if anyone wants to try the fix on Linux.
>>
>> cc'ing Simon and Jeroen in case they are able to help with checking
>> that this builds and works on Mac and/or Windows.
>>
>> NOTEs:
>> - 'symbolfamily' can only be specified when a graphics device
is
>> opened, and it is then fixed for that device.
>> - on Windows, for cairo-based devices, the "symbol" font is
still
>> hard-coded as "Standard Symbols L"
>>
>> Paul
>>
>> On 30/03/20 8:15 am, Paul Murrell wrote:
>>> Hi
>>>
>>> Thanks for your input on this I?aki and Nicolas.
>>>
>>> I am starting testing an R fix for this problem today.
>>>
>>> As suggested, the plan is to allow the R user to specify a font 
>>> family other than "symbol" for plotmath output (or, more
generally,
>>> in R parlance, for 'font=5' or 'fontface=5') on a
Cairo-based
>>> graphics device.
>>>
>>> Paul
>>>
>>>
>>> On 27/03/20 11:30 pm, I?aki Ucar wrote:
>>>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>>>> <nicolas.mailhot at laposte.net> wrote:
>>>>>
>>>>> <snip>
>>>>>
>>>>> R brought this all on itself by hardcoding a Windows-only
?Symbol?
>>>>> font
>>>>> family name in its default conf. Linux systems are UTF-8 by
default
>>>>> for
>>>>> ~20 years now, they don?t need the forcing of magic font
families to
>>>>> handle symbols not present in the 8-bit legacy Windows
encodings.
>>>>>
>>>>> The actual effect of this conf is not the selection of font
files with
>>>>> special and unusual symbols. It is to priorize fonts that
match the
>>>>> "Symbol" magic name. And those fonts are few and
crumbling on Linux
>>>>> systems, because no one has needed to bother with them
since Linux
>>>>> switched to UTF-8 last millenium.
>>>>>
>>>>> Just stop using ?Symbol? in R and things will work a lot
better.
>>>>> Alternatively, prepare to maintain the ?Symbol? aliasing
stack in
>>>>> fontconfig (and fight with wine for it), because *no* *one*
*else*
>>>>> *cares* about this legacy Windows-specific stuff.
>>>>
>>>> So, in the light of Nicolas' input (thanks!), I think that
font
>>>> selection should be fixed upstream in R. I'd be happy to
put all this
>>>> together in R's bugzilla, but I don't have an account.
Could someone
>>>> please invite me?
>>>>
>>>> I?aki
>>>>
>>>> ______________________________________________
>>>> R-devel at r-project.org mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>
> 
-- 
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
paul at stat.auckland.ac.nz
http://www.stat.auckland.ac.nz/~paul/

Paul et al,

I will try to do this tonight or tomorrow, though it will not be built with
th system tools because I have yet to get that tto work locally (spent a
good chunk of this morning trying).

I will send a separate messaage regarding those difficulties as well so
that we can at least confirm that they are due to a malconfiguration on my
part.

Best,
~G

On Tue, Apr 7, 2020 at 7:25 PM Paul Murrell <paul at stat.auckland.ac.nz>
wrote:
>
> The R-symfam branch (r78176) is now working, for my basic tests, on ...
>
> Ubuntu (pango < 1.44)
> Ubuntu (no pango)
> Fedora (pango > 1.44)
> Windows
>
> I need help to confirm that this builds on macOS and that the basic
> tests work ...
>
> https://github.com/pmur002/R-symfam-testing
>
> Brian has been helping with the build, but I am still looking for
> someone who can run the tests please.  Happy to be fed PDF files to
> scrutinize myself;  it's generating the PDF files on macOS that I need
> help with.
>
> Paul
>
> On 6/04/20 2:59 pm, Paul Murrell wrote:
> > Hi
> >
> > The R branch ...
> >
> > https://svn.r-project.org/R/branches/R-symfam/
> >
> > ... is now set up so that it works "out of the box" on
Fedora by setting
> > the default to be 'symbolfamily=cairoSymbolFont(family,
usePUA=FALSE)'
> > when grSoftVersion()["pango"] is greater than
"1.44".
> >
> > This means that on Fedora 31 (at least on the Docker container I am
> > testing on) "sans"->"NimubusSans" is used as
the symbol font by default
> > and R converts Adobe Symbol Encoding code points to non-PUA UTF8 code
> > points.  This is not the prettiest result, but it is a lot better than
> > the page full of missing glyphs that we had.
> >
> > The default on less "bleeding edge" systems, e.g., my Ubuntu
18.04,
> > remains 'symbolfamily="Symbol"'.
> >
> > The default on other platforms is supposed to be the same as it was,
but
> > I need help to confirm that.  I have set up a github repo ...
> >
> > https://github.com/pmur002/R-symfam-testing
> >
> > ... that describes how to test this on macOS and Windows if anyone has
> > time to do so.
> >
> > I will start trying to set up a Windows test unless someone beats me
to
> it.
> >
> > Paul
> >
> > On 30/03/20 3:24 pm, Paul Murrell wrote:
> >> Hi
> >>
> >> I have created an R branch that contains a potential fix ...
> >>
> >> https://svn.r-project.org/R/branches/R-symfam/
> >>
> >> This allows, for example, ...
> >>
> >> cairo_pdf(symbolfamily="OpenSymbol")
> >>
> >> ... to specify that the OpenSymbol family should be used as the
> >> "symbol" font (e.g., for "plotmath") in R.
> >>
> >> This is just a separate branch for now because, while I have
tested it
> >> under Unbuntu 18.04 and Fedora 31, I cannot even build R for
Windows
> >> (right now) or Mac (ever) and I do not want to drop a bomb on
R-devel
> >> at this stage of the release process for R 4.0.0.
> >>
> >> The attached file contains at least an outline of steps required
to do
> >> a minimal test if anyone wants to try the fix on Linux.
> >>
> >> cc'ing Simon and Jeroen in case they are able to help with
checking
> >> that this builds and works on Mac and/or Windows.
> >>
> >> NOTEs:
> >> - 'symbolfamily' can only be specified when a graphics
device is
> >> opened, and it is then fixed for that device.
> >> - on Windows, for cairo-based devices, the "symbol" font
is still
> >> hard-coded as "Standard Symbols L"
> >>
> >> Paul
> >>
> >> On 30/03/20 8:15 am, Paul Murrell wrote:
> >>> Hi
> >>>
> >>> Thanks for your input on this I?aki and Nicolas.
> >>>
> >>> I am starting testing an R fix for this problem today.
> >>>
> >>> As suggested, the plan is to allow the R user to specify a
font
> >>> family other than "symbol" for plotmath output (or,
more generally,
> >>> in R parlance, for 'font=5' or 'fontface=5')
on a Cairo-based
> >>> graphics device.
> >>>
> >>> Paul
> >>>
> >>>
> >>> On 27/03/20 11:30 pm, I?aki Ucar wrote:
> >>>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >>>> <nicolas.mailhot at laposte.net> wrote:
> >>>>>
> >>>>> <snip>
> >>>>>
> >>>>> R brought this all on itself by hardcoding a
Windows-only ?Symbol?
> >>>>> font
> >>>>> family name in its default conf. Linux systems are
UTF-8 by default
> >>>>> for
> >>>>> ~20 years now, they don?t need the forcing of magic
font families to
> >>>>> handle symbols not present in the 8-bit legacy Windows
encodings.
> >>>>>
> >>>>> The actual effect of this conf is not the selection of
font files
> with
> >>>>> special and unusual symbols. It is to priorize fonts
that match the
> >>>>> "Symbol" magic name. And those fonts are few
and crumbling on Linux
> >>>>> systems, because no one has needed to bother with them
since Linux
> >>>>> switched to UTF-8 last millenium.
> >>>>>
> >>>>> Just stop using ?Symbol? in R and things will work a
lot better.
> >>>>> Alternatively, prepare to maintain the ?Symbol?
aliasing stack in
> >>>>> fontconfig (and fight with wine for it), because *no*
*one* *else*
> >>>>> *cares* about this legacy Windows-specific stuff.
> >>>>
> >>>> So, in the light of Nicolas' input (thanks!), I think
that font
> >>>> selection should be fixed upstream in R. I'd be happy
to put all this
> >>>> together in R's bugzilla, but I don't have an
account. Could someone
> >>>> please invite me?
> >>>>
> >>>> I?aki
> >>>>
> >>>> ______________________________________________
> >>>> R-devel at r-project.org mailing list
> >>>> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>>>
> >>
> >
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> paul at stat.auckland.ac.nz
> http://www.stat.auckland.ac.nz/~paul/
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]