R devel - Mar 2020 - [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Le mardi 31 mars 2020 ? 10:14 +1300, Paul Murrell a ?crit
:
> Hi
> 
> On 30/03/20 11:12 pm, Nicolas Mailhot wrote:
> > Le lundi 30 mars 2020 ? 15:24 +1300, Paul Murrell a ?crit :
> > > Hi
> > > 
> > > I have created an R branch that contains a potential fix ...
> > > 
> > > https://svn.r-project.org/R/branches/R-symfam/
> > > 
> > > This allows, for example, ...
> > > 
> > > cairo_pdf(symbolfamily="OpenSymbol")
> > > 
> > > ... to specify that the OpenSymbol family should be used as the
> > > "symbol" font (e.g., for "plotmath") in R.
> > 
> > Thanks for looking at it!
> > 
> > But, really, there is no such thing as a Symbol font on Linux
> > anymore.
> > Symbol is pre-unicode thinking. Most modern general-purpose unicode
> > fonts will include every codepoint Symbol ever shipped, and
> > fontconfig
> > will fallback gracefully when that?s not the case (unless your
> > fontconfig integration is broken).
> 
> Yep, the "symbol" font is an (outdated) R "plotmath"
concept, but
> one 
> that would take a fair bit of surgery to remove.  R plotmath
> converts 
> certain R expressions (in certain contexts) to code points in the
> Adobe 
> Symbol Encoding (ASM), but for cairo-based devices, those are
> converted 
> to UTF8 code points.
> 
> > Just use the sans-serif or monospace fontconfig defaults. You don?t
> > need Symbol, or OpenSymbol, or any special font setup.
> 
> Agreed.  I got reasonable coverage from DejaVu Sans and FreeSerif. 
> There are still a number of ASM code points that are not covered
> though, 
> for example, ...
> 
> F8EB	E6	# LEFT PAREN TOP	# parenlefttp (CUS)
> F8EC	E7	# LEFT PAREN EXTENDER	# parenleftex (CUS)
> F8ED	E8	# LEFT PAREN BOTTOM	# parenleftbt (CUS)
> 
> Even OpenSymbol is missing a few (though perhaps not very common
> ones) ...
All the F8* codepoints are in the private use area. That means you
can?t rely on them existing in standard unicode fonts

You need to use correct Unicode values for things to work:
Ux239? for parenthesis, brackets

https://www.unicode.org/charts/PDF/U2300.pdf
> F8E6    BD      # VERTICAL ARROW EXTENDER       # arrowvertex (CUS)
> F8E7    BE      # HORIZONTAL ARROW EXTENDER     # arrowhorizex (CUS)
and 23AF/23D0 for arrow extensions (though arrow font support seems
messy, probably because it sees little use; it?s a pity R comes so late
to the party, those are just lines, it would have been trivial to get
them into DejaVu before the project gone dormant). GFS NeoHellenic
(Math block) seems complete but it?s not a common font family.
> F6DA    D2      # REGISTERED SIGN SERIF # registerserif (CUS)
> F6D9    D3      # COPYRIGHT SIGN SERIF  # copyrightserif (CUS)
> F6DB    D4      # TRADE MARK SIGN SERIF # trademarkserif (CUS)
> F8E8    E2      # REGISTERED SIGN SANS SERIF    # registersans (CUS)
> F8E9    E3      # COPYRIGHT SIGN SANS SERIF     # copyrightsans (CUS)
> F8EA    E4      # TRADE MARK SIGN SANS SERIF    # trademarksans (CUS)
Those are useless nowadays, just use normal
registered/copyright/trademark codepoints, and a font in the wished
style (serif sans serif, whatever looks nice to you)

Regards

-- 
Nicolas Mailhot

On 31/03/20 11:50 am, Nicolas Mailhot wrote:
> Le mardi 31 mars 2020 ? 10:14 +1300, Paul Murrell a ?crit :
>> Hi
>>
>> On 30/03/20 11:12 pm, Nicolas Mailhot wrote:
>>> Le lundi 30 mars 2020 ? 15:24 +1300, Paul Murrell a ?crit :
>>>> Hi
>>>>
>>>> I have created an R branch that contains a potential fix ...
>>>>
>>>> https://svn.r-project.org/R/branches/R-symfam/
>>>>
>>>> This allows, for example, ...
>>>>
>>>> cairo_pdf(symbolfamily="OpenSymbol")
>>>>
>>>> ... to specify that the OpenSymbol family should be used as the
>>>> "symbol" font (e.g., for "plotmath") in R.
>>>
>>> Thanks for looking at it!
>>>
>>> But, really, there is no such thing as a Symbol font on Linux
>>> anymore.
>>> Symbol is pre-unicode thinking. Most modern general-purpose unicode
>>> fonts will include every codepoint Symbol ever shipped, and
>>> fontconfig
>>> will fallback gracefully when that?s not the case (unless your
>>> fontconfig integration is broken).
>>
>> Yep, the "symbol" font is an (outdated) R
"plotmath" concept, but
>> one
>> that would take a fair bit of surgery to remove.  R plotmath
>> converts
>> certain R expressions (in certain contexts) to code points in the
>> Adobe
>> Symbol Encoding (ASM), but for cairo-based devices, those are
>> converted
>> to UTF8 code points.
>>
>>> Just use the sans-serif or monospace fontconfig defaults. You don?t
>>> need Symbol, or OpenSymbol, or any special font setup.
>>
>> Agreed.  I got reasonable coverage from DejaVu Sans and FreeSerif.
>> There are still a number of ASM code points that are not covered
>> though,
>> for example, ...
>>
>> F8EB	E6	# LEFT PAREN TOP	# parenlefttp (CUS)
>> F8EC	E7	# LEFT PAREN EXTENDER	# parenleftex (CUS)
>> F8ED	E8	# LEFT PAREN BOTTOM	# parenleftbt (CUS)
>>
>> Even OpenSymbol is missing a few (though perhaps not very common
>> ones) ...
> 
> All the F8* codepoints are in the private use area. That means you
> can?t rely on them existing in standard unicode fonts
> 
> You need to use correct Unicode values for things to work:
> Ux239? for parenthesis, brackets
> 
> https://www.unicode.org/charts/PDF/U2300.pdf
> 
>> F8E6    BD      # VERTICAL ARROW EXTENDER       # arrowvertex (CUS)
>> F8E7    BE      # HORIZONTAL ARROW EXTENDER     # arrowhorizex (CUS)
> 
> and 23AF/23D0 for arrow extensions (though arrow font support seems
> messy, probably because it sees little use; it?s a pity R comes so late
> to the party, those are just lines, it would have been trivial to get
> them into DejaVu before the project gone dormant). GFS NeoHellenic
> (Math block) seems complete but it?s not a common font family.
> 
>> F6DA    D2      # REGISTERED SIGN SERIF # registerserif (CUS)
>> F6D9    D3      # COPYRIGHT SIGN SERIF  # copyrightserif (CUS)
>> F6DB    D4      # TRADE MARK SIGN SERIF # trademarkserif (CUS)
>> F8E8    E2      # REGISTERED SIGN SANS SERIF    # registersans (CUS)
>> F8E9    E3      # COPYRIGHT SIGN SANS SERIF     # copyrightsans (CUS)
>> F8EA    E4      # TRADE MARK SIGN SANS SERIF    # trademarksans (CUS)
> 
> Those are useless nowadays, just use normal
> registered/copyright/trademark codepoints, and a font in the wished
> style (serif sans serif, whatever looks nice to you)
> 
> Regards
Thanks, that's useful.  For my own memory, this is the parenthesis block 
that might be useful ...

U+239b Sm LEFT PARENTHESIS UPPER HOOK ?
U+239c Sm LEFT PARENTHESIS EXTENSION ?
U+239d Sm LEFT PARENTHESIS LOWER HOOK ?
U+239e Sm RIGHT PARENTHESIS UPPER HOOK ?
U+239f Sm RIGHT PARENTHESIS EXTENSION ?
U+23a0 Sm RIGHT PARENTHESIS LOWER HOOK ?
U+23a1 Sm LEFT SQUARE BRACKET UPPER CORNER ?
U+23a2 Sm LEFT SQUARE BRACKET EXTENSION ?
U+23a3 Sm LEFT SQUARE BRACKET LOWER CORNER ?
U+23a4 Sm RIGHT SQUARE BRACKET UPPER CORNER ?
U+23a5 Sm RIGHT SQUARE BRACKET EXTENSION ?
U+23a6 Sm RIGHT SQUARE BRACKET LOWER CORNER ?
U+23a7 Sm LEFT CURLY BRACKET UPPER HOOK ?

However, the situation is still not completely straightforward.  The 
style of the symbols is also an issue and the DejaVu symbols are not as 
elegant as, say, the OpenSymbol symbols.  What makes things tricky is 
that, AFAICS, DejaVu has (TTX Unicode cmap output) ...

<map code="0x239b" name="uni239B"/><!-- LEFT
PARENTHESIS UPPER HOOK -->

... while OpenSymbol has ...

<map code="0xf8eb" name="parenlefttp"/><!-- ????
-->

... but neither has the other.  So we could not simply switch to 
standard Unicode code points because, while that would work with the 
"ugly" DejaVu glyphs, that would mean that we could not access the 
"pretty" OpenSymbol glyphs.

Paul
-- 
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
paul at stat.auckland.ac.nz
http://www.stat.auckland.ac.nz/~paul/

Le mardi 31 mars 2020 ? 15:07 +1300, Paul Murrell a ?crit :
> 
> Thanks, that's useful.  For my own memory, this is the parenthesis
> block 
> that might be useful ...
> 
> U+239b Sm LEFT PARENTHESIS UPPER HOOK ?
> U+239c Sm LEFT PARENTHESIS EXTENSION ?
> U+239d Sm LEFT PARENTHESIS LOWER HOOK ?
> U+239e Sm RIGHT PARENTHESIS UPPER HOOK ?
> U+239f Sm RIGHT PARENTHESIS EXTENSION ?
> U+23a0 Sm RIGHT PARENTHESIS LOWER HOOK ?
> U+23a1 Sm LEFT SQUARE BRACKET UPPER CORNER ?
> U+23a2 Sm LEFT SQUARE BRACKET EXTENSION ?
> U+23a3 Sm LEFT SQUARE BRACKET LOWER CORNER ?
> U+23a4 Sm RIGHT SQUARE BRACKET UPPER CORNER ?
> U+23a5 Sm RIGHT SQUARE BRACKET EXTENSION ?
> U+23a6 Sm RIGHT SQUARE BRACKET LOWER CORNER ?
> U+23a7 Sm LEFT CURLY BRACKET UPPER HOOK ?
> 
> However, the situation is still not completely straightforward.  The 
> style of the symbols is also an issue and the DejaVu symbols are not
> as elegant as, say, the OpenSymbol symbols.  What makes things tricky
> is that, AFAICS, DejaVu has (TTX Unicode cmap output) ...
Ah, the endless design discussions? Myself I prefer a consistent design
like Dejavu, over cobbling symbols of different designs, because they
used to be in separate fonts. Anyway:
> <map code="0x239b" name="uni239B"/><!-- LEFT
PARENTHESIS UPPER HOOK
> -->
> 
> ... while OpenSymbol has ...
> 
> <map code="0xf8eb" name="parenlefttp"/><!--
???? -->
> 
> ... but neither has the other.  
OpenSymbol is incorrect (it suffers from the same pre-unicode bias as
R). However, it is, to my knowledge, actively maintained. You can ask
its upstream (LibreOffice) for Unicode conformance fixes if you find
problems. Especially when it?s just fixing the map of an existing
glyph, that should not be hard for them to fix. Anything PUA-related
won?t interoperate well in an unicode world.

(you can ask DejaVu too, maybe a request from a project like R will
wake up its maintainers. But, that?s a long shot. DejaVu suffers from
an almost done state without enough remaining work to interest
designers).

Regards,

-- 
Nicolas Mailhot