R devel - Sep 2023 - Help requested: writing text to a raster in memory

Duncan,

drawing text is one of the most complicated things you can do, so it really
depends how for you want to go. You can do it badly with a simple cairo
show_text API. The steps involved in doing it properly are detecting the
direction of the language, finding fonts, finding glyphs (resolving ligatures),
applying hints, drawing glyphs etc. Fortunately there are libraries that help
with than, but even then it's non-trivial. Probably the most modern pipeline
is icu + harfbuzz + freetype + fontconfig + cairo. This is implemented, e.g in
https://github.com/s-u/Cairo/blob/master/src/cairotalk.c (the meat is in  L608-)
and for all but the drawing part there is an entire R package (in C++) devoted
to this: https://github.com/r-lib/textshaping/tree/main/src -- Thomas Lin
Pedersen is probably THE expert on this.

Cheers,
Simon

> On 24/09/2023, at 7:44 AM, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:
> 
> I am in the process of updating the rgl package.  One thing I'd like to
do is to change text support in it when using OpenGL to display to be more like
the way text is drawn in WebGL displays (i.e. the ones rglwidget() produces).
> 
> Currently in R, rgl uses the FTGL library to draw text.  That library is
unsupported these days, and uses the old fixed pipeline in OpenGL.
> 
> In WebGL, text is displayed by "shaders", programs that run on
the GPU. Javascript code prepares bitmap images of the text to display, then the
shader transfers parts of that bitmap to the output display.
> 
> I'd like to duplicate the WebGL process in the C++ code running the
OpenGL display in R.  The first step in this is to render a character vector
full of text into an in-memory raster, taking account of font, cex, etc.  (I
want one raster for the whole vector, with a recording of locations from which
the shader should get each component of it.)
> 
> It looks to me as though I could do this using the ragg::agg_capture device
in R code, but I'd prefer to do it all in C++ code because I may need to
make changes to the raster at times when it's not safe to call back to R,
e.g. if some user interaction requires the axis labels to be recomputed and
redrawn.
> 
> Does anyone with experience doing this kind of thing know of examples I can
follow, or have advice on how to proceed?  Or want to volunteer to help with
this?
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

I'm somewhat aware of how tricky it all is.  For now I'm going to do it 
in R (usng textshaping for layout and base graphics on the 
ragg::agg_capture device to draw to the bitmap).  I'll avoid allowing 
changes to happen in the C++ code.

Eventually I'll see if I can translate the code into C++.  I know 
textshaping has a C interface, but for the actual drawing I'll have to 
work something else out.  Or maybe just leave it in R, and only try to 
write a new bitmap when it's safe.

For future reference, will the measurements reported by 
textshaping::shape_text() match the values used by your Cairo package, 
or are equivalent measurements available elsewhere?

Duncan Murdoch

On 24/09/2023 6:55 p.m., Simon Urbanek wrote:
> Duncan,
> 
> drawing text is one of the most complicated things you can do, so it really
depends how for you want to go. You can do it badly with a simple cairo
show_text API. The steps involved in doing it properly are detecting the
direction of the language, finding fonts, finding glyphs (resolving ligatures),
applying hints, drawing glyphs etc. Fortunately there are libraries that help
with than, but even then it's non-trivial. Probably the most modern pipeline
is icu + harfbuzz + freetype + fontconfig + cairo. This is implemented, e.g in
https://github.com/s-u/Cairo/blob/master/src/cairotalk.c (the meat is in  L608-)
and for all but the drawing part there is an entire R package (in C++) devoted
to this: https://github.com/r-lib/textshaping/tree/main/src -- Thomas Lin
Pedersen is probably THE expert on this.
> 
> Cheers,
> Simon
> 
> 
>> On 24/09/2023, at 7:44 AM, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:
>>
>> I am in the process of updating the rgl package.  One thing I'd
like to do is to change text support in it when using OpenGL to display to be
more like the way text is drawn in WebGL displays (i.e. the ones rglwidget()
produces).
>>
>> Currently in R, rgl uses the FTGL library to draw text.  That library
is unsupported these days, and uses the old fixed pipeline in OpenGL.
>>
>> In WebGL, text is displayed by "shaders", programs that run
on the GPU. Javascript code prepares bitmap images of the text to display, then
the shader transfers parts of that bitmap to the output display.
>>
>> I'd like to duplicate the WebGL process in the C++ code running the
OpenGL display in R.  The first step in this is to render a character vector
full of text into an in-memory raster, taking account of font, cex, etc.  (I
want one raster for the whole vector, with a recording of locations from which
the shader should get each component of it.)
>>
>> It looks to me as though I could do this using the ragg::agg_capture
device in R code, but I'd prefer to do it all in C++ code because I may need
to make changes to the raster at times when it's not safe to call back to R,
e.g. if some user interaction requires the axis labels to be recomputed and
redrawn.
>>
>> Does anyone with experience doing this kind of thing know of examples I
can follow, or have advice on how to proceed?  Or want to volunteer to help with
this?
>>
>> Duncan Murdoch
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>