R devel - Sep 2023 - Help requested: writing text to a raster in memory

I am in the process of updating the rgl package.  One thing I'd like to 
do is to change text support in it when using OpenGL to display to be 
more like the way text is drawn in WebGL displays (i.e. the ones 
rglwidget() produces).

Currently in R, rgl uses the FTGL library to draw text.  That library is 
unsupported these days, and uses the old fixed pipeline in OpenGL.

In WebGL, text is displayed by "shaders", programs that run on the
GPU.
Javascript code prepares bitmap images of the text to display, then the 
shader transfers parts of that bitmap to the output display.

I'd like to duplicate the WebGL process in the C++ code running the 
OpenGL display in R.  The first step in this is to render a character 
vector full of text into an in-memory raster, taking account of font, 
cex, etc.  (I want one raster for the whole vector, with a recording of 
locations from which the shader should get each component of it.)

It looks to me as though I could do this using the ragg::agg_capture 
device in R code, but I'd prefer to do it all in C++ code because I may 
need to make changes to the raster at times when it's not safe to call 
back to R, e.g. if some user interaction requires the axis labels to be 
recomputed and redrawn.

Does anyone with experience doing this kind of thing know of examples I 
can follow, or have advice on how to proceed?  Or want to volunteer to 
help with this?

Duncan Murdoch

Duncan,

drawing text is one of the most complicated things you can do, so it really
depends how for you want to go. You can do it badly with a simple cairo
show_text API. The steps involved in doing it properly are detecting the
direction of the language, finding fonts, finding glyphs (resolving ligatures),
applying hints, drawing glyphs etc. Fortunately there are libraries that help
with than, but even then it's non-trivial. Probably the most modern pipeline
is icu + harfbuzz + freetype + fontconfig + cairo. This is implemented, e.g in
https://github.com/s-u/Cairo/blob/master/src/cairotalk.c (the meat is in  L608-)
and for all but the drawing part there is an entire R package (in C++) devoted
to this: https://github.com/r-lib/textshaping/tree/main/src -- Thomas Lin
Pedersen is probably THE expert on this.

Cheers,
Simon

> On 24/09/2023, at 7:44 AM, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:
> 
> I am in the process of updating the rgl package.  One thing I'd like to
do is to change text support in it when using OpenGL to display to be more like
the way text is drawn in WebGL displays (i.e. the ones rglwidget() produces).
> 
> Currently in R, rgl uses the FTGL library to draw text.  That library is
unsupported these days, and uses the old fixed pipeline in OpenGL.
> 
> In WebGL, text is displayed by "shaders", programs that run on
the GPU. Javascript code prepares bitmap images of the text to display, then the
shader transfers parts of that bitmap to the output display.
> 
> I'd like to duplicate the WebGL process in the C++ code running the
OpenGL display in R.  The first step in this is to render a character vector
full of text into an in-memory raster, taking account of font, cex, etc.  (I
want one raster for the whole vector, with a recording of locations from which
the shader should get each component of it.)
> 
> It looks to me as though I could do this using the ragg::agg_capture device
in R code, but I'd prefer to do it all in C++ code because I may need to
make changes to the raster at times when it's not safe to call back to R,
e.g. if some user interaction requires the axis labels to be recomputed and
redrawn.
> 
> Does anyone with experience doing this kind of thing know of examples I can
follow, or have advice on how to proceed?  Or want to volunteer to help with
this?
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>