It appears that grDevices (and perhaps other R libraries) is built
without passing C_VISIBILITY to PKG_CFLAGS (-fvisibility=hidden),
while other libraries (e.g. utils) do use the flag.
This results in potential crashes on linux in situations like using
embedded R in another application that happens to export a symbol
named similarly to one defined in R's library, since the runtime
linker substitutes the address of the wrong function into the
library's CallEntries.
Using -fvisibility=hidden or -Wl,-Bsymbolic[-functions] fixes this
since then the addresses of functions in CallEntries are fixed at
library build time, which is probably the intended behaviour.
Here is a small example, using grDevices::colors. I compile it with
gcc -O2 r.c -I/usr/include/R -L/usr/lib/R/lib -lR -DBUG -rdynamic
(remove -DBUG to disable triggering the bug) and run it with
R_HOME=/usr/lib/R ./a.out
without -DBUG colors() from grDevices is called as expected, however,
with -DBUG adding a colors() function definition in the hosting
binary, this version ends up being called which is most likely not the
intent. I suspect this might not manifest on Debian as it has a global
-Wl,-Bsymbolic-functions which likely gets passed on during the R
build process.
Is there any particular reason why grDevices isn't compiled with
-fvisibility=hidden like utils (which doesn't exhibit the described
problem), and shouldn't that flag and/or -Wl,-Bsymbolic also make its
way to R CMD's *LIB_*FLAGS to aid in getting correct builds of custom
R libraries too?
Thanks
--- r.c ---
#include<Rinternals.h>
#include<Rdefines.h>
#include<Rembedded.h>
#include<R_ext/Parse.h>
#ifdef BUG
void colors(){puts("all colours of the rainbow, but not the expected
ones");}
#endif
int cmd(char*x){SEXP c,e;ParseStatus s;int o;
PROTECT(c=allocVector(STRSXP,1));SET_STRING_ELT(c,0,mkChar(x))
PROTECT(e=R_ParseVector(c,1,&s,R_NilValue));
PROTECT(R_tryEval(VECTOR_ELT(e,0),R_GlobalEnv,&o));
UNPROTECT(3);return s;}
int
main(){Rf_initEmbeddedR(2,(char*[]){"R","--slave"});cmd("grDevices::colors()");return
0;}
---
