R devel - May 2024 - How to call directly "dotTcl" C-function of the tcltk-package from the C-code of an external package?

I have a use case with tcltk-package where I need to repeatedly call Tcl/Tk
functions
very quickly. For such purpose, the standard R-interface turned out to be too
slow, and
better option has been to call package's C-function dotTcl directly from my
own C-code.

Before R 4.4.0 it was possible to use
getNativeSymbolInfo("dotTcl","tcltk")$address (or
R_FindSymbol("dotTcl","tcltk",NULL) in C) to get the
function-pointer and then call the
function directly, even though it has not been registered as C-callable for
other
packages.

With R 4.4.0 these methods are unable to find the symbol anymore (tested in
Linux).
I was not able to find what change has caused this new behaviour.

Taking a look at tcltk source code, it can be seen that the dotTcl is called
using
.External within tcltk-package and there is a registration done for it with
R_registerRoutines. An object of class NativeSymbolInfo has also been created in
the
tcltk namespace, and that can be accessed using tcltk:::.C_dotTcl.

However, the tcltk:::.C_dotTcl$address is an external pointer of a class
RegisteredNativeSymbol and not directly the function pointer to the actual
routine. The
problem is that there appears not to be any R-level function that would extract
the actual
function-pointer and that the C-interface for R_RegisteredNativeSymbol has been
defined
in the internal Rdynpriv.h header that is not included in the public API
headers.

The only way I was able to access the function directly was using the following
C-level
approach in which essential parts of the headers are copied from the Rdynpriv.h:

#include <R.h>
#include <Rinternals.h>
#include <R_ext/Rdynload.h>

typedef struct {
    char       *name;
    DL_FUNC     fun;
    int         numArgs;

    R_NativePrimitiveArgType *types;   
} Rf_DotCSymbol;

typedef Rf_DotCSymbol Rf_DotFortranSymbol;

typedef struct {
    char       *name;
    DL_FUNC     fun;
    int         numArgs;
} Rf_DotCallSymbol;

typedef Rf_DotCallSymbol Rf_DotExternalSymbol;

struct Rf_RegisteredNativeSymbol {
    NativeSymbolType type;
    union {
	Rf_DotCSymbol        *c;
	Rf_DotCallSymbol     *call;
	Rf_DotFortranSymbol  *fortran;
	Rf_DotExternalSymbol *external;
    } symbol;
};

SEXP(*direct_dotTcl)(SEXP) = NULL;

SEXP FindRegFunc(SEXP symbol) {
    R_RegisteredNativeSymbol *tmp = NULL;
    tmp = (R_RegisteredNativeSymbol *) R_ExternalPtrAddr(symbol);
    if (tmp==NULL) return R_NilValue;
	direct_dotTcl = (SEXP(*)(SEXP)) tmp->symbol.external->fun;	
    return R_NilValue;
}	


Although that works for me, I'm aware that this kind of approach is
certainly not
recommmended for publicly available external packages. However, I couldn't
find any
other more legitimate way to access dotTcl function directly from my C-code in R
4.4.0.


I have two questions:

Would it be possible to get dotTcl C-function (in tcltk.c) of the tcltk-package
registered as C-callable from other packages?

Was it an intentional change that caused the hiding of the earlier visible
(registered)
symbols from other packages?

I asked Tomas. 


Apparently this works:

getNativeSymbolInfo("dotTcl",PACKAGE=getLoadedDLLs()$tcltk)

and the tcltk behavior was changed by 


r84265 | hornik | 2023-04-15 08:44:36 +0200 (Sat, 15 Apr 2023) | 1 line

Try forcing symbols in ff calls.

Index: library/tcltk/src/init.c
==================================================================---
library/tcltk/src/init.c    (revision 84264)
+++ library/tcltk/src/init.c    (revision 84265)
@@ -66,6 +66,6 @@
 {
     R_registerRoutines(dll, CEntries, NULL, NULL, ExternEntries);
     R_useDynamicSymbols(dll, FALSE);
-    R_forceSymbols(dll, FALSE);
+    R_forceSymbols(dll, TRUE);
 }

I don't know what that was all about, and I'm also a bit puzzled

a) that the .External lookup is so slow that you need to bypass it (would have
thought that the byte compiler could speed it up)
b) that you don't use dotTclObjv and friends to avoid parsing on the Tcl
side.

- pd

> On 29 May 2024, at 00:25 , Reijo Sund <reijo.sund at uef.fi> wrote:
> 
> I have a use case with tcltk-package where I need to repeatedly call Tcl/Tk
functions
> very quickly. For such purpose, the standard R-interface turned out to be
too slow, and
> better option has been to call package's C-function dotTcl directly
from my own C-code.
> 
> Before R 4.4.0 it was possible to use
getNativeSymbolInfo("dotTcl","tcltk")$address (or
> R_FindSymbol("dotTcl","tcltk",NULL) in C) to get the
function-pointer and then call the
> function directly, even though it has not been registered as C-callable for
other
> packages.
> 
> With R 4.4.0 these methods are unable to find the symbol anymore (tested in
Linux).
> I was not able to find what change has caused this new behaviour.
> 
> Taking a look at tcltk source code, it can be seen that the dotTcl is
called using
> .External within tcltk-package and there is a registration done for it with
> R_registerRoutines. An object of class NativeSymbolInfo has also been
created in the
> tcltk namespace, and that can be accessed using tcltk:::.C_dotTcl.
> 
> However, the tcltk:::.C_dotTcl$address is an external pointer of a class
> RegisteredNativeSymbol and not directly the function pointer to the actual
routine. The
> problem is that there appears not to be any R-level function that would
extract the actual
> function-pointer and that the C-interface for R_RegisteredNativeSymbol has
been defined
> in the internal Rdynpriv.h header that is not included in the public API
headers.
> 
> The only way I was able to access the function directly was using the
following C-level
> approach in which essential parts of the headers are copied from the
Rdynpriv.h:
> 
> #include <R.h>
> #include <Rinternals.h>
> #include <R_ext/Rdynload.h>
> 
> typedef struct {
>    char       *name;
>    DL_FUNC     fun;
>    int         numArgs;
> 
>    R_NativePrimitiveArgType *types;   
> } Rf_DotCSymbol;
> 
> typedef Rf_DotCSymbol Rf_DotFortranSymbol;
> 
> typedef struct {
>    char       *name;
>    DL_FUNC     fun;
>    int         numArgs;
> } Rf_DotCallSymbol;
> 
> typedef Rf_DotCallSymbol Rf_DotExternalSymbol;
> 
> struct Rf_RegisteredNativeSymbol {
>    NativeSymbolType type;
>    union {
> 	Rf_DotCSymbol        *c;
> 	Rf_DotCallSymbol     *call;
> 	Rf_DotFortranSymbol  *fortran;
> 	Rf_DotExternalSymbol *external;
>    } symbol;
> };
> 
> SEXP(*direct_dotTcl)(SEXP) = NULL;
> 
> SEXP FindRegFunc(SEXP symbol) {
>    R_RegisteredNativeSymbol *tmp = NULL;
>    tmp = (R_RegisteredNativeSymbol *) R_ExternalPtrAddr(symbol);
>    if (tmp==NULL) return R_NilValue;
> 	direct_dotTcl = (SEXP(*)(SEXP)) tmp->symbol.external->fun;	
>    return R_NilValue;
> }	
> 
> 
> Although that works for me, I'm aware that this kind of approach is
certainly not
> recommmended for publicly available external packages. However, I
couldn't find any
> other more legitimate way to access dotTcl function directly from my C-code
in R 4.4.0.
> 
> 
> I have two questions:
> 
> Would it be possible to get dotTcl C-function (in tcltk.c) of the
tcltk-package
> registered as C-callable from other packages?
> 
> Was it an intentional change that caused the hiding of the earlier visible
(registered)
> symbols from other packages?
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
-- 
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com