Registration of 'native routines' (entry points in compiled code loaded
into R) has been available for over 14 years, but few packages make use
of it (less than 10% of those on CRAN with compiled code).
Registration has similar benefits to name spaces in R code:
- it ensures that the routines used by .C, .Call etc are those in your
package (without needing a PACKAGE argument).
- it avoids polluting the search space for native routines with those
from your package.
- it checks the number of arguments passed to .Call/.External, and the
number and optionally the type for .C/.Fortran.
- it finds native routines faster, especially if 10s of name spaces are
loaded.
Kurt Hornik and I have written a tool to make adding registration much
easier. From NEWS in R-devel
? Package tools has a new function
package_native_routine_registration_skeleton() to assist adding
native-routine registration to a package. See its help and ?5.4.1
of ?Writing R Extensions? for how to use it. (At the time it was
added it successfully automated adding registration to over 90%
of CRAN packages which lacked it. Many of the failures were
newly-detected bugs in the packages, e.g. 50 packages called
entry points with varying numbers of arguments and 65 packages
called entry points not in the package.)
Of the 2450 CRAN packages with compiled code but not registration, this
tool successfully[*] converts all but 220 out-of-the-box. Another 25
packages already use R_init_pkgname and so need the registration info to
be merged.
Most of the rest fail because of errors in the package but some try to
do tricky things computing names of routines, and this is noted in the
skeleton output.
A few of you are using the older mechanism of specifying entry points in
a useDynLib call in the NAMESPACE file. This shares the first and fourth
benefits (but registration is faster), but not the second and third.
$5.4 of the 'Writing R Extensions' manual has been re-written with a
worked example of adding registration.
Shortly, R CMD check --as-cran will note if registration is not fully
used. Expect to be asked to add registration: as increasingly large
numbers of packages are used, avoiding polluting the search space is
becoming important.
(There are reports that the check in 'R CMD check' on Windows sometimes
fails to detect use of registration. This is being looked into:
meanwhile say so in a CRAN submission if it happens to you.)
[*] R CMD check output is unchanged.
--
Brian D. Ripley, ripley at stats.ox.ac.uk
Emeritus Professor of Applied Statistics, University of Oxford
On Tue, Feb 14, 2017 at 11:25 AM, Prof Brian Ripley <ripley at stats.ox.ac.uk> wrote:> Registration of 'native routines' (entry points in compiled code loaded into > R) has been available for over 14 years, but few packages make use of it > (less than 10% of those on CRAN with compiled code). > > Registration has similar benefits to name spaces in R code: > > - it ensures that the routines used by .C, .Call etc are those in your > package (without needing a PACKAGE argument). > > - it avoids polluting the search space for native routines with those from > your package. > > - it checks the number of arguments passed to .Call/.External, and the > number and optionally the type for .C/.Fortran. > > - it finds native routines faster, especially if 10s of name spaces are > loaded. > > Kurt Hornik and I have written a tool to make adding registration much > easier. From NEWS in R-devel > > ? Package tools has a new function > package_native_routine_registration_skeleton() to assist adding > native-routine registration to a package. See its help and ?5.4.1 > of ?Writing R Extensions? for how to use it. (At the time it was > added it successfully automated adding registration to over 90% > of CRAN packages which lacked it. Many of the failures were > newly-detected bugs in the packages, e.g. 50 packages called > entry points with varying numbers of arguments and 65 packages > called entry points not in the package.)Hello, Dr., Ripley. This is fantastic. Is there a way to install this functionality into an existing 3.3.2 installation, or is it exclusive to R-deve;/R-3.4-to-be? Thank you, Avi
On 14/02/2017 17:28, Avraham Adler wrote:> On Tue, Feb 14, 2017 at 11:25 AM, Prof Brian Ripley > <ripley at stats.ox.ac.uk> wrote: >> Registration of 'native routines' (entry points in compiled code loaded into >> R) has been available for over 14 years, but few packages make use of it >> (less than 10% of those on CRAN with compiled code). >> >> Registration has similar benefits to name spaces in R code: >> >> - it ensures that the routines used by .C, .Call etc are those in your >> package (without needing a PACKAGE argument). >> >> - it avoids polluting the search space for native routines with those from >> your package. >> >> - it checks the number of arguments passed to .Call/.External, and the >> number and optionally the type for .C/.Fortran. >> >> - it finds native routines faster, especially if 10s of name spaces are >> loaded. >> >> Kurt Hornik and I have written a tool to make adding registration much >> easier. From NEWS in R-devel >> >> ? Package tools has a new function >> package_native_routine_registration_skeleton() to assist adding >> native-routine registration to a package. See its help and ?5.4.1 >> of ?Writing R Extensions? for how to use it. (At the time it was >> added it successfully automated adding registration to over 90% >> of CRAN packages which lacked it. Many of the failures were >> newly-detected bugs in the packages, e.g. 50 packages called >> entry points with varying numbers of arguments and 65 packages >> called entry points not in the package.) > > Hello, Dr., Ripley. > > This is fantastic. Is there a way to install this functionality into > an existing 3.3.2 installation, or is it exclusive to > R-deve;/R-3.4-to-be?You need to run the tool in R-devel to get the skeleton for your package. Everything else will work in any recent version of R.> > Thank you, > > Avi >-- Brian D. Ripley, ripley at stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford
On 14/02/2017 16:25, Prof Brian Ripley wrote:> Registration of 'native routines' (entry points in compiled code loaded > into R) has been available for over 14 years,...> (There are reports that the check in 'R CMD check' on Windows sometimes > fails to detect use of registration. This is being looked into: > meanwhile say so in a CRAN submission if it happens to you.)One instance has been resolved: for at least 5 years R CMD check for a package with compiled code has assumed commands 'nm' and (on Windows) 'objdump' were on the PATH: these are also needed to detect use of registration. There is now a warning if they are not found. -- Brian D. Ripley, ripley at stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford
On 2017-02-19 09:19, Prof Brian Ripley wrote:> On 14/02/2017 16:25, Prof Brian Ripley wrote: >> Registration of 'native routines' (entry points in compiled code loaded >> into R) has been available for over 14 years, > > ... > >> (There are reports that the check in 'R CMD check' on Windows sometimes >> fails to detect use of registration. This is being looked into: >> meanwhile say so in a CRAN submission if it happens to you.) > > One instance has been resolved: for at least 5 years R CMD check for a > package with compiled code has assumed commands 'nm' and (on Windows) > 'objdump' were on the PATH: these are also needed to detect use of > registration. There is now a warning if they are not found.Many thanks for this! However, I think that there's a subtlety here: On a newly installed Windows 10 64-bit system, with the latest R-devel and Rtools, 'R CMD check' failed to detect use of registration despite the fact that both 'nm' and 'objdump' were on the path. It turned out that I had the 32-bit versions on the path, i.e., '\\...\Rtools\mingw_32\bin', which, I believe, is the default setting in the Rtools installer. After switching to the 64-bit versions, the use of registration now seems to be reliably detected. Henric Winell
On Tue, Feb 14, 2017 at 5:25 PM, Prof Brian Ripley <ripley at stats.ox.ac.uk> wrote:> > Registration has similar benefits to name spaces in R code: > > - it ensures that the routines used by .C, .Call etc are those in your package (without needing a PACKAGE argument). > - it avoids polluting the search space for native routines with those from your package. > - it checks the number of arguments passed to .Call/.External, and the number and optionally the type for .C/.Fortran. > - it finds native routines faster, especially if 10s of name spaces are loaded.Do these benefits also hold for packages that currently use useDynLib exclusively in combination symbol names? E.g for the example from WRE: useDynLib(foo, myRoutine, myOtherRoutine) Which is invoked via: .Call(myRoutine, x, y) What ambiguity or pollution is introduced by foo:::myRoutine? Should manually registering 'myRoutine' in C still be mandatory in this case? It was nice having the NAMESPACE as the central declaration of callable C routines. The "R_registerRoutines" system will require maintaining additional C code which re-declares all callable C functions from other compilation units. This introduces additional complexity for package authors and might become a source of bugs when we forget to update the registrations when C functions have changed.
I believe Dr. Ripley said that the UseDynLib doesn't address the middle two points of the list. I experimented with the Delaporte package timing calls using both methods, and they were within 1~2 microseconds of each other using microbenchmark. Since the more complicated way is now expressly preferred, it isn't a big deal for me personally to change so the next version of Delaporte will use CallMethod/useDynLib(foo, .registration=TRUE) etc. I can see how in more complicated packages that could become more burdensome. Thanks, Avi On Wed, Feb 22, 2017 at 9:34 AM, Jeroen Ooms <jeroen.ooms at stat.ucla.edu> wrote:> On Tue, Feb 14, 2017 at 5:25 PM, Prof Brian Ripley > <ripley at stats.ox.ac.uk> wrote: >> >> Registration has similar benefits to name spaces in R code: >> >> - it ensures that the routines used by .C, .Call etc are those in your package (without needing a PACKAGE argument). >> - it avoids polluting the search space for native routines with those from your package. >> - it checks the number of arguments passed to .Call/.External, and the number and optionally the type for .C/.Fortran. >> - it finds native routines faster, especially if 10s of name spaces are loaded. > > Do these benefits also hold for packages that currently use useDynLib > exclusively in combination symbol names? E.g for the example from WRE: > > useDynLib(foo, myRoutine, myOtherRoutine) > > Which is invoked via: > > .Call(myRoutine, x, y) > > What ambiguity or pollution is introduced by foo:::myRoutine? Should > manually registering 'myRoutine' in C still be mandatory in this case? > > It was nice having the NAMESPACE as the central declaration of > callable C routines. The "R_registerRoutines" system will require > maintaining additional C code which re-declares all callable C > functions from other compilation units. This introduces additional > complexity for package authors and might become a source of bugs when > we forget to update the registrations when C functions have changed. > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel