R devel - Dec 2002 - Samples of external code with various compilers?

R can run external code in C, C++, Fortran, Delphi, etc., but the R
extensions manual only gives limited documentation for anything but C
and C++.  It would be useful to have a collection of sample code
showing how to dyn.load functions written in other languages, if
necessary for a variety of different compilers, in case that makes a
difference.

Does such a collection already exist?  If not, would it be better to
put it into the R extensions manual as a series of appendices, or just
have a web page about it?

I'd volunteer to maintain the web page to hold the samples.

Duncan Murdoch

Documentation for other languages would be great. Perhaps a web page
referred to in the manual would be most appropriate, as that avoids the need
for the R team to have to vet advice concerning languages they neither know
nor care about.

I'd be happy to contribute some examples for Delphi-- a wonderfully
well-organized Pascal-derived language that works on Windows and Linux
(Intel platforms only).

Although I merrily and perpetually write Delphi functions invoked via .C, I
haven't completely figured out how to Delphi-fy the  headers that would be
needed if I wanted to write .Call functions. I almost had this going at some
point-- I did write an automatic header translator in 2001 that seemed to
produce something sensible for the (rather nasty) S6 header files-- but have
lost the thread since then. Would anyone else be interested in having a
Delphi unit that provides matches for the R header files? Would this be
useful for a web page?

cheers
Mark

*******************************

Mark Bravington
CSIRO (CMIS)
PO Box 1538
Castray Esplanade
Hobart
TAS 7001

phone (61) 3 6232 5118
fax (61) 3 6232 5012
Mark.Bravington@csiro.au 

#-----Original Message-----
#From: Duncan Murdoch [mailto:dmurdoch@pair.com]
#Sent: Monday, 2 December 2002 3:57 AM
#To: r-devel@stat.math.ethz.ch
#Subject: [Rd] Samples of external code with various compilers?
#
#
#R can run external code in C, C++, Fortran, Delphi, etc., but the R
#extensions manual only gives limited documentation for anything but C
#and C++.  It would be useful to have a collection of sample code
#showing how to dyn.load functions written in other languages, if
#necessary for a variety of different compilers, in case that makes a
#difference.
#
#Does such a collection already exist?  If not, would it be better to
#put it into the R extensions manual as a series of appendices, or just
#have a web page about it?
#
#I'd volunteer to maintain the web page to hold the samples.
#
#Duncan Murdoch
#
#______________________________________________
#R-devel@stat.math.ethz.ch mailing list
#http://www.stat.math.ethz.ch/mailman/listinfo/r-devel
#

On Sun, 1 Dec 2002, Duncan Murdoch wrote:
> R can run external code in C, C++, Fortran, Delphi, etc., but the R
> extensions manual only gives limited documentation for anything but C
> and C++.  It would be useful to have a collection of sample code
> showing how to dyn.load functions written in other languages, if
> necessary for a variety of different compilers, in case that makes a
> difference.
I think there is information and lots of examples already for Fortran. I
am not sure what more there is to say in `Writing R Extensions'. We could
point at an example in the R code I you think it would be helpful.  But
I'm puzzled here as to what you find lacking. (Fortran is supported both
via .Fortran and by some of the functions described in the section on the
R API.)
> Does such a collection already exist?  If not, would it be better to
> put it into the R extensions manual as a series of appendices, or just
> have a web page about it?
Base R has only interfaces for C and Fortran (and that via C-style
linkage): SJava adds .Java.  So is the issue how to write in other
languages to use a C interface?  `how to dyn.load functions' is easy: you
just create an appropriate compiled object, a shared library, a DLL or (on
MacOS X as I understand it) a module.  The issues seem to be to export the
symbols correctly, and even more to import ones from R correctly,

I will be interested to hear if there are any other (than Java) candidate
languages that are widely available.  Delphi as I understand it is
Windows-only (and the related Kylix is Linux-only?).  Information on using
Delphi would be very helpful in the R for Windows documentation, but you
are one of the experts there.

Brian

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

I can remember hitting several issues with Delphi, that might be applicable
to other languages too. From simple to complicated (in Delphi), these are:

how to write a DLL and export procedures (easy).

how to declare parameters (VAR or pointers to arrays; pretty simple).

how arrays of >1 dimension map to R arrays (easy).

what call mode to use for procedures (i.e. stack order and removal of
parameters; I had always used STDCALL with S and R, and then found I was
getting bugs with R 1.6.1. So in desperation I eventually changed this at
random to C-CALL, and things started working again in R-- and continued to
work in S. To my continued puzzlement, actually.)

how to handle & return character strings (The only way I've found to
return
a string of unknown length, was to call a Delphi function twice; the first
time it merely returns the number of characters in the string. Then create a
string of spaces of the correct length in R, and explicitly put that string
in a .C call to the Delphi function, which this time can fill in the actual
characters.)

how to get the Delphi debugger to work on a DLL being called by R. This is
incredibly easy in S but rather difficult in R, because of a quirk to do
with starting directories.

Then there are one or two tricks I've sometimes used: e.g. how to return a
Delphi pointer to R so that a "persistent" Delphi object can be
created with
one call to a Delphi function from R, then accessed again on a subsequent
call. Useful even with simple things like returning a string, and essential
with really complex structures. All languages will have their own tricks,
which are well worth some informal documentation somewhere.

And then there is the business of turning the R headers into Delphi
equivalents-- a work in progress.

cheers
Mark

*******************************

Mark Bravington
CSIRO (CMIS)
PO Box 1538
Castray Esplanade
Hobart
TAS 7001

phone (61) 3 6232 5118
fax (61) 3 6232 5012
Mark.Bravington@csiro.au 

#-----Original Message-----
#From: Duncan Murdoch [mailto:murdoch@stats.uwo.ca]
#Sent: Tuesday, 3 December 2002 1:15 AM
#To: ripley@stats.ox.ac.uk
#Cc: r-devel@stat.math.ethz.ch
#Subject: Re: [Rd] Samples of external code with various compilers?
#
#
#On Mon, 2 Dec 2002 08:07:19 +0000 (GMT), you wrote in message
#<Pine.LNX.4.31.0212020753570.1625-100000@gannet.stats>:
#
#>I think there is information and lots of examples already for 
#Fortran. 
#
#I was thinking of two additions:
#
# 1.  Rewriting the samples in sections 4.2 and/or 4.5 in Fortran,
#Delphi, etc.  I might make them a little more elaborate, e.g. showing
#how to return a character string.
#
# 2.  Write up the details of how to do it in various specific
#compilers.  For example, if you're using Microsoft Visual Fortran, how
#do you create a DLL, how do you set the exported entry points, what
#bugs do you need to work around.
#
#>Base R has only interfaces for C and Fortran (and that via C-style
#>linkage): SJava adds .Java.  So is the issue how to write in other
#>languages to use a C interface?  `how to dyn.load functions' 
#is easy: you
#>just create an appropriate compiled object, a shared library, 
#a DLL or (on
#>MacOS X as I understand it) a module.  The issues seem to be 
#to export the
#>symbols correctly, and even more to import ones from R correctly.
#
#Yes, that's the main issue.  There are also issues even with C:  if
#you're using some compiler other than gcc, you probably won't compile
#using R SHLIB, so what do you need to do?
#
#Duncan
#
#______________________________________________
#R-devel@stat.math.ethz.ch mailing list
#http://www.stat.math.ethz.ch/mailman/listinfo/r-devel
#

>
#> what call mode to use for procedures (i.e. stack order and removal of
#> parameters; I had always used STDCALL with S and R, and then 
#found I was
#> getting bugs with R 1.6.1. So in desperation I eventually 
#changed this at
#> random to C-CALL, and things started working again in R-- 
#and continued to
#> work in S. To my continued puzzlement, actually.)
#
#Well, that *is* documented.  For R in readme.packages under 
#`Using other
#compilers and languages' which is quite a short section.  So if people
#don't read what is already there, is there any point in expanding it?

Well, here are some reasons:

documentation is distributed in quite a few different places and it is not
always easy to find the right bit. Duncan Murdoch's email said that he'd
also forgotten about readme.packages;

the location of documentation can change between R versions;

In this specific case, not all DLLs are loaded as part of packages, so
readme.packages might not occur to everyone. As Duncan said, it might be
good to refer to that file via dyn.load documentation.

BTW I still use Splus 2000 not S6, and I don't recall finding this in the
documentation for earlier version of S. 

Anyway, the STDCALL stuff was just an example of something that confused me
for a while. Different things will confuse different people. At any rate,
it's not a crime to duplicate the same pieces of information in different
places in the documentation, if it helps the user.

cheers
Mark

#
#It's also documented for S-PLUS (if that is what you mean by `S': it
#reflects a lot of work by Insightful on top of Lucent S4).

Duplicating documentatino

#-- 
#Brian D. Ripley,                  ripley@stats.ox.ac.uk
#Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
#University of Oxford,             Tel:  +44 1865 272861 (self)
#1 South Parks Road,                     +44 1865 272860 (secr)
#Oxford OX1 3TG, UK                Fax:  +44 1865 272595
#

#-----Original Message-----
#From: Duncan Murdoch [mailto:murdoch@stats.uwo.ca]
#Subject: Re: [Rd] Samples of external code with various compilers?
#
#
#On Wed, 4 Dec 2002 17:52:29 +1100 , you wrote in message
#<A8877251964B294BAB5BA1FC58B43FED025B440B@molly.tas.csiro.au>:
#
#>In this specific case, not all DLLs are loaded as part of packages, so
#>readme.packages might not occur to everyone. As Duncan said, 
#it might be
#>good to refer to that file via dyn.load documentation.
#
#I've put the reference there for the next release.  Any other places
#it should be mentioned?
#
#Duncan
#

I was completely unaware of readme.packages, I must say. It's not in my
distributions of 1.3.1 or 1.5.1, but it is in 1.6.1. I seem to recall that I
installed the "source" option for 1.6.1 but not for the others, which
might
explain why. Note that non-C users are quite likely not to bother with the
"source" option when downloading/installing, but they might still
write
DLLs.

There are three places besides dyn.load where mention or expansion might be
useful:

(1) In the main README. At present, "readme.packages" is mentioned,
but only
under "Adding packages" under "installing packages from source
code". Since
I was never likely to do that with other peoples' source, I didn't look
at
this. Perhaps a short section in README explictly called "Writing
DLLs"
(underlined with = signs) could alert people to the existence of
"readme.packages".

(2) In the help for .C etc., where the "Writing R Extensions" manual
is
already referred to.

(3) And presumably in "Writing R Extensions" itself, if
"readme.packages" is
likely to be a semi-permanent feature of R for Windows.

cheers
Mark

*******************************

Mark Bravington
CSIRO (CMIS)
PO Box 1538
Castray Esplanade
Hobart
TAS 7001

phone (61) 3 6232 5118
fax (61) 3 6232 5012
Mark.Bravington@csiro.au

PS correction to previous my email, where I wrote the 2nd part before the
1st part:

Places to mention "readme.packages":

#(3) And presumably in "Writing R Extensions" itself, if 
#"readme.packages" is
#likely to be a semi-permanent feature of R for Windows.

I'm now thinking that readme.packages is already semi-permanent; it's
just
that you don't get it unless you install the "with-source"
version. (or am I
confused? I can't remember exactly what I've installed with each
version.)


cheers
Mark

*******************************

Mark Bravington
CSIRO (CMIS)
PO Box 1538
Castray Esplanade
Hobart
TAS 7001

phone (61) 3 6232 5118
fax (61) 3 6232 5012
Mark.Bravington@csiro.au

______________________________________________
R-devel@stat.math.ethz.ch mailing list
http://www.stat.math.ethz.ch/mailman/listinfo/r-devel

#>
#>There are three places besides dyn.load where mention or 
#expansion of "readme.packages" might be
#>useful:
#>
#>(1) In the main README. At present, "readme.packages" is 
#mentioned, but only
#>under "Adding packages" under "installing packages from 
#source code". Since
#>I was never likely to do that with other peoples' source, I 
#didn't look at
#>this. Perhaps a short section in README explictly called 
#"Writing DLLs"
#>(underlined with = signs) could alert people to the existence of
#>"readme.packages".
#
#I don't see it in README.  Are you thinking of rw-FAQ maybe?  
#
#The reason it wouldn't be in README is that README is supposed to give
#information common to all platforms.  But perhaps it would be useful
#to add a pointer to platform-specific files to the RESOURCES file?

Hmmm-- it is in my README for RW1061. The README file for the Windows binary
distribution specifically mentions Windows in its title line. Also, there
doesn't seem to be a file called RESOURCES in my distributions (1.3.1,
1.5.0, 1.5.1, 1.6.1, all for Windows). I installed pre-compiled binary
versions-- don't know if the "completely from source" method gives
different
files?


cheers
Mark