Hi all,
I've just been working with a piece of fortran code (Leo Breiman's
random forest code) that needs a lot of arguments. When I use .Fortran,
I get the message 'too many arguments in foreign function call'.
Neither the help page for .Fortran (actually Foreign) nor "Writing R
Extensions" explains how many arguments are too many.
Looking at the code in src/main/dotcode.c, I see that MAX_ARGS is set to
65.
While it is possible to work around this limitation, it seems
arbitrary. I propose increasing the arbitrary value to 255 and adding
appropriate documentation to the Foreign manual page.
Any objections/comments?
-Greg
-----Patch-Starts-Here------
diff -ur R-1.3.0-orig/src/library/base/man/Foreign.Rd
R-1.3.0-patched/src/library/base/man/Foreign.Rd
--- R-1.3.0-orig/src/library/base/man/Foreign.Rd Sun May 6
12:14:16 2001+++ R-1.3.0-patched/src/library/base/man/Foreign.Rd Fri
Jul 13 06:58:42 2001@@ -20,7 +20,7 @@
\arguments{
\item{name}{a character string giving the name of a C function or
Fortran subroutine.}
- \item{\dots}{arguments to be passed to the foreign function.}
+ \item{\dots}{arguments (maximum of 255) to be passed to the foreign
function} \item{NAOK}{if \code{TRUE} then any \code{\link{NA}} or
\code{\link{NaN}} or \code{\link{Inf}} values in the arguments are
passed on to the foreign function. If \code{FALSE}, the presence
of
Only in R-1.3.0-patched/src/library/base/man: Foreign.Rd~
diff -ur R-1.3.0-orig/src/main/dotcode.c
R-1.3.0-patched/src/main/dotcode.c
--- R-1.3.0-orig/src/main/dotcode.c Sat Jun 9 19:35:19 2001
+++ R-1.3.0-patched/src/main/dotcode.c Fri Jul 13 06:58:56 2001
@@ -372,7 +372,7 @@
return args;
}
-#define MAX_ARGS 65
+#define MAX_ARGS 255
SEXP do_symbol(SEXP call, SEXP op, SEXP args, SEXP env)
{
-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !) To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Gregory R. Warnes wrote:> > Hi all, > > I've just been working with a piece of fortran code (Leo Breiman's > random forest code) that needs a lot of arguments. When I use .Fortran, > I get the message 'too many arguments in foreign function call'. > Neither the help page for .Fortran (actually Foreign) nor "Writing R > Extensions" explains how many arguments are too many. > > Looking at the code in src/main/dotcode.c, I see that MAX_ARGS is set to > 65. > > While it is possible to work around this limitation, it seems > arbitrary. I propose increasing the arbitrary value to 255 and adding > appropriate documentation to the Foreign manual page. > > Any objections/comments?Unfortunately, it isn't quite as simple as that. The code that uses MAX_ARGS would also need to be extended and that involves adding the remaining cases to the switch statement in do_dotCode() and do_dotcall() in src/main/dotcode.c There's no reason why it can't be done. Somebody simply has to generate the appropriate code, presumably programmatically with a little script. I recall this was extended in S4, but 64 was adequate for that context. I imagine that there are some systems limit the number of arguments one can have to a routine. D.> > -Greg > > -----Patch-Starts-Here------ > > diff -ur R-1.3.0-orig/src/library/base/man/Foreign.Rd > R-1.3.0-patched/src/library/base/man/Foreign.Rd > --- R-1.3.0-orig/src/library/base/man/Foreign.Rd Sun May 6 > 12:14:16 2001+++ R-1.3.0-patched/src/library/base/man/Foreign.Rd Fri > Jul 13 06:58:42 2001@@ -20,7 +20,7 @@ > \arguments{ > \item{name}{a character string giving the name of a C function or > Fortran subroutine.} > - \item{\dots}{arguments to be passed to the foreign function.} > + \item{\dots}{arguments (maximum of 255) to be passed to the foreign > function} \item{NAOK}{if \code{TRUE} then any \code{\link{NA}} or > \code{\link{NaN}} or \code{\link{Inf}} values in the arguments are > passed on to the foreign function. If \code{FALSE}, the presence > of > Only in R-1.3.0-patched/src/library/base/man: Foreign.Rd~ > diff -ur R-1.3.0-orig/src/main/dotcode.c > R-1.3.0-patched/src/main/dotcode.c > --- R-1.3.0-orig/src/main/dotcode.c Sat Jun 9 19:35:19 2001 > +++ R-1.3.0-patched/src/main/dotcode.c Fri Jul 13 06:58:56 2001 > @@ -372,7 +372,7 @@ > return args; > } > > -#define MAX_ARGS 65 > +#define MAX_ARGS 255 > > SEXP do_symbol(SEXP call, SEXP op, SEXP args, SEXP env) > { > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- > r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html > Send "info", "help", or "[un]subscribe" > (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._-- _______________________________________________________________ Duncan Temple Lang duncan@research.bell-labs.com Bell Labs, Lucent Technologies office: (908)582-3217 700 Mountain Avenue, Room 2C-259 fax: (908)582-3340 Murray Hill, NJ 07974-2070 http://cm.bell-labs.com/stat/duncan -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Duncan Templ Lang wrote:> > Gregory R. Warnes wrote: > > > > Hi all, > > > > > > Looking at the code in src/main/dotcode.c, I see that MAX_ARGS is set to > > 65. > > > > While it is possible to work around this limitation, it seems > > arbitrary. I propose increasing the arbitrary value to 255 and adding > > appropriate documentation to the Foreign manual page. > > > > Any objections/comments? > > Unfortunately, it isn't quite as simple as that. The code that uses > MAX_ARGS would also need to be extended and that involves adding the > remaining cases to the switch statement in do_dotCode() and > do_dotcall() in src/main/dotcode.c There's no reason why it can't be > done. Somebody simply has to generate the appropriate code, > presumably programmatically with a little script. I recall this was > extended in S4, but 64 was adequate for that context. > > I imagine that there are some systems limit the number of arguments > one can have to a routine. >OK. I wrote a little R script to generate the necessary case statements. Since these code chunks are rather large (7140 lines each!), I modifed src/main/dotcode to #include the case statements from the (new) files /src/main/dotcode_cases.c and /src/main/dotcall_cases.c . I'm not sure that this is the best way to do this, but including these code chunks in dotcode.c makes it unmanagable. While I was making and (this time) testing the changes, I noticed that one declaration had the value 65 hardcoded rather than using MAX_ARGS. I've uploaded a tgz file containing the patch and the R script that generated the case statement/call code to ftp::/cran.r-project.org/incoming/dotcall_255.tgz With this patch applied, the fortran subroutin I'm using now works fine with 85 arguments. I imagine that it should be tested on several platforms to ensure that there isn't (as Duncan menioned) that a problem with compilers/platforms restricting the number of arguments. -Greg -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
I would like to create an R package that will depend on the ability to use more than 65 parameters. Would it be reasonable to expect this patch (or something equivalent) to be part of , say R > 1.3.1 ? Would it help if I submit an official bug report? -Greg From: Gregory R. Warnes [mailto:greg@warnes.net] > > Duncan Templ Lang wrote: > > > > Gregory R. Warnes wrote: > > > > > > Hi all, > > > > > > > > > Looking at the code in src/main/dotcode.c, I see that > MAX_ARGS is set to > > > 65. > > > > > > While it is possible to work around this limitation, it seems > > > arbitrary. I propose increasing the arbitrary value to > 255 and adding > > > appropriate documentation to the Foreign manual page. > > > > > > Any objections/comments? > > > > Unfortunately, it isn't quite as simple as that. The code > that uses > > MAX_ARGS would also need to be extended and that involves > adding the > > remaining cases to the switch statement in do_dotCode() and > > do_dotcall() in src/main/dotcode.c There's no reason why > it can't be > > done. Somebody simply has to generate the appropriate code, > > presumably programmatically with a little script. I > recall this was > > extended in S4, but 64 was adequate for that context. > > > > I imagine that there are some systems limit the number of > arguments > > one can have to a routine. > > > > OK. I wrote a little R script to generate the necessary case > statements. Since these code chunks are rather large (7140 lines > each!), I modifed src/main/dotcode to #include the case > statements from > the (new) files /src/main/dotcode_cases.c and > /src/main/dotcall_cases.c > . I'm not sure that this is the best way to do this, but including > these code chunks in dotcode.c makes it unmanagable. > > While I was making and (this time) testing the changes, I > noticed that > one declaration had the value 65 hardcoded rather than > using MAX_ARGS. > > I've uploaded a tgz file containing the patch and the R script that > generated the case statement/call code to > ftp::/cran.r-project.org/incoming/dotcall_255.tgz > > With this patch applied, the fortran subroutin I'm using > now works fine > with 85 arguments. I imagine that it should be tested on several > platforms to ensure that there isn't (as Duncan menioned) > that a problem > with compilers/platforms restricting the number of arguments. > > -Greg > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > -.-.-.-.-.-.-.-.-.- > r-devel mailing list -- Read > http://www.ci.tuwien.ac.at/~hornik/R/R- > FAQ.html > Send > "info", "help", or "[un]subscribe" > (in the > "body", not the subject !) To: r-devel-request@stat.math.ethz.ch > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. > _._._._._._._._._._ > LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately. -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
On Mon, 16 Jul 2001 10:50:25 -0400, you wrote in message <5429125E11E4D411AF7300805FE603A8014CEC74@groexmbcr02.pfizer.com>:> >I would like to create an R package that will depend on the ability to use >more than 65 parameters. Would it be reasonable to expect this patch (or >something equivalent) to be part of , say R > 1.3.1 ?People are slow to upgrade, so you might be better off in the short run to somehow reduce the number of parameters below 65. For example, write a wrapper function that groups a number of scalar parameters into one vector. Duncan Murdoch -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Warnes, Gregory R
2001-Jul-16 17:01 UTC
[R] RE: [Rd] too many arguments in foreign function call
Duncan Murdoch <murdoch at stats.uwo.ca> wrote: > Gregory R. Warnes <gregory_r_warnes at groton.pfizer.com> wrote > > > >I would like to create an R package that will depend on > the ability to use > >more than 65 parameters. Would it be reasonable to expect > this patch (or > >something equivalent) to be part of , say R > 1.3.1 ? > > People are slow to upgrade, so you might be better off in the short > run to somehow reduce the number of parameters below 65. > For example, > write a wrapper function that groups a number of scalar parameters > into one vector. Actually, the whole point of the patch was to get out of this. I want to keep the code as close to what was written by the original author and as simple as possible... -Greg LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately. -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Warnes, Gregory R
2001-Jul-16 23:30 UTC
[R] RE: [Rd] too many arguments in foreign function call
From: Prof Brian D Ripley [mailto:ripley@stats.ox.ac.uk] > > On Mon, 16 Jul 2001, Warnes, Gregory R wrote: > > > Duncan Murdoch <murdoch@stats.uwo.ca> wrote: > > > Gregory R. Warnes <gregory_r_warnes@groton.pfizer.com> wrote > > > > > > > > I would like to create an R package that will depend on > > > > the ability to use > > > > more than 65 parameters. Would it be reasonable to expect > > > > this patch (or > > > > something equivalent) to be part of , say R > 1.3.1 ? > > > > > > People are slow to upgrade, so you might be better > > > off in the short > > > run to somehow reduce the number of parameters below 65. > > > For example, > > > write a wrapper function that groups a number of > > > scalar parameters > > > into one vector. > > > > Actually, the whole point of the patch was to get out of > > this. I want to keep the code as close to what was written > > by the original author and as > > simple as possible... > > > Sorry, but I don't see that. You want to write the wrapper > in R? Why is > it not as simple to write it in C? It is, in fact *not* as simple to write it in C. Now the user must go from 2 languages (R and FORTRAN) to 3. > I really don't see the need to remove > the limitation in R (or even to have it as high as 65) > since R now has > .External() which allows an unlimited list of arguments to > be passed to C > code. I can't believe that the natural R representation is > 85 separate > objects, No the actual R representation isn't anywhere near 85 separate objects. The actual R representation is more like 10 objects. The point is that existing FORTRAN code---which is used for R, S-Plus, and for a stand alone program---makes use of some 60 objects which need to be dynamically sized, hence the other 25 parameters specifying object sizes. Indeed many of these objects are merely 'scratch space' for the code to work in. Still, its easier for the user to allocate these in R (and have R manage the memory) than to write the equivalent code in C. > so the natural way looks like a C wrapper > translating between R > objects and Fortran pointers. You can even copy selectively. > > The argument is that new code should probably be using .Call (if S > compatibility is relevant) or .External rather than .C, at > least provided > that a C wrapper is feasible. Those interfaces seem very > much under-used. > (Including by me, for S3-compatibility reasons.) Brian, do you really expect everyone to get in the habit of writing C code to translate R SEXP into appropriate C or Fortran objects? For the large number of cases where the objects to be passed to and from C/Fortran are simple scalars and vectors it really makes sense to keep the code to do the translation in a standard library routine like do_dotcode(). This keeps the bugs down to a minimum and makes it as easy as possible for people to interface code. All I suggested doing was removing the arbitrary restriction to <= 65 parameters. I even provided the code. What's the problem? > > -- > Brian D. Ripley, ripley@stats.ox.ac.uk > Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ > University of Oxford, Tel: +44 1865 272861 (self) > 1 South Parks Road, +44 1865 272860 (secr) > Oxford OX1 3TG, UK Fax: +44 1865 272595 > > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > -.-.-.-.-.-.-.-.-.- > r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. _._ LEGAL NOTICE Unless expressly stated otherwise, this message is confidential and may be privileged. It is intended for the addressee(s) only. Access to this E-mail by anyone else is unauthorized. If you are not an addressee, any disclosure or copying of the contents of this E-mail or any action taken (or not taken) in reliance on it is unauthorized and may be unlawful. If you are not an addressee, please inform the sender immediately. -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Prof Brian D Ripley
2001-Jul-17 06:45 UTC
[R] RE: [Rd] too many arguments in foreign function call
On Mon, 16 Jul 2001, Warnes, Gregory R wrote:> > From: Prof Brian D Ripley [mailto:ripley@stats.ox.ac.uk] > > > > On Mon, 16 Jul 2001, Warnes, Gregory R wrote: > > > > > Duncan Murdoch <murdoch@stats.uwo.ca> wrote: > > > > Gregory R. Warnes <gregory_r_warnes@groton.pfizer.com> wrote > > > > > > > > > > I would like to create an R package that will depend on > > > > > the ability to use > > > > > more than 65 parameters. Would it be reasonable to expect > > > > > this patch (or > > > > > something equivalent) to be part of , say R > 1.3.1 ? > > > > > > > > People are slow to upgrade, so you might be better > > > > off in the short > > > > run to somehow reduce the number of parameters below 65. > > > > For example, > > > > write a wrapper function that groups a number of > > > > scalar parameters > > > > into one vector. > > > > > > Actually, the whole point of the patch was to get out of > > > this. I want to keep the code as close to what was written > > > by the original author and as > > > simple as possible... > > > > > > Sorry, but I don't see that. You want to write the wrapper > > in R? Why is > > it not as simple to write it in C? > > It is, in fact *not* as simple to write it in C. Now the user must go from > 2 languages (R and FORTRAN) to 3.But the user is you, who knows C. One can write the wrapper in Fortran too, as ppr.f exemplifies.> > I really don't see the need to remove > > the limitation in R (or even to have it as high as 65) > > since R now has > > .External() which allows an unlimited list of arguments to > > be passed to C > > code. I can't believe that the natural R representation is > > 85 separate > > objects, > > No the actual R representation isn't anywhere near 85 separate objects. The > actual R representation is more like 10 objects. The point is that existing > FORTRAN code---which is used for R, S-Plus, and for a stand alone > program---makes use of some 60 objects which need to be dynamically sized, > hence the other 25 parameters specifying object sizes. Indeed many of these > objects are merely 'scratch space' for the code to work in. Still, its > easier for the user to allocate these in R (and have R manage the memory) > than to write the equivalent code in C.Really? Using R_alloc in C seems exactly equivalent to me.> > so the natural way looks like a C wrapper > > translating between R > > objects and Fortran pointers. You can even copy selectively. > > > > The argument is that new code should probably be using .Call (if S > > compatibility is relevant) or .External rather than .C, at > > least provided > > that a C wrapper is feasible. Those interfaces seem very > > much under-used. > > (Including by me, for S3-compatibility reasons.) > > Brian, do you really expect everyone to get in the habit of writing C code > to translate R SEXP into appropriate C or Fortran objects? For the largeNo, but I am suggesting that most package writers should.> number of cases where the objects to be passed to and from C/Fortran are > simple scalars and vectors it really makes sense to keep the code to do the > translation in a standard library routine like do_dotcode(). This keeps the > bugs down to a minimum and makes it as easy as possible for people to > interface code.In most cases it is one or two vectors. Beyond that, the lack of type-checking causes a large number of bugs, so I do think using .Call is the way to keep `bugs down to a minimum'.> All I suggested doing was removing the arbitrary restriction to <= 65 > parameters. I even provided the code. What's the problem?1) Your code had as I understand it a different arbitrary restriction. 2) We don't know what the OS-specific limits are. Certainly in the days of S-PLUS for Windows 3.3 there were quite low limits on the total size of the arguments in use. These days I don't even know how to find out such information. 3) We always have to weigh the support issues of changing the code to support a rare case. (I do believe it is rare: this is the first instance I have seen in ca 14 years of listening to S and R programmers.) Brian -- Brian D. Ripley, ripley@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-devel-request@stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._