Hi
I recently gave a presentation about Automatic Differentiation (AD) and R at the
Eighth Euro AD Workshop in Oxford (17/07/09). The presentation was intended as
a general introduction to R and the desire for a generic AD interface for R.
During the presentation I emphasised the need and the high level of interest
that the R community has in developing such an interface and that input from the
AD community is essential to achieve it.
The presentation seemed to generate a good level of interest and several
suggestions for potential directions were made. However, it is not clear at the
moment what the best way is of implementing an AD interface in R in terms of
success, difficulty and amount of work.
One suggestion was that it may be possible to develop it entirely in R, using
the S4 classes and overloading the intrinsic functions. This may not result in
the fastest solution, in terms of time taken to calculate the derivatives, but
could result in a 'clean' interface that was relatively straightforward
to extend.
Alternatively, it may be possible to make use of a pre-exisiting AD library in
another language (e.g. ADOL-C or the AutoDiff libraries from AD Model Builder).
As these libraries are C++ based this would require functions that were written
in R to be parsed into C++ and then compiled before their derviatives could be
found. Although this has the advantage of using work that is already well
developed, the parsing and compilation from R to C++ may make the interface
'clunky'.
It is likely that we won't know how successful either of these approaches
could be until someone actually sits down and has a go at doing it. Is there
anyone out there who is interested in getting involved with this project or who
has advice / time to offer?
Also, would it be possible to also post this message in the R-devel list? I
realise that this would mean 'double posting' but the potential for a
generic AD interface affects many packages and may also be of interest to
developers (e.g. John Nash's 'optimx' package)
Finlay Scott
***********************************************************************************
This email and any attachments are intended for the name...{{dropped:10}}
R does not currently have AD (except for the Ryacas package
which can do true AD for certain simple one line functions, i..e.
input the function and output a function representing its
derivative); however, for specific problems one can get close
using deriv and associated functions or the approach explained
below using rSymPy:
1. R has symbolic differentiation (not quite the same as AD but
it could be built upon) already:
?deriv
2. the rSymPy package on CRAN is an interface to the sympy
computer algebra system. It is meant for symbolic manipulation
rather than AD but it can be used for semi-automatic differentiation
as in this example:
http://code.google.com/p/rsympy/#Semi-Automatic_Differentiation
You just replace the indented code with your function, suitably modified,
as in the example.
Home page is at:
http://rsympy.googlecode.com
3. the Ryacas R package on CRAN can do AD for certain simple one
line functions (and an approach similar to rSymPy is also possible):
library(Ryacas)
demo("Ryacas-Function")
Home page is at:
http://ryacas.googlecode.com
If its your intention to use it that way you would probably want
to write a short wrapper function to make it more convenient to
use; however, the basic capability is there.
4. if numeric differentiation is sufficient see:
?numericDeriv
On Wed, Jul 22, 2009 at 5:21 AM, Finlay Scott
(Cefas)<finlay.scott at cefas.co.uk> wrote:> Hi
> I recently gave a presentation about Automatic Differentiation (AD) and R
at the Eighth Euro AD Workshop in Oxford (17/07/09). ?The presentation was
intended as a general introduction to R and the desire for a generic AD
interface for R. ?During the presentation I emphasised the need and the high
level of interest that the R community has in developing such an interface and
that input from the AD community is essential to achieve it.
>
> The presentation seemed to generate a good level of interest and several
suggestions for potential directions were made. ?However, it is not clear at the
moment what the best way is of implementing an AD interface in R in terms of
success, difficulty and amount of work.
>
> One suggestion was that it may be possible to develop it entirely in R,
using the S4 classes and overloading the intrinsic functions. ?This may not
result in the fastest solution, in terms of time taken to calculate the
derivatives, but could result in a 'clean' interface that was relatively
straightforward to extend.
>
> Alternatively, it may be possible to make use of a pre-exisiting AD library
in another language (e.g. ADOL-C or the AutoDiff libraries from AD Model
Builder). ?As these libraries are C++ based this would require functions that
were written in R to be parsed into C++ and then compiled before their
derviatives could be found. ?Although this has the advantage of using work that
is already well developed, the parsing and compilation from R to C++ may make
the interface 'clunky'.
>
> It is likely that we won't know how successful either of these
approaches could be until someone actually sits down and has a go at doing it.
?Is there anyone out there who is interested in getting involved with this
project or who has advice / time to offer?
>
> Also, would it be possible to also post this message in the R-devel list?
?I realise that this would mean 'double posting' but the potential for a
generic AD interface affects many packages and may also be of interest to
developers (e.g. John Nash's 'optimx' package)
>
> Finlay Scott
>
>
>
***********************************************************************************
> This email and any attachments are intended for the name...{{dropped:10}}
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
>> Gabor G. wrote>> "R does not currently have AD (except for the Ryacas package >> which can do true AD for certain simple one line functions, i..e. >> input the function and output a function representing its >> derivative); however, for specific problems one can get close >> using deriv and associated functions or the approach explained >> below using rSymPy: >> ... As the instigator of Finlay's participation in this work, I probably didn't express clearly enough the contribution Gabor has made to get as far as he has with Ryacas and rSympy, which may show another pathway for AD/Symbolic diff. development. At UseR all conversations seemed more rushed than I'd like. Gabor showed Ravi Varadhan and I a way to get some derivatives via his tools that "worked". We need to play with this a bit more to see how general it could be -- Gabor is very fair in his post that some work is needed for each instance. On the other hand, if analytic gradients were straightforward, we wouldn't be exchanging posts about them. The clear issue in my mind is that users who need gradients/Jacobians for R want to be able to send a function X to some process that will return another function gradX or JacX that computes analytic derivatives. This has to be "easy", which implies a very simple command or GUI interface. I am pretty certain the users have almost no interest in the mechanism, as long as it works. Currently, most use numerical derivatives, not realizing the very large time penalty and quite large loss in accuracy that can compromise some optimization and differential equation codes. I'll try to prepare a few examples to illustrate this and post them somewhere in the next few weeks. Time, as always, ... However, the topic does appear to be on the table. JN