No, this is not about Rcpp, but a comment in that overly long discussion raised a question that has been in my mind for a while. This is that one may have work that is used in R in the base functionality and there are improvements that should be incorporated. For me, this concerns the BFGS, Nelder-Mead and CG options of optim(), which are based on the 1990 edition (Pascal codes) of my 1979 book "Compact numerical methods...", which were themselves derived from other people's work. By the time Brian Ripley took that work (with permission, even though not strictly required. Thanks!) there were already some improvements to these same algorithms (mainly bounds and masks) in the BASIC codes of the 1987 book by Mary Walker-Smith and I. However, BASIC to R is not something I'd wish on anyone. Now there are some R packages, including some I've been working on, that do offer improvements on the optim() offerings. I would not say mine are yet fully ready for incorporation into the base, but they are pretty close. Equally I think some of the tools in the base should be deprecated and users encouraged to try other routines. It is also getting more and more important that novice users be provided with sensible guidance and robust default settings and choices. In many areas, users are faced with more choice than is efficient for the majority of problems. My question is: How should such changes be suggested / assisted? It seems to me that this is beyond a simple feature request. Some discussion on pros and cons would be appropriate, and those like myself who are familiar with particular tools can and should offer help. Alternatively, is there a document available in the style "Writing R Extensions" that has a title like "How the R Base Packages are Updated"? A brief search was negative. I'm happy to compete with my own prior work to provide improvements. It would be nice to see some of those improvements become the benchmark for further progress. Best, John Nash
On 03/12/2010 10:57 AM, Prof. John C Nash wrote:> No, this is not about Rcpp, but a comment in that overly long discussion raised a question > that has been in my mind for a while. > > This is that one may have work that is used in R in the base functionality and there are > improvements that should be incorporated. > > For me, this concerns the BFGS, Nelder-Mead and CG options of optim(), which are based on > the 1990 edition (Pascal codes) of my 1979 book "Compact numerical methods...", which were > themselves derived from other people's work. By the time Brian Ripley took that work (with > permission, even though not strictly required. Thanks!) there were already some > improvements to these same algorithms (mainly bounds and masks) in the BASIC codes of the > 1987 book by Mary Walker-Smith and I. However, BASIC to R is not something I'd wish on > anyone. > > Now there are some R packages, including some I've been working on, that do offer > improvements on the optim() offerings. I would not say mine are yet fully ready for > incorporation into the base, but they are pretty close. Equally I think some of the tools > in the base should be deprecated and users encouraged to try other routines. It is also > getting more and more important that novice users be provided with sensible guidance and > robust default settings and choices. In many areas, users are faced with more choice than > is efficient for the majority of problems. > > My question is: How should such changes be suggested / assisted? It seems to me that this > is beyond a simple feature request. Some discussion on pros and cons would be appropriate, > and those like myself who are familiar with particular tools can and should offer help. > > Alternatively, is there a document available in the style "Writing R Extensions" that has > a title like "How the R Base Packages are Updated"? A brief search was negative. > > I'm happy to compete with my own prior work to provide improvements. It would be nice to > see some of those improvements become the benchmark for further progress.There are answers at many different levels to your questions. The simplest is that base packages are part of R, so they get updated when a member of R Core updates them, and the updates get released when a new version of R is released. So if you want a change, you need to convince a member of the core to make it. Pointing out a bug is the easiest way to do this: bugs usually get fixed quickly, if they are clearly demonstrated. If you want a bigger change, you need to make a convincing argument in favour of it. If you pick a topic that is of particular interest to one core member, and you can convince him to make the change, then it will happen. If pick some obscure topic that's not of interest to anyone, you'll need a very strong argument to make it interesting. Part of any of these arguments is explaining why the change needs to be made to the base, why it can't just be published in a contributed package. (That's why bug fixes are easy, and big additions to the base packages are not.) Duncan Murdoch
Ben raised an interesting point: "A better question might be how
packages get added to the *recommended*
package list (rather than how code gets added to "base R")."
As maintainer of survival one of the surprising things is the number
of packages that depend on mine. This has caused me to change my
opinion over the last few years about how much expansion of the package
should occur. I used to feel bad that the package doesn't keep up with
everything, now I tend to vote for putting new ideas elsewhere.
Consider competing risks for instance. The cleanest way to code this is
to extend the Surv(time, status) construct to allow more than a 0/1
status variable. I thought about this seriously, and realized that such
a change would have ripple effects on some of the other routines -- an
extra if statement here and there. This is doable, but what about the
effect on the 153 dependent packages! The stability of survival is now
more important than its feature set.
My first extention to random effects (a very simplistic one) was
incorporated into the main survival package. I've pushed the later
developments into thier own package. It was definitely the right
choice.
Terry T