Recently Brian Ripley had occasion to mock my inability to see a
comment in the code for arima(), in the stats package. After
considerable dredging around in the r-news archives I found reference
to keep.source() and keep.source.pkgs(), which I conjectured just
***might*** possibly be the ``obvious'' resource to which Prof.
Ripley thought I ought to have resorted.
However, after substantial experimentation, I am still at a loss to
get at the original source code for arima() (replete with comments)
save by accessing the original, ``unmade'' source for R. Not too
hard, but not exactly ``immediate''.
In the course of experimenting with keep.source.pkgs() I encountered
behaviour which I found mysterious and which I'd like to understand
better. Also if there is an easier way to get at comments in code of
``system'' functions I'd like to know about it.
In experimenting with keep.source.pkgs() I did the following
o executed options("keep.source.pkgs"), and got ``FALSE''
o executed library(spatstat)
o typed the name of a function in spatstat whose code has
comments (rmh.ppm) --- no comments appeared.
o executed options(keep.source.pkgs=TRUE)
o typed ``rmh.ppm'' again --- still no comments, as is to
be expected.
o detached spatstat
o reloaded it
o type ``rmh.ppm'' again --- comments were there in all
their glory. OMMMMMMMMMMMMMMMMM!
Then I went through the same procedure with the package util and the
function vignette therefrom. (This function has comments in its
code.) Except that I didn't do an initial library(util), since util
gets loaded automatically on startup.
After setting keep.source.pkgs to TRUE via options(), detaching util,
and reloading it --- still no comments, and attr(vignette,"source")
was NULL.
The help for options() says that keep.source.pkgs defaults to TRUE
if the environment variable R_KEEP_PKG_SOURCE is set to `yes'.
So my next experiment was to stop R, set this variable, restart
R, and then type ``vignette''. Bingo. Comments appeared, and
attr(vignette,"source") was there bigtime.
However attr(arima,"source") was still NULL (and of course comments
were non-existant). Apparently the functions in stats get
***loaded*** from a binary file ``all.rda'' (in the
.../library/stats/R directory) rather than getting read in from ascii
files. (One hesitates to ask ***why***, but .... why?) Anyhow, I
guess that the binary versions in all.rda got built effectively with
keep.source.pkgs=FALSE, and so starting R with keep.source.pkgs=TRUE
doesn't help in this case.
So, some questions:
1) Why do ``system'' packages, e.g. util, behave differently
from ``optional'' packages? I.e. why does the sequence
. detach package
. set keep.source.pkgs=TRUE
. re-load package
``work'', i.e. make the source available for functions in
optional packages but not for functions in system packages?
2) Is there a better way of getting to see the
code-with-comments of system functions, rather than dicking
around with the R_KEEP_PKG_SOURCE environment variable?
3) Is there any way, other than retrieving the unmade source
of R, to see the commented source code of functions in the
stat package (and presumably methods, stats4, lattice, and
nlme which also boast all.rda files)?
I wonder if someone with better people skills and more patience
than Prof. Ripley would be so kind as to answer!
cheers,
Rolf Turner
rolf at math.unb.ca
On Tue, 3 Aug 2004, Rolf Turner wrote:> > However, after substantial experimentation, I am still at a loss to > get at the original source code for arima() (replete with comments) > save by accessing the original, ``unmade'' source for R. Not too > hard, but not exactly ``immediate''.Personally, I find this the easiest way to look at the source. You can browse the source on https://svn.r-project.org/R/ if you are using a binary distribution, rather than downloading the whole thing.> In the course of experimenting with keep.source.pkgs() I encountered > behaviour which I found mysterious and which I'd like to understand > better. Also if there is an easier way to get at comments in code of > ``system'' functions I'd like to know about it. > > In experimenting with keep.source.pkgs() I did the following> o executed options("keep.source.pkgs"), and got ``FALSE'' > o executed library(spatstat) > o typed the name of a function in spatstat whose code has > comments (rmh.ppm) --- no comments appeared. > o executed options(keep.source.pkgs=TRUE) > o typed ``rmh.ppm'' again --- still no comments, as is to > be expected. > o detached spatstat > o reloaded it > o type ``rmh.ppm'' again --- comments were there in all > their glory. OMMMMMMMMMMMMMMMMM!Yes.> Then I went through the same procedure with the package util and the > function vignette therefrom. (This function has comments in its > code.) Except that I didn't do an initial library(util), since util > gets loaded automatically on startup. > > After setting keep.source.pkgs to TRUE via options(), detaching util, > and reloading it --- still no comments, and attr(vignette,"source") > was NULL. > > The help for options() says that keep.source.pkgs defaults to TRUE > if the environment variable R_KEEP_PKG_SOURCE is set to `yes'. > So my next experiment was to stop R, set this variable, restart > R, and then type ``vignette''. Bingo. Comments appeared, and > attr(vignette,"source") was there bigtime.Isn't that nice.> However attr(arima,"source") was still NULL (and of course comments > were non-existant). Apparently the functions in stats get > ***loaded*** from a binary file ``all.rda'' (in the > .../library/stats/R directory) rather than getting read in from ascii > files. (One hesitates to ask ***why***, but .... why?)Because R starts much faster that way.> Anyhow, I > guess that the binary versions in all.rda got built effectively with > keep.source.pkgs=FALSE, and so starting R with keep.source.pkgs=TRUE > doesn't help in this case. > > So, some questions: > > 1) Why do ``system'' packages, e.g. util, behave differently > from ``optional'' packages? I.e. why does the sequence > > . detach package > . set keep.source.pkgs=TRUE > . re-load package > > ``work'', i.e. make the source available for functions in > optional packages but not for functions in system packages?Because, as you noted, the base packages are stored in binary form. This already speeds things up, and will have even more impact in 2.0.0 with "lazy loading" of functions.> 2) Is there a better way of getting to see the > code-with-comments of system functions, rather than dicking > around with the R_KEEP_PKG_SOURCE environment variable?Downloading the source, or looking at svn.r-project.org> 3) Is there any way, other than retrieving the unmade source > of R, to see the commented source code of functions in the > stat package (and presumably methods, stats4, lattice, and > nlme which also boast all.rda files)?It doesn't look like it.> I wonder if someone with better people skills and more patience > than Prof. Ripley would be so kind as to answer!When a Certain Guru rips strips off people (God knows he's done it to me often enough) on this list, there's a damned good reason for it. -- Rolf Turner (in a discussion about whether a friendly mailing list with more `customer service' attitude than R-help was needed) R-help (December 2003) -thomas
Thomas Lumley wrote:> > However attr(arima,"source") was still NULL (and of course comments > > were non-existant). Apparently the functions in stats get > > ***loaded*** from a binary file ``all.rda'' (in the > > .../library/stats/R directory) rather than getting read in from ascii > > files. (One hesitates to ask ***why***, but .... why?) > > Because R starts much faster that way.Ah-ha!!! That indeed makes perfect sense. Thank you.> > So, some questions: > > > > 1) Why do ``system'' packages, e.g. utils, behave differently > > from ``optional'' packages? I.e. why does the sequence > > > > . detach package > > . set keep.source.pkgs=TRUE > > . re-load package > > > > ``work'', i.e. make the source available for functions in > > optional packages but not for functions in system packages? > > Because, as you noted, the base packages are stored in binary form. This > already speeds things up, and will have even more impact in 2.0.0 with > "lazy loading" of functions.This CANNOT be the (complete) explanation, because the utils package behaves differently from the stats package. What I mean is, your explanation explains why you never get the source attributes ***at all*** with the stats package. It doesn't seem to me to explain why setting keep.source.pkgs=TRUE, detaching and reloading doesn't work for utils, whereas effectively setting keep.source.pkgs=TRUE ``a priori'' and then loading utils ***does*** work. And ***nothing*** works with the stats package. Moreover in .../library/utils/R you find the ascii source file ``utils'' and ***NOT*** a binary ``all.rda'' file.> > I wonder if someone with better people skills and more patience > > than Prof. Ripley would be so kind as to answer! > > When a Certain Guru rips strips off people (God knows he's done it to me > often enough) on this list, there's a damned good reason for it. > -- Rolf Turner (in a discussion about whether a friendly mailing list with > more `customer service' attitude than R-help was needed) > R-help (December 2003)Oh, far be it from me to disagree. Especially with myself! I do not dispute the right to rip strips. I just wanted a comprehensible answer to my question, that's all. cheers, Rolf Turner rolf at math.unb.ca
Thomas Lumley wrote:> The reason why detach and reload doesn't work but restarting does work for > utils is that detach() does not unload the package, it only detaches it. > This distinction was introduced with namespaces. A package can be loaded > but not in the search path.Ah-ha. The distinction between unloading and detaching --- which I hadn't grokked --- would seem to be the key issue. Now I get it. Dankaschoen beaucoup. cheers, Rolf Turner