R devel - Jan 2006 - Loading of namespace on load of .Rdata (was strange behaviour of load)

Last week Giovanni Parrinello posted a message asking why various packages were
loaded when he loaded an .Rdata file. Brian Ripley replied saying he thought it
was because the saved workspace contained a reference to the namespace of ipred.
(Correspondence copied below).

This begs the question: how did the reference to the namespace of ipred come to
be in the .Rdata file? Brian did say it is likely to be because the workspace
contained object(s) saved with environment the namespace of ipred - but how
would this come about?

In this case I think is because the .Rdata file contained an object whose
*parent* environment was the namespace of ipred. Take the following example from
?bagging (having loaded ipred):
> data(BreastCancer)
> 
> mod <- bagging(Class ~ Cl.thickness + Cell.size
+                 + Cell.shape + Marg.adhesion   
+                 + Epith.c.size + Bare.nuclei   
+                 + Bl.cromatin + Normal.nucleoli
+                 + Mitoses, data=BreastCancer,
coob=TRUE)
>
> environment(mod$mtrees[[1]]$btree$terms)
<environment: 024E8138>
>
> parent.env(environment(mod$mtrees[[1]]$btree$terms))
<environment: namespace:ipred>

This occurs because the terms object is taken from the model frame which was
evaluated within the environment of a function from the ipred package (here
ipred:::irpart).

Therefore I think the behaviour observed by Giovanni will only occur in unusual
circumstances: when the workspace contains a formula object, a terms object, a
function, or some other object with a non-NULL environment, which has been
created in the environment of a packaged function. In particular, this would not
always occur with a packaged model fitting function, e.g. (from ?loglm in MASS)
> library(MASS)
> minn38a <- array(0, c(3,4,7,2), lapply(minn38[, -5], levels))
> minn38a[data.matrix(minn38[,-5])] <- minn38$f
> fm <- loglm(~1 + 2 + 3 + 4, minn38a)  
> environment(fm$terms)
<environment: R_GlobalEnv>

in this case because the terms component is obtained from the formula, whose
environment is .GlobalEnv.

So, I have two points on this (more for R-devel than R-help now)

1. There is a more general situation where it would be useful to load the
namespace of a package after loading a saved workspace: when the workspace
contains objects of a class for which special methods are required. E.g. if
'fm' from the example above were saved in a workspace, the namespace of
MASS would not be loaded when the workspace was loaded into R. Thus unless MASS
was loaded by the user, default methods would be used by summary(), print() etc
rather than the specialised methods for objects of class "loglm".

Of course the user should quickly realise this, but there may be cases where the
default method gives a convincing but incorrect or unhelpful result. An
alternative would be to add an attribute to objects of class "loglm"
(say), e.g. attr(loglmObject, ".Environment") <- environment(MASS)
so that the namespace would automatically be loaded when it is required. [In
fact alternatives such as environment(loglmObject) <- environment(MASS) or
loglmObject$anyoldthing <- environment(MASS) would work just as well, but
perhaps the first suggestion is neatest.].

What do others think of this idea? Should it (or an equivalent idea) be
encouraged amongst package writers?

2. In the case highlighted by Giovanni, the namespace of ipred was loaded, but
the package was not. This would be fine, except that the packages on which ipred
depends *were* loaded. This seems inconsistent. I guess as long as there are
packages without namespaces though, this is the only way to proceed. Perhaps in
the meantime, package authors should be encouraged to use importFrom() rather
than import()? Or perhaps where packages do have namespaces, only the namespace
should be loaded in such a case.

Heather
> From: Prof Brian Ripley <ripley at stats.ox.ac.uk>
> Date: 12 January 2006 08:21:35 GMT
> To: giovanni parrinello <parrinel at med.unibs.it>
> Cc: r-help at stat.math.ethz.ch
> Subject: Re: [R] Strange behaviour of load
>
> On Wed, 11 Jan 2006, giovanni parrinello wrote:
>
>> Dear All,
>> simetimes when I load an Rdata I get this message
>>
>> #######
>> Code:
>>
>> load('bladder1.RData')
>> Carico il pacchetto richiesto: rpart ( Bad traslastion: Load required 
>> package-...)
>> Carico il pacchetto richiesto: MASS
>> Carico il pacchetto richiesto: mlbench
>> Carico il pacchetto richiesto: survival
>> Carico il pacchetto richiesto: splines
>>
>> Carico il pacchetto richiesto: 'survival'
>>
>>
>>        The following object(s) are masked from package:Hmisc :
>>
>>         untangle.specials
>>
>> Carico il pacchetto richiesto: class
>> Carico il pacchetto richiesto: nnet
>> #########
>>
>> So  I have many unrequired packages loaded.
>> Any idea?
>
> They are required!  My guess is that you have object(s) saved with
> environment the namespace of some package, and loading that namespace 
> is
> pulling these in.  The only CRAN package which requires mlbench 
> appears to
> be ipred, and that requires all of those except splines, required by
> survival.
>
> So I believe you have been using ipred and have saved a reference to 
> its
> namespace.
>
> -- 
> Brian D. Ripley,                  ripley at stats.ox.ac.uk
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
Dr H Turner
Research Assistant
Dept. of Statistics
The University of Warwick
Coventry
CV4 7AL

Tel: 024 76575870
Url: www.warwick.ac.uk/go/heatherturner

On Wed, 18 Jan 2006, Heather Turner wrote:

[Lines wrapped for legibility and R-help removed as it is not an 
appropriate list.]
> Last week Giovanni Parrinello posted a message asking why various 
> packages were loaded when he loaded an .Rdata file. Brian Ripley replied 
> saying he thought it was because the saved workspace contained a 
> reference to the namespace of ipred. (Correspondence copied below).
>
> This begs the question: how did the reference to the namespace of ipred 
> come to be in the .Rdata file? Brian did say it is likely to be because 
> the workspace contained object(s) saved with environment the namespace 
> of ipred - but how would this come about?
>
> In this case I think is because the .Rdata file contained an object 
> whose *parent* environment was the namespace of ipred. Take the 
> following example from ?bagging (having loaded ipred):
Excuse me: environments do not have parents but enclosures according to 
?environment.

Of course, the environment of mod is itself an object, and so my statement 
holds true.  Saving a workspace saves all the objects (possibly as 
references) whether named or not.  I was fully aware that the namespace 
was likely to be up the environment tree of a named object when I chose my 
words carefully.
>> data(BreastCancer)
>>
>> mod <- bagging(Class ~ Cl.thickness + Cell.size
> +                 + Cell.shape + Marg.adhesion
> +                 + Epith.c.size + Bare.nuclei
> +                 + Bl.cromatin + Normal.nucleoli
> +                 + Mitoses, data=BreastCancer, coob=TRUE)
>>
>> environment(mod$mtrees[[1]]$btree$terms)
> <environment: 024E8138>
>>
>> parent.env(environment(mod$mtrees[[1]]$btree$terms))
> <environment: namespace:ipred>
parent.env is a very confusing name.  To quote the draft R language 
definition:

 	`The parent.env function may be used to access the enclosure
 	of an environment.'

[...]


-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

[R-help removed]

See below.

Heather Turner wrote:

<snip>
> 
> 1. There is a more general situation where it would be useful to load
> the namespace of a package after loading a saved workspace: when the
> workspace contains objects of a class for which special methods are
> required. E.g. if 'fm' from the example above were saved in a
> workspace, the namespace of MASS would not be loaded when the
> workspace was loaded into R. Thus unless MASS was loaded by the user,
> default methods would be used by summary(), print() etc rather than
> the specialised methods for objects of class "loglm".
> 
> Of course the user should quickly realise this, but there may be
> cases where the default method gives a convincing but incorrect or
> unhelpful result. An alternative would be to add an attribute to
> objects of class "loglm" (say), e.g. attr(loglmObject,
> ".Environment") <- environment(MASS) so that the namespace
would
> automatically be loaded when it is required. [In fact alternatives
> such as environment(loglmObject) <- environment(MASS) or
> loglmObject$anyoldthing <- environment(MASS) would work just as well,
> but perhaps the first suggestion is neatest.].
> 
> What do others think of this idea? Should it (or an equivalent idea)
> be encouraged amongst package writers?
If I understand you correctly here, what you are talking about works 
with S4 classes.  For example, if I load an object 'x' (say, from a 
saved workspace) of class "foo" and it has a 'show()' method
in package
"bar", then

show(x)

will automatically load package "bar".  I this is accomplished because
there is a "package" attribute that is part of the class of the
object.

It's true, what you're talking about does not exist in S3, but since it 
does in S4, I'm not sure it's worth enhancing the older system.
> 
> 2. In the case highlighted by Giovanni, the namespace of ipred was
> loaded, but the package was not. This would be fine, except that the
> packages on which ipred depends *were* loaded. This seems
> inconsistent. I guess as long as there are packages without
> namespaces though, this is the only way to proceed. Perhaps in the
> meantime, package authors should be encouraged to use importFrom()
> rather than import()? Or perhaps where packages do have namespaces,
> only the namespace should be loaded in such a case.
> 
> Heather
> 
> 
>> From: Prof Brian Ripley <ripley at stats.ox.ac.uk> Date: 12
January
>> 2006 08:21:35 GMT To: giovanni parrinello <parrinel at
med.unibs.it>
>> Cc: r-help at stat.math.ethz.ch Subject: Re: [R] Strange behaviour of
>> load
>> 
>> On Wed, 11 Jan 2006, giovanni parrinello wrote:
>> 
>> 
>>> Dear All, simetimes when I load an Rdata I get this message
>>> 
>>> ####### Code:
>>> 
>>> load('bladder1.RData') Carico il pacchetto richiesto: rpart
( Bad
>>> traslastion: Load required package-...) Carico il pacchetto
>>> richiesto: MASS Carico il pacchetto richiesto: mlbench Carico il
>>> pacchetto richiesto: survival Carico il pacchetto richiesto:
>>> splines
>>> 
>>> Carico il pacchetto richiesto: 'survival'
>>> 
>>> 
>>> The following object(s) are masked from package:Hmisc :
>>> 
>>> untangle.specials
>>> 
>>> Carico il pacchetto richiesto: class Carico il pacchetto
>>> richiesto: nnet #########
>>> 
>>> So  I have many unrequired packages loaded. Any idea?
>> 
>> They are required!  My guess is that you have object(s) saved with 
>> environment the namespace of some package, and loading that
>> namespace is pulling these in.  The only CRAN package which
>> requires mlbench appears to be ipred, and that requires all of
>> those except splines, required by survival.
>> 
>> So I believe you have been using ipred and have saved a reference
>> to its namespace.
>> 
>> -- Brian D. Ripley,                  ripley at stats.ox.ac.uk 
>> Professor of Applied Statistics,
>> http://www.stats.ox.ac.uk/~ripley/ University of Oxford,
>> Tel:  +44 1865 272861 (self) 1 South Parks Road,
>> +44 1865 272866 (PA) Oxford OX1 3TG, UK                Fax:  +44
>> 1865 272595
>> 
>> ______________________________________________ 
>> R-help at stat.math.ethz.ch mailing list 
>> https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the
>> posting guide! http://www.R-project.org/posting-guide.html
> 
> 
> Dr H Turner Research Assistant Dept. of Statistics The University of
> Warwick Coventry CV4 7AL
> 
> Tel: 024 76575870 Url: www.warwick.ac.uk/go/heatherturner
> 
> ______________________________________________ R-devel at r-project.org
> mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
> 
-- 
Roger D. Peng  |  http://www.biostat.jhsph.edu/~rpeng/