R devel - Oct 2004 - R 2.0.0: namespaces, S4 classes & versioned package installation: failure to resolve correct pkg version

Hi,

further down from the problem I asked about yesterday I encounter the following 
problem (please appologize for the lengthy mail - I have not found a brief
example
making the issue clear):

  - using R packages making use of namespaces &
  - S4 classes
    when using
    --with-package-versions in the install call:

Pkgs install ok and the first pkg 'base' can be loaded in R (2.0.0) ok
but
loading a second pkg 'algo' that defines classes derived from
'base'
fails as 'algo' searches for 'base' in the search path without
the version
extension. The problem does not occur if installed wo --with-package-versions 
option.

The traceback of the actual call is given below, but there is no simple 
reproducible example I have come up with yet.

An reproducible case that might be of some help and give some further insights:
I installed stats4 from the R-source-tree/src/library
with --with-package-versions

In R
 >library("stats4_2.0.0")
 >search()
[1] ".GlobalEnv"           "package:stats4_2.0.0"
"package:stats"
[4] "package:graphics"     "package:grDevices"   
"package:RUnit_0.4.0"
[7] "package:methods"      "Autoloads"           
"package:base"

carrel>obj <- new("mle")
Loading required package: stats4

Attaching package 'stats4':


         The following object(s) are masked from package:stats4_2.0.0 :

          mle
 >class(obj)
[1] "mle"
attr(,"package")
[1] "stats4"


as far as i could understand the problem lies in the 'package' attribute
not
being set to package + version string.

In the call sequence
new -> getClass -> getClassDef: here .requirePackage(package)
is called where package is retrieved from package = packageSlot(Class)

Is this speculation correct or did I make a mistake somewhere else?
Is there any way to teach methods to look for the classes in the versioned 
package in the search path?
Or is this just asking more than can be achieved with the current methods 
implementation?

Any help or suggestion is greatly appreciated.


Regards,

   Matthhias


Motivation:
For ensuring long term reproducible results we so far have used the versioned 
package maintenance concept and certaily would like to continue doing so.


traceback:
library("EpiR.algo_1.4.0")
Loading required package: EpiR.base
Error in as.environment(pkg) : no item called "package:EpiR.base" in
the search list
In addition: Warning message:
replacing previous import: filter in: namespaceImportFrom(self, asNamespace(ns))

Call Stack:
1 :  library("EpiR.algo_1.4.0")
2 :  try({
3 :  loadNamespace(package, c(which.lib.loc, lib.loc))
4 :  sys.source(codeFile, env, keep.source = keep.source)
5 :  eval(i, envir)
6 :  eval(expr, envir, enclos)
7 :  .defineRankingClass(where = .where)
8 :  setClass("Ranking", representation("VIRTUAL", rowGroup
= "Grouping",
colGroup = "Grouping"), prototype(rowGroup = newGrouping(), colGroup =
newGrouping()), vali
9 :  makeClassRepresentation(Class, properties, superClasses, prototype, 
package, validity, access, version, sealed, where = where)
10 :  prototype(rowGroup = newGrouping(), colGroup = newGrouping())
11 :  .prototype(...)
12 :  newGrouping()
13 :  new("Grouping")
14 :  initialize(value, ...)
15 :  initialize(value, ...)
16 :  "@<-"(`*tmp*`, valid, value = TRUE)
17 :  "slot<-"(object, name, TRUE, value)
18 :  checkSlotAssignment(object, name, value)
19 :  getClass(class(obj))
20 :  .requirePackage(package)
21 :  .asEnvironmentPackage(package)
Error in library("EpiR.algo_1.4.0") : package/namespace load failed


info: class Grouping is defined in EpiR.base and used for slot rowGroup,
newGrouping is a simple wrapper to new("Grouping") providing some
meaningful
defaults.

-- 
Matthias Burger                     Project Manager/ Biostatistician
Epigenomics AG    Kleine Praesidentenstr. 1    10178 Berlin, Germany
phone:+49-30-24345-371                          fax:+49-30-24345-555
http://www.epigenomics.com           matthias.burger@epigenomics.com

The first issue here is that

library("stats4_2.0.0")

is incorrect: the package name is "stats4".  You meant, I hope,

library("stats4", version="2.0.0")

and the methods package is correctly using "stats4" as the package
name.
The idea of versioned installs is that only the package name is changed, 
so that binary versions of packages can be installed as is or under the 
version name.

Those who implemented this have not envisaged situations such as you
describe.  For example, require("stats4") requires an unversioned load
if
possible, even though there may already be a versioned install loaded, and
as.environment("package:stats4") fails since search() is reporting
package:stats4_2.0.0.  Because this has not been thought through, some
functions require decorated package names and some do not.

Because versioning of namespaces is not implemented, using versioned 
installs and import/export with namespaces is broken in 2.0.0 (but will 
work up to a point in 2.0.1).
> For ensuring long term reproducible results we so far have used the
> versioned package maintenance concept and certaily would like to
> continue doing so.
I really don't understand that.  R packages are _always_ versioned.
You only need versioned installs if you want to install two versions into 
the same library (or load into the same session, but the latter is not 
supported and was said originally not to be a goal).  Some of us have for 
many years just used separate libraries, which does work.


On Wed, 6 Oct 2004, Matthias Burger wrote:
> 
> Hi,
> 
> further down from the problem I asked about yesterday I encounter the
following
> problem (please appologize for the lengthy mail - I have not found a brief
example
> making the issue clear):
> 
>   - using R packages making use of namespaces &
>   - S4 classes
>     when using
>     --with-package-versions in the install call:
> 
> Pkgs install ok and the first pkg 'base' can be loaded in R (2.0.0)
ok but
> loading a second pkg 'algo' that defines classes derived from
'base'
> fails as 'algo' searches for 'base' in the search path
without the version
> extension. The problem does not occur if installed wo
--with-package-versions
> option.
> 
> The traceback of the actual call is given below, but there is no simple 
> reproducible example I have come up with yet.
> 
> An reproducible case that might be of some help and give some further
insights:
> I installed stats4 from the R-source-tree/src/library
> with --with-package-versions
> 
> In R
>  >library("stats4_2.0.0")
>  >search()
> [1] ".GlobalEnv"           "package:stats4_2.0.0"
"package:stats"
> [4] "package:graphics"     "package:grDevices"   
"package:RUnit_0.4.0"
> [7] "package:methods"      "Autoloads"           
"package:base"
> 
> carrel>obj <- new("mle")
> Loading required package: stats4
> 
> Attaching package 'stats4':
> 
> 
>          The following object(s) are masked from package:stats4_2.0.0 :
> 
>           mle
>  >class(obj)
> [1] "mle"
> attr(,"package")
> [1] "stats4"
> 
> 
> as far as i could understand the problem lies in the 'package'
attribute not
> being set to package + version string.
> 
> In the call sequence
> new -> getClass -> getClassDef: here .requirePackage(package)
> is called where package is retrieved from package = packageSlot(Class)
> 
> Is this speculation correct or did I make a mistake somewhere else?
> Is there any way to teach methods to look for the classes in the versioned 
> package in the search path?
> Or is this just asking more than can be achieved with the current methods 
> implementation?
> 
> Any help or suggestion is greatly appreciated.
> 
> 
> Regards,
> 
>    Matthhias
> 
> 
> Motivation:
> For ensuring long term reproducible results we so far have used the
versioned
> package maintenance concept and certaily would like to continue doing so.
> 
> 
> traceback:
> library("EpiR.algo_1.4.0")
> Loading required package: EpiR.base
> Error in as.environment(pkg) : no item called "package:EpiR.base"
in the search list
> In addition: Warning message:
> replacing previous import: filter in: namespaceImportFrom(self,
asNamespace(ns))
> 
> Call Stack:
> 1 :  library("EpiR.algo_1.4.0")
> 2 :  try({
> 3 :  loadNamespace(package, c(which.lib.loc, lib.loc))
> 4 :  sys.source(codeFile, env, keep.source = keep.source)
> 5 :  eval(i, envir)
> 6 :  eval(expr, envir, enclos)
> 7 :  .defineRankingClass(where = .where)
> 8 :  setClass("Ranking", representation("VIRTUAL",
rowGroup = "Grouping",
> colGroup = "Grouping"), prototype(rowGroup = newGrouping(),
colGroup =
> newGrouping()), vali
> 9 :  makeClassRepresentation(Class, properties, superClasses, prototype, 
> package, validity, access, version, sealed, where = where)
> 10 :  prototype(rowGroup = newGrouping(), colGroup = newGrouping())
> 11 :  .prototype(...)
> 12 :  newGrouping()
> 13 :  new("Grouping")
> 14 :  initialize(value, ...)
> 15 :  initialize(value, ...)
> 16 :  "@<-"(`*tmp*`, valid, value = TRUE)
> 17 :  "slot<-"(object, name, TRUE, value)
> 18 :  checkSlotAssignment(object, name, value)
> 19 :  getClass(class(obj))
> 20 :  .requirePackage(package)
> 21 :  .asEnvironmentPackage(package)
> Error in library("EpiR.algo_1.4.0") : package/namespace load
failed
> 
> 
> info: class Grouping is defined in EpiR.base and used for slot rowGroup,
> newGrouping is a simple wrapper to new("Grouping") providing some
meaningful
> defaults.
> 
> 
-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595