R devel - Apr 2007 - package incompatibility under 2.5.0 (please respond directly, I am not on r-devel)

Dear all,

For my package "ref" I have implemented extensive regression testing.
It now fails to compile since primitives "dim" and
"dimnames" (and their assignment methods) no longer allow for
additional arguments. I was using an additional argument "ref" with
several methods. For "].refdata" it still works, with
"dim.refdata" no longer. Could you please allow for additional
arguments for the following generic functions (or primitives):

dim <- function (x, ...)
UseMethod("dim")

"dim<-" <- function (x, ..., value)
UseMethod("dim<-")

dimnames <- function (x, ...)
UseMethod("dimnames")

"dimnames<-" <- function (x, ..., value)
UseMethod("dimnames<-")

row.names <- function (x, ...)
UseMethod("row.names")

"row.names<-" <- function (x, ..., value)
UseMethod("row.names<-")

names <- function (x, ...)
UseMethod("names")

"names<-" <- function (x, ..., value)
UseMethod("names<-")

BTW: why does get("dim") returns 
function (x) .Primitive("dim")
and args() works on it, 
while get("[") returns
.Primitive("[")
and args() doesn't work on it? 

Furthermore, until now "rownames", "colnames" have been
convenience wrappers for "dimnames". Consequently implementing
"dimnames" and "dimnames<-" would indirectly implement
"rownames", "colnames" and their assignment methods. This no
longer works for classes inheriting from "data.frame" because the
assignment methods no longer work via "dimnames<-". I can imagine
that this change breaks existing code in other packages as well - without
formally throwing errors at package check time (as I said, I have unusually
strict regression testing included in the example section, that other packages
may not have). If it is really necessary to treat data.frames differently,
I'd recommend to change "rownames" and "colnames"
accordingly, in order to have symmetry between accessor and assignment
functions. That would mean defining "names" and "row.names"
and their assignment methods for any classes inheriting from data.frame, instead
of "dimnames", correct? Maybe *all* package maintainers should be
warned about this or R CMD CHECK should check whether anyone defines
"dimnames" or "dimnames<-" for any class inheriting from
"data.frame".

Best regards


Jens Oehlschl?gel


> -----Urspr?ngliche Nachricht-----
> Von: Uwe.Ligges at R-Project.org
> Gesendet: 08.04.07 16:50:29
> An: joehl at web.de
> CC: Uwe.Ligges at R-Project.org,olafm at kimberly.tako.de
> Betreff: Package ref_0.92.tar.gz did not pass R CMD check
> Dear package maintainer,
> 
> this notification has been generated automatically.
> Your package ref_0.92.tar.gz did not pass 'R CMD check' on
> Windows and will be omitted from the corresponding CRAN directory
> (CRAN/bin/windows/contrib/2.5/).
> Please check the attached log-file and consider to resubmit a version
> with increased version number that passes R CMD check on Windows.
> R version 2.5.0 alpha (2007-04-05 r41063)
> 
> All the best,
> Uwe Ligges
> (Maintainer of binary packages for Windows)
> 
> 
> <hr>
> * using log directory 'd:/Rcompile/CRANpkg/local/2.5/ref.Rcheck'
> * using R version 2.5.0 alpha (2007-04-05 r41063)
> * checking for file 'ref/DESCRIPTION' ... OK
> * this is package 'ref' version '0.92'
> * checking package dependencies ... OK
> * checking if this is a source package ... OK
> * checking whether package 'ref' can be installed ... OK
> * checking package directory ... OK
> * checking for portable file names ... OK
> * checking DESCRIPTION meta-information ... OK
> * checking top-level files ... OK
> * checking index information ... OK
> * checking package subdirectories ... OK
> * checking R files for non-ASCII characters ... OK
> * checking R files for syntax errors ... OK
> * checking whether the package can be loaded ... OK
> * checking whether the package can be loaded with stated dependencies ...
OK
> * checking for unstated dependencies in R code ... OK
> * checking S3 generic/method consistency ... WARNING
> dim:
> function(x)
> dim.refdata:
> function(x, ref)
> 
> dimnames:
> function(x)
> dimnames.refdata:
> function(x, ref)
> 
> dimnames<-:
> function(x, value)
> dimnames<-.refdata:
> function(x, ref, value)
> 
> See section 'Generic functions and methods' of the 'Writing R
Extensions'
> manual.
> * checking replacement functions ... OK
> * checking foreign function calls ... OK
> * checking R code for possible problems ... OK
> * checking Rd files ... OK
> * checking Rd cross-references ... OK
> * checking for missing documentation entries ... OK
> * checking for code/documentation mismatches ... OK
> * checking Rd \usage sections ... OK
> * creating ref-Ex.R ... OK
> * checking examples ... ERROR
> Running examples in 'ref-Ex.R' failed.
> The error most likely occurred in:
> 
> > ### * refdata
> > 
> > flush(stderr()); flush(stdout())
> > 
> > ### Name: refdata
> > ### Title: subsettable reference to matrix or data.frame
> > ### Aliases: refdata [.refdata [<-.refdata [[.refdata
[[<-.refdata
> > ### $.refdata $<-.refdata dim.refdata dim<-.refdata
dimnames.refdata
> > ### dimnames<-.refdata print.refdata
> > ### Keywords: programming manip
> > 
> > ### ** Examples
> > 
> > 
> > ## Simple usage Example
> > x <- cbind(1:5, 5:1) # take a matrix or data frame
> > rx <- refdata(x) # wrap it into an refdata object
> > rx # see the autoprinting
> refdata (matrix) with [5,2] of [5,2]
> use x[] to get the complete actual subset
> use x[...] for standard extraction
> use x[..., ref=TRUE] to get a newly indexed refdata object
> use x[...] <- value to overwrite x with a refdata object containing a
new env containing a modified dataset
> use x[..., ref=TRUE] <- value to modify the original dataset
> > rm(x) # delete original to save memory
> > rx[] # extract all data
> [,1] [,2]
> [1,] 1 5
> [2,] 2 4
> [3,] 3 3
> [4,] 4 2
> [5,] 5 1
> > rx[-1, ] # extract part of data
> [,1] [,2]
> [1,] 2 4
> [2,] 3 3
> [3,] 4 2
> [4,] 5 1
> > rx2 <- rx[-1, , ref=TRUE] # create refdata object referencing part
of data (only index, no data is duplicated)
> > rx2 # compare autoprinting
> refdata (matrix) with [4,2] of [5,2]
> use x[] to get the complete actual subset
> use x[...] for standard extraction
> use x[..., ref=TRUE] to get a newly indexed refdata object
> use x[...] <- value to overwrite x with a refdata object containing a
new env containing a modified dataset
> use x[..., ref=TRUE] <- value to modify the original dataset
> > rx2[] # extract 'all' data
> [,1] [,2]
> [1,] 2 4
> [2,] 3 3
> [3,] 4 2
> [4,] 5 1
> > rx2[-1, ] # extract part of (part of) data
> [,1] [,2]
> [1,] 3 3
> [2,] 4 2
> [3,] 5 1
> > cat("for more examples look the help pages\n")
> for more examples look the help pages
> > 
> > ## Not run: 
> > ##D # Memory saving demos
> > ##D square.matrix.size <- 1000
> > ##D recursion.depth.limit <- 10
> > ##D non.referenced.matrix <-
matrix(1:(square.matrix.size*square.matrix.size), nrow=square.matrix.size,
ncol=square.matrix.size)
> > ##D rownames(non.referenced.matrix) <- paste("a",
seq(length=square.matrix.size), sep="")
> > ##D colnames(non.referenced.matrix) <- paste("b",
seq(length=square.matrix.size), sep="")
> > ##D referenced.matrix <- refdata(non.referenced.matrix)
> > ##D recurse.nonref <- function(m, depth.limit=10){
> > ##D x <- m[1,1] # need read access here to create local copy
> > ##D gc()
> > ##D cat("depth.limit=", depth.limit, "
memory.size=", memsize.wrapper(), "\n", sep="")
> > ##D if (depth.limit)
> > ##D Recall(m[-1, -1, drop=FALSE], depth.limit=depth.limit-1)
> > ##D invisible()
> > ##D }
> > ##D recurse.ref <- function(m, depth.limit=10){
> > ##D x <- m[1,1] # read access, otherwise nothing happens
> > ##D gc()
> > ##D cat("depth.limit=", depth.limit, "
memory.size=", memsize.wrapper(), "\n", sep="")
> > ##D if (depth.limit)
> > ##D Recall(m[-1, -1, ref=TRUE], depth.limit=depth.limit-1)
> > ##D invisible()
> > ##D }
> > ##D gc()
> > ##D memsize.wrapper()
> > ##D recurse.ref(referenced.matrix, recursion.depth.limit)
> > ##D gc()
> > ##D memsize.wrapper()
> > ##D recurse.nonref(non.referenced.matrix, recursion.depth.limit)
> > ##D gc()
> > ##D memsize.wrapper()
> > ##D rm(recurse.nonref, recurse.ref, non.referenced.matrix,
referenced.matrix, square.matrix.size, recursion.depth.limit)
> > ##D 
> > ## End(Not run)
> > cat("for even more examples look at
regression.test.refdata()\n")
> for even more examples look at regression.test.refdata()
> > regression.test.refdata() # testing correctness of refdata
functionality
> testing refdata with matrix
> Error in identical(dim(rx3, ref = TRUE), dim(x)) : 
> 2 arguments passed to 'dim' which requires 1
> Execution halted
> 
>

Unfortunately a claim of 'extensive regression testing' is not going to 
help you if the original contains the mistake.  It seems that your package 
is not using these functions as they are (and always have been, AFAIK) 
documented in R or the Blue and White Books.

A summary would be that new checks in R 2.5.0 have found several 
long-standing incompatibilities of your code with R.

R 2.5.0 is in feature freeze, and changing the argument list of basic 
functions is not something we would do lightly (and certainly not at this 
late stage).  In essence you are asking everyone else to change their 
usage to conform to yours, since if we change these many other packages 
(and R itself) will show warnings in 'R CMD check'.

There is an advantage in not having a '...' argument in generics that 
Martin Maechler often points out: it means that errors in actual argument 
names are caught.  And these new checks have caught unintended usage 
errors in other packages.

On Wed, 11 Apr 2007, Jens Oehlschl?gel wrote:
> Dear all,
>
> For my package "ref" I have implemented extensive regression
testing. It
> now fails to compile since primitives "dim" and
"dimnames" (and their
> assignment methods) no longer allow for additional arguments. I was 
> using an additional argument "ref" with several methods. For
"].refdata"
If you mean '[.refdata', '[' is really part of the language and
documented
to allow zero or more arguments.
> it still works, with "dim.refdata" no longer. Could you please
allow for
> additional arguments for the following generic functions (or 
> primitives):
>
> dim <- function (x, ...)
> UseMethod("dim")
>
> "dim<-" <- function (x, ..., value)
> UseMethod("dim<-")
>
> dimnames <- function (x, ...)
> UseMethod("dimnames")
>
> "dimnames<-" <- function (x, ..., value)
> UseMethod("dimnames<-")
>
> row.names <- function (x, ...)
> UseMethod("row.names")
>
> "row.names<-" <- function (x, ..., value)
> UseMethod("row.names<-")
>
> names <- function (x, ...)
> UseMethod("names")
>
> "names<-" <- function (x, ..., value)
> UseMethod("names<-")
>
> BTW: why does get("dim") returns
> function (x) .Primitive("dim")
> and args() works on it,
> while get("[") returns
> .Primitive("[")
> and args() doesn't work on it?
It is regarded as part of the language (like '{' and '(', which
equally
are primitives).  It would be unusual to apply them as functions, and 
args() applies to functions.
> Furthermore, until now "rownames", "colnames" have been
convenience
> wrappers for "dimnames". Consequently implementing
"dimnames" and
> "dimnames<-" would indirectly implement "rownames",
"colnames" and their
> assignment methods. This no longer works for classes inheriting from 
> "data.frame" because the assignment methods no longer work via 
> "dimnames<-". I can imagine that this change breaks existing
code in
> other packages as well - without formally throwing errors at package 
> check time (as I said, I have unusually strict regression testing 
> included in the example section, that other packages may not have).
> If it is really necessary to treat data.frames differently, I'd
It is.
> recommend to change "rownames" and "colnames"
accordingly, in order to
> have symmetry between accessor and assignment functions. That would mean 
> defining "names" and "row.names" and their assignment
methods for any
> classes inheriting from data.frame, instead of "dimnames",
correct?
> Maybe *all* package maintainers should be warned about this or R CMD 
> CHECK should check whether anyone defines "dimnames" or
"dimnames<-" for
> any class inheriting from "data.frame".
The documentation says (and used to say)

      For a data frame, 'rownames' and
      'colnames' are equivalent to 'row.names' and
'names' respectively,
      but the latter are preferred (and can be much faster).

and it now works as documented.
>
> Best regards
>
>
> Jens Oehlschl?gel
>
>
>
>> -----Urspr?ngliche Nachricht-----
>> Von: Uwe.Ligges at R-Project.org
>> Gesendet: 08.04.07 16:50:29
>> An: joehl at web.de
>> CC: Uwe.Ligges at R-Project.org,olafm at kimberly.tako.de
>> Betreff: Package ref_0.92.tar.gz did not pass R CMD check
>
>
>> Dear package maintainer,
>>
>> this notification has been generated automatically.
>> Your package ref_0.92.tar.gz did not pass 'R CMD check' on
>> Windows and will be omitted from the corresponding CRAN directory
>> (CRAN/bin/windows/contrib/2.5/).
>> Please check the attached log-file and consider to resubmit a version
>> with increased version number that passes R CMD check on Windows.
>> R version 2.5.0 alpha (2007-04-05 r41063)
[...]
>> Error in identical(dim(rx3, ref = TRUE), dim(x)) :
>> 2 arguments passed to 'dim' which requires 1
>> Execution halted
-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595