R devel - May 2015 - Unexpected failure when calling new() with unnamed arg and

Hi,

The man page for new() suggests that if 'a' is an object with slots
"slot1" and "slot2" and C is a class that extends the class
of 'a',
then the 2 following calls should be equivalent:

   new("C", a, ...)
   new("C", slot1=a at slot1, slot2=a at slot2, ...)

This is generally the case but I just ran into a situation where it's
not. In the following example the former fails while the latter works:

   setClass("A", representation(slot1="numeric",
slot2="logical"))
   setClass("B", contains="A",
representation(design="formula"))
   setClass("C", contains="B")
   a <- new("A", slot1=77, slot2=TRUE)

   new("C", a, design=x ~ y)  # fails
   new("C", slot1=a at slot1, slot2=a at slot2, design=x ~ y)  # works

Note that new("B", a, design=x ~ y) works so the 3-level class
hierarchy is really needed in order to reproduce.

Probably related to this, I also noted that new("B") and/or
new("C")
return invalid objects:

   c <- new("C")

   validObject(c)
   # Error in validObject(c) :
   #  invalid class ?C? object: invalid object for slot "design"
   #  in class "C": got class "S4", should be or extend
class "formula"

   is(c at design, "formula")
   # [1] FALSE

   class(c at design)
   # [1] "S4"

Note that 'c' can be fixed:

   c at design <- formula(NULL)

   validObject(c)
   # [1] TRUE

Maybe something that the default "initialize" method should take care
of?

Another singularity that is maybe at the root of all of this is that
the "formula" S4 class is virtual:

   showClass("formula")
   # Virtual Class "formula" [package "methods"]
   #
   # Slots:
   #
   # Name:   .S3Class
   # Class: character
   #
   # Extends: "oldClass"

so a bare call to new("formula") fails:

   new("formula")
   # Error in new("formula") :
   #   trying to generate an object from a virtual class ("formula")

Shouldn't new("formula") just return an "empty" S3
formula (like
formula(NULL) does), in the same way that new("integer") returns
an empty ordinary integer vector?

Thanks,
H.

 > sessionInfo()
R version 3.2.0 Patched (2015-04-17 r68202)
Platform: x86_64-unknown-linux-gnu (64-bit)
Running under: Ubuntu 14.04.2 LTS

locale:
  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
  [9] LC_ADDRESS=C               LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base

-- 
Herv? Pag?s

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319

>>>>> Herv? Pag?s <hpages at fredhutch.org>
>>>>>     on Tue, 12 May 2015 15:18:42 -0700 writes:
> Hi,
> The man page for new() suggests that if 'a' is an object with slots
> "slot1" and "slot2" and C is a class that extends the
class of 'a',
> then the 2 following calls should be equivalent:
>    new("C", a, ...)
>    new("C", slot1=a at slot1, slot2=a at slot2, ...)
> This is generally the case but I just ran into a situation where it's
> not. In the following example the former fails while the latter works:
>    setClass("A", representation(slot1="numeric",
slot2="logical"))
>    setClass("B", contains="A",
representation(design="formula"))
>    setClass("C", contains="B")
>    a <- new("A", slot1=77, slot2=TRUE)
>    new("C", a, design=x ~ y)  # fails
>    new("C", slot1=a at slot1, slot2=a at slot2, design=x ~ y)  #
works
> Note that new("B", a, design=x ~ y) works so the 3-level class
> hierarchy is really needed in order to reproduce.
> Probably related to this, I also noted that new("B") and/or
new("C")
> return invalid objects:
>    c <- new("C")
>    validObject(c)
>    # Error in validObject(c) :
>    #  invalid class ?C? object: invalid object for slot "design"
>    #  in class "C": got class "S4", should be or extend
class "formula"
>    is(c at design, "formula")
>    # [1] FALSE
>    class(c at design)
>    # [1] "S4"
> Note that 'c' can be fixed:
>    c at design <- formula(NULL)
>    validObject(c)
>    # [1] TRUE
> Maybe something that the default "initialize" method should take
care
> of?
> Another singularity that is maybe at the root of all of this is that
> the "formula" S4 class is virtual:
>    showClass("formula")
>    # Virtual Class "formula" [package "methods"]
>    #
>    # Slots:
>    #
>    # Name:   .S3Class
>    # Class: character
>    #
>    # Extends: "oldClass"
> so a bare call to new("formula") fails:
>    new("formula")
>    # Error in new("formula") :
>    #   trying to generate an object from a virtual class
("formula")
> Shouldn't new("formula") just return an "empty" S3
formula (like
> formula(NULL) does), in the same way that new("integer") returns
> an empty ordinary integer vector?
Interesting .. and at least worth following.

One problem and historical reason for the current setup seems
that the "formula" S3 class is not from 'base' but
'stats' :

R's source, src/library/methods/R/BasicClasses.R,
lines 524 ff has the following comment block

|  .OldClassesPrototypes is a list of S3 classes for which prototype
|  objects are known & reasonable.  The classes will reappear in
|  .OldClassesList, but will have been initialized first in
|  .InitBasicClasses.  NB:  the methods package will NOT set up
|  prototypes for S3 classes except those in package base and for "ts"
|  (and would rather not do those either).  The package that owns the
|  S3 class should have code to call setOldClass in its
|  initialization.

So, when John Chambers wrote this, he envisioned that the
'stats' package would do "the correct thing" for its own
classes.
OTOH, as history went, the stats package was never allowed to
depend on methods.
There are many other S3 classes from 'stats' which also end up
similarly, being defined via  setOldClass() and that itself
produces a VIRTUAL class.
Also, another part of the (R source) comment above is no longer
quite accurate, e.g., "data.frame" is in .OldClassesPrototypes
but not in .OldClassesList ...

As I do agree that "formula" is much more basic than these other
classes,
I'm currently looking at tweaks to the methods (and stats) code,
to get this to work.... as indeed - you mentioned above -  we
already allow "empty S3 formula" objects anyway.

... half an hour later : Indeed, I've been able to use the above information
such that  new("formula") and new("formula", y ~ x)  
work.

However, your code above now --- with my changes --- would  fail :

 > setClass("A", representation(slot1="numeric",
slot2="logical"))
 > setClass("B", contains="A",
representation(design="formula"))
 > setClass("C", contains="B")
 Error in reconcilePropertiesAndPrototype(name, slots, prototype, superClasses, 
:
   "B" is not eligible to be the data part of another class (must be a
basic class or a virtual class with no slots)
 > 

So, I am not yet committing my changes to R-devel.
Hopefully more on this, later today.

Martin Maechler,
ETH Zurich

> Thanks,
> H.
>  > sessionInfo()
> R version 3.2.0 Patched (2015-04-17 r68202)
> Platform: x86_64-unknown-linux-gnu (64-bit)
> Running under: Ubuntu 14.04.2 LTS
> -- 
> Herv? Pag?s
> Fred Hutchinson Cancer Research Center
 [..................]

Thanks Martin for looking into this.  H.

On 05/13/2015 03:57 AM, Martin Maechler wrote:
>>>>>> Herv? Pag?s <hpages at fredhutch.org>
>>>>>>      on Tue, 12 May 2015 15:18:42 -0700 writes:
>
>> Hi,
>
>> The man page for new() suggests that if 'a' is an object with
slots
>> "slot1" and "slot2" and C is a class that extends
the class of 'a',
>> then the 2 following calls should be equivalent:
>
>>     new("C", a, ...)
>>     new("C", slot1=a at slot1, slot2=a at slot2, ...)
>
>> This is generally the case but I just ran into a situation where
it's
>> not. In the following example the former fails while the latter works:
>
>>     setClass("A", representation(slot1="numeric",
slot2="logical"))
>>     setClass("B", contains="A",
representation(design="formula"))
>>     setClass("C", contains="B")
>>     a <- new("A", slot1=77, slot2=TRUE)
>
>>     new("C", a, design=x ~ y)  # fails
>>     new("C", slot1=a at slot1, slot2=a at slot2, design=x ~
y)  # works
>
>> Note that new("B", a, design=x ~ y) works so the 3-level
class
>> hierarchy is really needed in order to reproduce.
>
>> Probably related to this, I also noted that new("B") and/or
new("C")
>> return invalid objects:
>
>>     c <- new("C")
>
>>     validObject(c)
>>     # Error in validObject(c) :
>>     #  invalid class ?C? object: invalid object for slot
"design"
>>     #  in class "C": got class "S4", should be or
extend class "formula"
>
>>     is(c at design, "formula")
>>     # [1] FALSE
>
>>     class(c at design)
>>     # [1] "S4"
>
>> Note that 'c' can be fixed:
>
>>     c at design <- formula(NULL)
>
>>     validObject(c)
>>     # [1] TRUE
>
>> Maybe something that the default "initialize" method should
take care
>> of?
>
>> Another singularity that is maybe at the root of all of this is that
>> the "formula" S4 class is virtual:
>
>>     showClass("formula")
>>     # Virtual Class "formula" [package "methods"]
>>     #
>>     # Slots:
>>     #
>>     # Name:   .S3Class
>>     # Class: character
>>     #
>>     # Extends: "oldClass"
>
>> so a bare call to new("formula") fails:
>
>>     new("formula")
>>     # Error in new("formula") :
>>     #   trying to generate an object from a virtual class
("formula")
>
>> Shouldn't new("formula") just return an "empty"
S3 formula (like
>> formula(NULL) does), in the same way that new("integer")
returns
>> an empty ordinary integer vector?
>
> Interesting .. and at least worth following.
>
> One problem and historical reason for the current setup seems
> that the "formula" S3 class is not from 'base' but
'stats' :
>
> R's source, src/library/methods/R/BasicClasses.R,
> lines 524 ff has the following comment block
>
> |  .OldClassesPrototypes is a list of S3 classes for which prototype
> |  objects are known & reasonable.  The classes will reappear in
> |  .OldClassesList, but will have been initialized first in
> |  .InitBasicClasses.  NB:  the methods package will NOT set up
> |  prototypes for S3 classes except those in package base and for
"ts"
> |  (and would rather not do those either).  The package that owns the
> |  S3 class should have code to call setOldClass in its
> |  initialization.
>
> So, when John Chambers wrote this, he envisioned that the
> 'stats' package would do "the correct thing" for its own
classes.
> OTOH, as history went, the stats package was never allowed to
> depend on methods.
> There are many other S3 classes from 'stats' which also end up
> similarly, being defined via  setOldClass() and that itself
> produces a VIRTUAL class.
> Also, another part of the (R source) comment above is no longer
> quite accurate, e.g., "data.frame" is in .OldClassesPrototypes
> but not in .OldClassesList ...
>
> As I do agree that "formula" is much more basic than these other
classes,
> I'm currently looking at tweaks to the methods (and stats) code,
> to get this to work.... as indeed - you mentioned above -  we
> already allow "empty S3 formula" objects anyway.
>
> ... half an hour later : Indeed, I've been able to use the above
information
> such that  new("formula") and new("formula", y ~ x)
> work.
>
> However, your code above now --- with my changes --- would  fail :
>
>   > setClass("A", representation(slot1="numeric",
slot2="logical"))
>   > setClass("B", contains="A",
representation(design="formula"))
>   > setClass("C", contains="B")
>   Error in reconcilePropertiesAndPrototype(name, slots, prototype,
superClasses,  :
>     "B" is not eligible to be the data part of another class
(must be a basic class or a virtual class with no slots)
>   >
>
> So, I am not yet committing my changes to R-devel.
> Hopefully more on this, later today.
>
> Martin Maechler,
> ETH Zurich
>
>
>> Thanks,
>> H.
>
>>   > sessionInfo()
>> R version 3.2.0 Patched (2015-04-17 r68202)
>> Platform: x86_64-unknown-linux-gnu (64-bit)
>> Running under: Ubuntu 14.04.2 LTS
>
>> --
>> Herv? Pag?s
>> Fred Hutchinson Cancer Research Center
>
>   [..................]
>
>
-- 
Herv? Pag?s

Program in Computational Biology
Division of Public Health Sciences
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N, M1-B514
P.O. Box 19024
Seattle, WA 98109-1024

E-mail: hpages at fredhutch.org
Phone:  (206) 667-5791
Fax:    (206) 667-1319