R devel - Apr 1999 - R's UseMethod() does not dispatch on changed class() (PR#167)

(opening new bug report thread, on suggestion of BDR ..)

I said on vaguely related topic:

 MM> There's  another long-standing  
 MM> UseMethod / NextMethod / Dispatch problem that I've never
traced/solved..
 MM>  
 MM> Look at the  $RHOME/tests/mode-methods.R   file !

Peter Dalgaard already remarked :

 PD> ..i.e. this:

 PD> abc <- function(x, ...) {
 PD>     if (is.null(class(x))) class(x) <- data.class(x)
 PD>     UseMethod("abc")
 PD> }
 PD> 
 PD> will dispatch on the data class in S but not in R.
 PD> 
 PD> This puzzles me, because normally you can't mess with the arguments
of
 PD> a generic function, in *either* language:
 PD> 
 PD> > cde<-function(x,...){x<-x*2;UseMethod("print")}
 PD> > cde(2)
 PD> [1] 2
 PD> 
 PD> So, either it's because class<- is non-duplicating in S, or S is
doing
 PD> something *really* strange here. 
 PD> 
 PD> Hmm, then again: class<- *is* non-duplicating in R:
 PD> 
 PD> > x<-1
 PD> > y<-.Alias(x)
 PD> > class(x)<-'foo'
 PD> > y
 PD> [1] 1
 PD> attr(,"class")
 PD> [1] "foo"
 PD> 
 PD> I think this is related to evaluation of promises:
 PD> 
 PD> > f<-function(x)substitute(x)
 PD> > f(sin(1))
 PD> sin(1)
 PD> > f<-function(x){x;substitute(x)}
 PD> > f(sin(1))
 PD> sin(1)
 PD> > f<-function(x){class(x)<-'foo';substitute(x)}
 PD> > f(sin(1))
 PD> [1] 0.841471
 PD> attr(,"class")
 PD> [1] "foo"
 PD> 
 PD> Notice that just forcing evaluation of x does not destroy the link
 PD> between x and the argument expression, but changing its class does.
 PD> This is probably what shouldn't happen.
 PD> 
 PD> You can get the dispatching done by using
 PD> 
 PD> abc <- function(x, ...) {
 PD>     if (is.null(class(x))) {
 PD> 	class(x) <- data.class(x)
 PD> 	abc(x, ...)
 PD>     } else
 PD>         UseMethod("abc")
 PD> }
 PD> 
 PD> which (seemingly) works in both languages, at least as long as you
 PD> don't have to mess with sys.parent()

So we don't know if this is a bug in R.
However, R is behaving inexpectedly to ``Joe Average-User''
and at least we should know if (and why)
we want to be different than S here.

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

I'm pretty sure we've been through this one several times already,
although probably in different guises.
The discussion in the white book, section A.6 is rather specific about
this. And as near as I can tell is contrary to what S does. I don't think
that you want to do what S does.

Do you really want to have a language that accepts any change in x?
It is rather hard to see how it's object oriented then!
Or, alternatively do you really want to see if the only thing that
has changed about x is its class and that will be ok?
What about the other arguments?
The whole point of UseMethod is that we got an object x, with a
particular class and now we're going to do the right thing.
If you wanted to do the right thing with an object of a different
class then you need to do that before you call UseMethod.

Basically, one of the differences between UseMethod and NextMethod is
that with the first you don't get to change anything in the body of
the function, or at least, if you do then it will be ignored. If you
want to change things, do so and then call the generic.

UseMethod believes that the x in the call was evaluated and will
use that value in the call to the method.
NextMethod looks in the local environment for an x, uses one if it
finds one (otherwise gets the x in the call).
Semantically different, well documented and if Joe User doesn't
understand it's because S doesn't do what it says it does and
people write bad code (but generally not as bad as mine ;-).


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

On 15 Apr 1999, Peter Dalgaard BSA wrote:
> Robert Gentleman <rgentlem@hsph.harvard.edu> writes:
> 
> > Do you really want to have a language that accepts any change in x?
> > It is rather hard to see how it's object oriented then!
> > Or, alternatively do you really want to see if the only thing that
> > has changed about x is its class and that will be ok?
> > What about the other arguments?
> > The whole point of UseMethod is that we got an object x, with a
> > particular class and now we're going to do the right thing.
> > If you wanted to do the right thing with an object of a different
> > class then you need to do that before you call UseMethod.
> 
> I'm inclined to agree here. On the other hand, what might be the case
> is that you could make class<- be a little more careful with its
> argument so that UseMethod would see the modified x. Sneaky, but
> compatible, and *perhaps* not very hard to do. How often is this kind
> of construct (class(x)<-foo;UseMethod()) used in actual S code?
Often by certain people with a massive presence!

We have indeed been here before.  In our lda function we do

class(x) <- data.class(x)
UseMethod("lda")

That fails to use the new class in R, and according to `your' reading of
the White book (but not mine nor apparently the author of S's) should do
so. `You' suggested

   UseMethod("lda", x, ...)

which works in some versions of S-PLUS 3.x but crashes the DEC Alpha
version and works on no version of S-PLUS 5.x. Hence I have

lda <- function(x, ...)
{
   if(is.null(class(x))) class(x) <- data.class(x)
#ifdef R
   UseMethod("lda", x, ...)
#ifdef SP3
   UseMethod("lda")
}

and in SP5 I do something with new-style classes.
> - and Joe User generally gets in trouble with methods anyway...
and not just Joe User....

Brian

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

On Wed, 14 Apr 1999, Robert Gentleman wrote:
> On Wed, 14 Apr 1999, Prof Brian D Ripley wrote:
> 
> > On 15 Apr 1999, Peter Dalgaard BSA wrote:
> > 
> > > Robert Gentleman <rgentlem@hsph.harvard.edu> writes:
> > > 
> > > > Do you really want to have a language that accepts any
change in x?
> > > > It is rather hard to see how it's object oriented then!
> > > > Or, alternatively do you really want to see if the only
thing that
> > > > has changed about x is its class and that will be ok?
> > > > What about the other arguments?
> > > > The whole point of UseMethod is that we got an object x,
with a
> > > > particular class and now we're going to do the right
thing.
> > > > If you wanted to do the right thing with an object of a
different
> > > > class then you need to do that before you call UseMethod.
> > > 
> > > I'm inclined to agree here. On the other hand, what might be
the case
> > > is that you could make class<- be a little more careful with
its
> > > argument so that UseMethod would see the modified x. Sneaky, but
> > > compatible, and *perhaps* not very hard to do. How often is this
kind
> > > of construct (class(x)<-foo;UseMethod()) used in actual S
code?
> > 
> > Often by certain people with a massive presence!
> > 
> > We have indeed been here before.  In our lda function we do
> > 
> > class(x) <- data.class(x)
> > UseMethod("lda")
> > 
> > That fails to use the new class in R, and according to `your'
reading of
> > the White book (but not mine nor apparently the author of S's)
should do
> > so. `You' suggested
> 
>  If you could point out the error of my ways I would appreciate it.
I did not mean to imply that you were wrong, rather that the White Book
can be read both ways (and it seems has been).
> But, it as I understand it, UseMethod is supposed to call the method
> with the arguments the same as those that called the generic. You
> must evaluate x to figure out what method to call, but I don't see
> how you can change x (adding classes is changing it).
Then why are

UseMethod("foo")
UseMethod("foo", x)

different in R? (They are: lda works.) The actual definition of UseMethod
in S is

       UseMethod(generic, object, ...)

and by the standard evaluation model, `object' is subject to lazy
evaluation. Nothing I can see in the White Book says otherwise. Indeed,
page 467 says the arguments are `re-matched by the standard rules'. It then
says `The method will see argument matches as it would if the user's call
had been directly to the method'.  I think those claims are contradictory,
and in particular whichever way they are read suggest my pair of calls
should be the same.

[In S UseMethod("foo") is semantic shorthand for
UseMethod("foo", <name of
first argument of caller>), according to the help page.]

To make matters worse, in S it is a common mistake to have

foo <- function(x, ...) UseMethod("foo")
foo.lm <- function(y, ...) { .... }

and that change of name of argument causes re-evaluation.  What happens in
R?  This makes an apparently small point, that generics should have the
same argument name as S, potentially rather important. I know of four
outstanding discrepancies:

R: deviance <- function (x, ...) UseMethod("deviance")
S: deviance <- function (x, ...) UseMethod("deviance")

and ditto for coef, fitted, residuals (which I had not realized, in part
because some of these only recently acquited defaults).

There are inconsistencies in the current R code: any objections if
I change these to uniformly follow S and use `object'?


The other common one is `print'. As in
> print
function (x, ...) 
UseMethod("print")
> print.socket
function (socket) 
{
    cat("Socket connection #", socket$socket, "to",
socket$host,
        "on port", socket$port, "\n")
}

print methods should have first argument x, a ... argument _and_ return
invisible(x) for an unchanged x (in S, at least).  print.plot
does not even have an argument, so how does that ever work?
>   I realize it is frustrating and if you think it's important
> you should change it. But, by doing that you move the language
> further away from being object oriented. 
What we do need is to get this documented!  In particular, in R
> ?UseMethod
Class Methods

     UseMethod(name)

is incomplete, and I only know about UseMethod("foo", x, ...) because
`you' (Kurt?) suggested I use it. 

BTW, I have never believed S3's claim to be `object-oriented': it has a
class-based dispatcher. S has changed (although it remains to be seen if
users will follow) and I would be happy for R to be different from both.
However, as Bill and I are currently trying to write about this for V&R3v2,
I would not like R to change dramatically after the next few months!


Brian

-- 
Brian D. Ripley,                  ripley@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272860 (secr)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595


-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

I've done some editing to get things together (hopefully without
changing the semantics)

On Thu, 15 Apr 1999, Prof Brian D Ripley wrote:
> On Wed, 14 Apr 1999, Robert Gentleman wrote:
> 
> > On Wed, 14 Apr 1999, Prof Brian D Ripley wrote:
> > 
> > > On 15 Apr 1999, Peter Dalgaard BSA wrote:
> > > 
> > > > Robert Gentleman <rgentlem@hsph.harvard.edu> writes:
> > > > 
> 
> I did not mean to imply that you were wrong, rather that the White Book
> can be read both ways (and it seems has been).
> 
> > But, it as I understand it, UseMethod is supposed to call the method
> > with the arguments the same as those that called the generic. You
> > must evaluate x to figure out what method to call, but I don't see
> > how you can change x (adding classes is changing it).
> 
> Then why are
> 
> UseMethod("foo")
> UseMethod("foo", x)
> different
> 
> 
> and by the standard evaluation model, `object' is subject to lazy
> evaluation. Nothing I can see in the White Book says otherwise. Indeed,
> page 467 says the arguments are `re-matched by the standard rules'. It
then
> says `The method will see argument matches as it would if the user's
call
> had been directly to the method'.  I think those claims are
contradictory,
> and in particular whichever way they are read suggest my pair of calls
> should be the same.
> 
> [In S UseMethod("foo") is semantic shorthand for
UseMethod("foo", <name of
> first argument of caller>), according to the help page.]
> 
  Now, with my reading, the method will see the argument as it would
if the users call had been directly to the the method, coupled with
the later comment (top of p. 469) that for NextMethod you must take local
bindings for the arguments, yields
   with UseMethod you don't take local bindings, the method gets what
the generic did, but it will rematch (argument names and positions
may have changed). That means you need to ignore what happened
inside of the generic.

  Now, we have the variable definition problem raise it's head,
  UseMethod("foo")
  UseMethod("foo",x)
  If the second version doesn't use the local binding of x then
 UseMethod behaves differently, in terms of how it treats its arguments
 than every other function (which is ok, we just need to be specific
 about it).
  So we have to make a decision, UseMethod does what S does (which is
 fine) or it doesn't.
> To make matters worse, in S it is a common mistake to have
> 
> foo <- function(x, ...) UseMethod("foo")
> foo.lm <- function(y, ...) { .... }
> 
> and that change of name of argument causes re-evaluation.  What happens in
> R?  
  The standard argument matching system takes over. If there was an
argument y=blah, in the ... then it will get matched to y in foo.lm
even though something different was matched to x in the call to the 
generic. I don't believe that a reevaluation takes place in either 
situation. If x was evaluated and that is what is matched then we
look in the value side of the expression, see that it's there and
carry on. If we match to something new then the standard rules take
over.
  Promises are basically cells with three slots, an unevaluated
expression, an environment in which to carry out the evaluation
of that expression and a value. When we want a value, we look in
the value slot first, if there is no value, we evaluate the
expression and stick the value into the value slot. So promises
should never be evaluated twice.

 For those not bored, here is a difference between R and S that 
we may not want.
bar<-function(w,...) {
    UseMethod("bar")
  }
 bar.foo<-function(y, w) print(w)
x<-30
class(x)<-"foo"

Now in R,
> bar(x,10)
[1] 10
> bar(x,y=10)
[1] 30
attr(,"class")
[1] "foo"

Notice that if the arguments are not named we use default matching and
that the symbolic name for the first argument in the generic is not
carried over.

In S bar(x,10) and bar(x,y=10) both evaluate to the same thing,
[1] 30
attr(,"class")
[1] "foo"
it appears that the name w has been applied to the first argument and
that is carried with it for subsequent evaluations.

  I'm not sure what you meant by "the change of name causes
reevaluation,
when I try
> bar(x<-x+1)
I get,
[1] 31
attr(, "class"):
[1] "foo"
[1] 31
attr(, "class"):
[1] "foo"
  which is only one evaluation (and the same in R)
> This makes an apparently small point, that generics should have the
> same argument name as S, potentially rather important. I know of four
> outstanding discrepancies:
> 
> R: deviance <- function (x, ...) UseMethod("deviance")
> S: deviance <- function (x, ...) UseMethod("deviance")
> 
> and ditto for coef, fitted, residuals (which I had not realized, in part
> because some of these only recently acquited defaults).
> 
> There are inconsistencies in the current R code: any objections if
> I change these to uniformly follow S and use `object'?
  Change to your hearts content. I think we want something that works
and that makes sense.
> What we do need is to get this documented!  In particular, in R
> 
> > ?UseMethod
> 
> Class Methods
> 
>      UseMethod(name)
> 

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._

Robert Gentleman <rgentlem@hsph.harvard.edu> writes:
>   Does that mean that our implementaion of substitute is not quite
> right? The documentation suggests that you get the expression typed as
> the value of a formal argument; right now we're getting the current
> value.
No. It means that S's implementation is quite wrong...

R's substitute works by fairly clean rules of (lazy) evaluation: If
you pass it a an expression containing x and x is a PROMSXP, you get
the argument expression. If you change x before the substitute, you get
the current value (unless you used x<-delay(something), in which case
you get the unevaluated something). 

S does --- something else:
> f<-function(x){x<-2;substitute(x)}
> f(5)
[1] 2
> y<-5     
> f(y)
y

This, I *really* don't think we want to emulate.

The documentation is badly off, though... and the quote() doc is
really off target. (I've written about this before, but it never found
its way to the help page, obviously).

-- 
   O__  ---- Peter Dalgaard             Blegdamsvej 3  
  c/ /'_ --- Dept. of Biostatistics     2200 Cph. N   
 (*) \(*) -- University of Copenhagen   Denmark      Ph: (+45) 35327918
~~~~~~~~~~ - (p.dalgaard@biostat.ku.dk)             FAX: (+45) 35327907

-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-
r-devel mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html
Send "info", "help", or "[un]subscribe"
(in the "body", not the subject !)  To:
r-devel-request@stat.math.ethz.ch
_._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._