Consider the following example: # substitute a with b in the indicated function. Seems to work.> z <- substitute( function()a+1, list(a=quote(b)) ) > zfunction() b + 1 # z is an object of class call so use eval # to turn it into an object of class expression; however, # when z is evaluated, the variable a returns.> eval(z)function()a+1 Why did a suddenly reappear again after it had already been replaced?
I left out the brackets in my last email but the problem (a reappears after have been substituted out) still remains:> z <- substitute( function(){a+1}, list(a=quote(b)) ) > zfunction() { b + 1 }> eval(z)function(){a+1} --- Date: Wed, 17 Mar 2004 20:10:43 -0500 (EST) From: Gabor Grothendieck <ggrothendieck at myway.com> [ Add to Address Book | Block Address | Report as Spam ] To: <R-help at stat.math.ethz.ch> Subject: [R] substitute question Consider the following example: # substitute a with b in the indicated function. Seems to work.> z <- substitute( function()a+1, list(a=quote(b)) ) > zfunction() b + 1 # z is an object of class call so use eval # to turn it into an object of class expression; however, # when z is evaluated, the variable a returns.> eval(z)function()a+1 Why did a suddenly reappear again after it had already been replaced?
Tony, Thomas. Thanks for your help. Your comments were very useful. Unfortunately, my next step gives me a new round of problems. The following is the same as the last example except that instead of hard coding the function into the expression I wanted to pass it to the expression. It seems like one has to do a double substitute to get this effect but I am having problems getting this to work. In the code below we first show that the keep.source option has been set to FALSE so that the source attribute does not mislead us. Then we do a double substitute. The outer substitute just creates the substitute that was in my previous question. This outer substitute produces z, a call object. So far its as expected. We then do ze <- eval(z) but we get a function object right away. I was expecting that we get an expression object. Even worse, the function does not have a replaced with b -- even though this did work in the previous example that I posted. What's wrong?> options()$keep.source[1] FALSE> f <- function(){a+1} > z <- substitute(substitute(f,list(a=quote(b))),list(f=f)) > class(z)[1] "call"> as.list(z)[[1]] substitute [[2]] function () { a + 1 } [[3]] list(a = quote(b))> ze <- eval(z) > class(ze)[1] "function"> zefunction () { a + 1 }> attr(ze,"source")NULL>--- Date: Thu, 18 Mar 2004 09:00:02 -0800 (PST) From: Thomas Lumley <tlumley at u.washington.edu> To: Gabor Grothendieck <ggrothendieck at myway.com> Cc: <R-help at stat.math.ethz.ch> Subject: RE: [R] substitute question On Wed, 17 Mar 2004, Gabor Grothendieck wrote:> > > I left out the brackets in my last email but the problem > (a reappears after have been substituted out) still remains: > > > z <- substitute( function(){a+1}, list(a=quote(b)) ) > > z > function() { > b + 1 > } > > eval(z) > function(){a+1}Interesting. Appearances are misleading, however:> z <- substitute( function(){a+1}, list(a=quote(b)) ) > zfunction() { b + 1 }> f<-eval(z) > f()Error in f() : Object "b" not found> ffunction(){a+1}> attr(f,"source")<-NULL > ffunction () { b + 1 } So it isn't that eval(z) has a+1 inside, it just has a "source" attribute with a+1. Looking more carefully at z> as.list(z)[[1]] `function` [[2]] NULL [[3]] { b + 1 } [[4]] [1] "function(){a+1}" so the original construction of z has kept the source (not, however, as a "source" attribute). There is method to our madness here. It is impossible (or at least too complicated) to keep comments in the right place as a function is parsed and deparsed. In the old days, comments would occasionally move around, sometimes in very misleading ways (IIRC with if(){}else{} cases) Now we keep a copy of the source code with functions created interactively or with source(), and drop the comments on parsing. This is controlled by options("keep.source"). If you do a lot of substitute()-style programming you may want options(keep.source=FALSE). -thomas PS: There are, of course, interesting possibilities for creative abuse of the source attribute.... --- Date: Thu, 18 Mar 2004 08:41:54 -0700 From: Tony Plate <tplate at blackmesacapital.com> To: <ggrothendieck at myway.com>, <R-help at stat.math.ethz.ch> Subject: RE: [R] substitute question This is because of the saved attribute "source" on z (that doesn't get printed out before evaluating z, because z is not yet then a function). To complete your example:> z <- substitute( function(){a+1}, list(a=quote(b)) ) > zfunction() { b + 1 }> eval(z)function(){a+1}> ze <- eval(z) > attributes(ze)$source [1] "function(){a+1}"> attr(ze, "source") <- NULL > zefunction () { b + 1 }>I previously wrote on this topic: Date: Fri, 24 Oct 2003 09:42:55 -0600 To: R-help at stat.math.ethz.ch, Peter Dalgaard <p.dalgaard at biostat.ku.dk> From: Tony Plate <tplate at blackmesacapital.com> Subject: Re: [R] what's going on here with substitute() ? Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Peter, thank you for the explanation. This is indeed what is happening. Might I suggest the following passage for inclusion in the help page for "function", and possibly also "body", in the DETAILS or WARNING section: "Note that the text of the original function definition is saved as an attribute "source" on the function, and this is printed out when the function is printed. Hence, if the function body is changed in some way other than by assigning a value via body() (which removes the "source" attribute), the printed form of the function may not be the same as the actual function body." Something along these lines could also go in the help for "eval", though if it were only there it might be very difficult to find if one were trying to look up puzzling behavior of a function. Here is a transcript that shows what is happening, with another suggestion following it.> eval(substitute(this.is.R <- function() X,list(X=!is.null(options("CRAN")[[1]]))))> this.is.Rfunction() X> body(this.is.R)[1] TRUE> attributes(this.is.R)$source [1] "function() X"> attributes(this.is.R) <- NULL > this.is.Rfunction () TRUE> # the "source" attribute comes from function definition: > attributes(function() X)$source [1] "function() X"> # and seems to be added by "eval": > attr(eval(parse(text="function() TRUE")[[1]]), "source")[1] "function() TRUE">> # we can assign bogus "source" > attr(this.is.R, "source") <- "a totally bogus function body" > this.is.Ra totally bogus function body> # assigning to body() removes "source" > body(this.is.R) <- list(666) > this.is.Rfunction () 666> attr(this.is.R, "source")NULL>An even better approach might be something that gave a warning on printing if the parsed "source" attribute was not identical to the language object being printed. This would probably belong in the code for "case LANGSXP:" in the function PrintValueRec in main/print.c (if it were written in R, I could contribute a patch, but right now I don't have time to try to understand the C there.) R code to do the test could be something like this:> f <- this.is.R > identical(f, eval(parse(text=attr(f, "source"))[[1]]))[1] FALSE> f <- function() TRUE > identical(f, eval(parse(text=attr(f, "source"))[[1]]))[1] TRUE>-- Tony Plate At Wednesday 08:09 PM 3/17/2004, Gabor Grothendieck wrote:>I left out the brackets in my last email but the problem >(a reappears after have been substituted out) still remains: > > > z <- substitute( function(){a+1}, list(a=quote(b)) ) > > z >function() { > b + 1 >} > > eval(z) >function(){a+1} > > > >--- >Date: Wed, 17 Mar 2004 20:10:43 -0500 (EST) >From: Gabor Grothendieck <ggrothendieck at myway.com> >[ Add to Address Book | Block Address | Report as Spam ] >To: <R-help at stat.math.ethz.ch> >Subject: [R] substitute question > > > > > > >Consider the following example: > ># substitute a with b in the indicated function. Seems to work. > > z <- substitute( function()a+1, list(a=quote(b)) ) > > z >function() b + 1 > ># z is an object of class call so use eval ># to turn it into an object of class expression; however, ># when z is evaluated, the variable a returns. > > eval(z) >function()a+1 > >Why did a suddenly reappear again after it had already been replaced?
On Thu, 18 Mar 2004, Gabor Grothendieck wrote:> > Tony, Thomas. Thanks for your help. Your comments were very > useful. > > Unfortunately, my next step gives me a new round of problems. > > The following is the same as the last example except that instead > of hard coding the function into the expression I wanted to > pass it to the expression. It seems like one has to do a double > substitute to get this effect but I am having problems getting this > to work. >The problem is that f is a function, not an expression. You need to work with body(f) Either> f<-function(){a+1}>body(f)<-do.call("substitute",list(body(f),list(a=quote(b))))> f function () { b + 1 } or> f<-function(){a+1} > body(f)<-eval(substitute(substitute(expr,list(a=quote(b))),list(expr=body(f)))) > ffunction () { b + 1 } -thomas
Anyone Knows How Shoud I Build Rasch Models (Partial Credit Models) In R? []s Leonard Assis Estat?stico - CONFE 7439
Tony, Thomas, Thanks, again.
If the problem with my last example was just that I was passing a function
rather than an unevaluated expression, then why don't the following
return f with b in place of a? In both cases, a is still there in the
final output.
f <- function() { a + 1 }
z <-
substitute(substitute(f=f,list(a=quote(b))),list(f=parse(text=deparse(f))))
eval(eval(z))
or
f <- function() { a + 1 }
z <- substitute(substitute(expression(f),list(a=quote(b))),list(f=f))
eval(eval(eval(z)))
---
Date: Thu, 18 Mar 2004 11:29:39 -0800 (PST)
From: Thomas Lumley <tlumley at u.washington.edu>
To: Gabor Grothendieck <ggrothendieck at myway.com>
Cc: <tplate at blackmesacapital.com>, <R-help at
stat.math.ethz.ch>
Subject: RE: [R] substitute question
On Thu, 18 Mar 2004, Gabor Grothendieck wrote:
>
> Tony, Thomas. Thanks for your help. Your comments were very
> useful.
>
> Unfortunately, my next step gives me a new round of problems.
>
> The following is the same as the last example except that instead
> of hard coding the function into the expression I wanted to
> pass it to the expression. It seems like one has to do a double
> substitute to get this effect but I am having problems getting this
> to work.
>
The problem is that f is a function, not an expression. You need to work
with body(f)
Either> f<-function(){a+1}>
body(f)<-do.call("substitute",list(body(f),list(a=quote(b))))> f
function ()
{
b + 1
}
or> f<-function(){a+1}
>
body(f)<-eval(substitute(substitute(expr,list(a=quote(b))),list(expr=body(f))))
> f
function ()
{
b + 1
}
-thomas
---
Date: Thu, 18 Mar 2004 12:15:45 -0700
From: Tony Plate <tplate at blackmesacapital.com>
To: <ggrothendieck at myway.com>
Cc: <R-help at stat.math.ethz.ch>
Subject: RE: [R] substitute question
Gabor, I suspect you are overlooking the fact that substitute() does not
evaluate its first argument. So, in the previous example you were giving
an unevaluated expression as the first argument of substitute(). In your
most recent example you are supplying a function object as the first
argument of substitute() (by supplying it as the value to be substituted
for f). I think you can get what you want by first doing f <-
Quote(function(){a+1}):
> f <- Quote(function(){a+1})
> f
function() {
a + 1
}> z <- substitute(substitute(f,list(a=quote(b))),list(f=f))
> z
substitute(function() {
a + 1
}, list(a = quote(b)))> eval(z)
function() {
b + 1
}> eval(eval(z)) # this displays the "source" attribute
function(){a+1}> body(eval(eval(z)))
{
b + 1
}>
hope this helps,
Tony Plate
At Thursday 12:00 PM 3/18/2004, you wrote:
>Tony, Thomas. Thanks for your help. Your comments were very
>useful.
>
>Unfortunately, my next step gives me a new round of problems.
>
>The following is the same as the last example except that instead
>of hard coding the function into the expression I wanted to
>pass it to the expression. It seems like one has to do a double
>substitute to get this effect but I am having problems getting this
>to work.
>
>In the code below we first show that the keep.source option has been
>set to FALSE so that the source attribute does not mislead us.
>
>Then we do a double substitute. The outer substitute just creates
>the substitute that was in my previous question. This outer
>substitute produces z, a call object. So far its as expected.
>
>We then do ze <- eval(z) but we get a function object right away. I was
>expecting that we get an expression object. Even worse, the function does
not
>have a replaced with b -- even though this did work in the previous
>example
>that I posted.
>
>What's wrong?
>
>
> > options()$keep.source
>[1] FALSE
> > f <- function(){a+1}
> > z <- substitute(substitute(f,list(a=quote(b))),list(f=f))
> > class(z)
>[1] "call"
> > as.list(z)
>[[1]]
>substitute
>
>[[2]]
>function ()
>{
> a + 1
>}
>
>[[3]]
>list(a = quote(b))
>
> > ze <- eval(z)
> > class(ze)
>[1] "function"
> > ze
>function ()
>{
> a + 1
>}
> > attr(ze,"source")
>NULL
> >
>
>---
>
>Date: Thu, 18 Mar 2004 09:00:02 -0800 (PST)
>From: Thomas Lumley <tlumley at u.washington.edu>
>To: Gabor Grothendieck <ggrothendieck at myway.com>
>Cc: <R-help at stat.math.ethz.ch>
>Subject: RE: [R] substitute question
>
>
>On Wed, 17 Mar 2004, Gabor Grothendieck wrote:
>
> >
> >
> > I left out the brackets in my last email but the problem
> > (a reappears after have been substituted out) still remains:
> >
> > > z <- substitute( function(){a+1}, list(a=quote(b)) )
> > > z
> > function() {
> > b + 1
> > }
> > > eval(z)
> > function(){a+1}
>
>
>Interesting.
>
>Appearances are misleading, however:
> > z <- substitute( function(){a+1}, list(a=quote(b)) )
> > z
>function() {
>b + 1
>}
> > f<-eval(z)
> > f()
>Error in f() : Object "b" not found
> > f
>function(){a+1}
> > attr(f,"source")<-NULL
> > f
>function ()
>{
>b + 1
>}
>
>So it isn't that eval(z) has a+1 inside, it just has a
"source" attribute
>with a+1.
>
>Looking more carefully at z
> > as.list(z)
>[[1]]
>`function`
>
>[[2]]
>NULL
>
>[[3]]
>{
>b + 1
>}
>
>[[4]]
>[1] "function(){a+1}"
>
>so the original construction of z has kept the source (not, however, as a
>"source" attribute).
>
>There is method to our madness here. It is impossible (or at least too
>complicated) to keep comments in the right place as a function is
>parsed and deparsed. In the old days, comments would occasionally move
>around, sometimes in very misleading ways (IIRC with if(){}else{} cases)
>
>Now we keep a copy of the source code with functions created interactively
>or with source(), and drop the comments on parsing. This is controlled by
>options("keep.source").
>
>If you do a lot of substitute()-style programming you may want
>options(keep.source=FALSE).
>
> -thomas
>
>PS: There are, of course, interesting possibilities for creative abuse of
>the source attribute....
>
>---
>
>Date: Thu, 18 Mar 2004 08:41:54 -0700
>From: Tony Plate <tplate at blackmesacapital.com>
>To: <ggrothendieck at myway.com>, <R-help at stat.math.ethz.ch>
>Subject: RE: [R] substitute question
>
>
>This is because of the saved attribute "source" on z (that
doesn't get
>printed out before evaluating z, because z is not yet then a function).
>
>To complete your example:
>
> > z <- substitute( function(){a+1}, list(a=quote(b)) )
> > z
>function() {
>b + 1
>}
> > eval(z)
>function(){a+1}
> > ze <- eval(z)
> > attributes(ze)
>$source
>[1] "function(){a+1}"
>
> > attr(ze, "source") <- NULL
> > ze
>function ()
>{
>b + 1
>}
> >
>
>I previously wrote on this topic:
>
>Date: Fri, 24 Oct 2003 09:42:55 -0600
>To: R-help at stat.math.ethz.ch, Peter Dalgaard <p.dalgaard at
biostat.ku.dk>
>From: Tony Plate <tplate at blackmesacapital.com>
>Subject: Re: [R] what's going on here with substitute() ?
>Mime-Version: 1.0
>Content-Type: text/plain; charset="us-ascii"; format=flowed
>
>Peter, thank you for the explanation. This is indeed what is happening.
>
>Might I suggest the following passage for inclusion in the help page for
>"function", and possibly also "body", in the DETAILS or
WARNING section:
>
>"Note that the text of the original function definition is saved as an
>attribute "source" on the function, and this is printed out when
the
>function is printed. Hence, if the function body is changed in some way
>other than by assigning a value via body() (which removes the
"source"
>attribute), the printed form of the function may not be the same as the
>actual function body."
>
>Something along these lines could also go in the help for "eval",
though if
>it were only there it might be very difficult to find if one were trying to
>look up puzzling behavior of a function.
>
>Here is a transcript that shows what is happening, with another suggestion
>following it.
>
> > eval(substitute(this.is.R <- function() X,
>list(X=!is.null(options("CRAN")[[1]]))))
> > this.is.R
>function() X
> > body(this.is.R)
>[1] TRUE
> > attributes(this.is.R)
>$source
>[1] "function() X"
> > attributes(this.is.R) <- NULL
> > this.is.R
>function ()
>TRUE
> > # the "source" attribute comes from function definition:
> > attributes(function() X)
>$source
>[1] "function() X"
> > # and seems to be added by "eval":
> > attr(eval(parse(text="function() TRUE")[[1]]),
"source")
>[1] "function() TRUE"
> >
>
> > # we can assign bogus "source"
> > attr(this.is.R, "source") <- "a totally bogus
function body"
> > this.is.R
>a totally bogus function body
> > # assigning to body() removes "source"
> > body(this.is.R) <- list(666)
> > this.is.R
>function ()
>666
> > attr(this.is.R, "source")
>NULL
> >
>
>An even better approach might be something that gave a warning on printing
>if the parsed "source" attribute was not identical to the language
object
>being printed. This would probably belong in the code for "case
LANGSXP:"
>in the function PrintValueRec in main/print.c (if it were written in R, I
>could contribute a patch, but right now I don't have time to try to
>understand the C there.) R code to do the test could be something like this:
>
> > f <- this.is.R
> > identical(f, eval(parse(text=attr(f, "source"))[[1]]))
>[1] FALSE
> > f <- function() TRUE
> > identical(f, eval(parse(text=attr(f, "source"))[[1]]))
>[1] TRUE
> >
>
>-- Tony Plate
>
>
>At Wednesday 08:09 PM 3/17/2004, Gabor Grothendieck wrote:
>
>
> >I left out the brackets in my last email but the problem
> >(a reappears after have been substituted out) still remains:
> >
> > > z <- substitute( function(){a+1}, list(a=quote(b)) )
> > > z
> >function() {
> > b + 1
> >}
> > > eval(z)
> >function(){a+1}
> >
> >
> >
> >---
> >Date: Wed, 17 Mar 2004 20:10:43 -0500 (EST)
> >From: Gabor Grothendieck <ggrothendieck at myway.com>
> >[ Add to Address Book | Block Address | Report as Spam ]
> >To: <R-help at stat.math.ethz.ch>
> >Subject: [R] substitute question
> >
> >
> >
> >
> >
> >
> >Consider the following example:
> >
> ># substitute a with b in the indicated function. Seems to work.
> > > z <- substitute( function()a+1, list(a=quote(b)) )
> > > z
> >function() b + 1
> >
> ># z is an object of class call so use eval
> ># to turn it into an object of class expression; however,
> ># when z is evaluated, the variable a returns.
> > > eval(z)
> >function()a+1
> >
> >Why did a suddenly reappear again after it had already been replaced?
>
From: Peter Dalgaard <p.dalgaard at biostat.ku.dk>> (The real pain in these examples is that substitute autoquotes its > expr argument. Therefore, when you want to modify an expression that > is already stored in a variable, you need an extra outer layer of > eval(substitute(...)) to poke the content of the variable into the > inner substitute. An "esub" function with standard evaluation > semantics would make this much easier.)That is one of the frustrations of using substitute. The other is that even if you do perform two levels of substitute, as I have been trying, you still can't count on it working for an arbitrary unevaluated expression, as my examples show. Even putting aside the source attribute which is super confusing until you know about it, all the solutions that I can see to the problem I presented are ugly. (1) One can either pick apart the function using body, or (2) I assume one could convert the function to text and paste together the correct substitute command with the text of the function inserted in its argument. The quote mechanism works but is not applicable if all you have is the function itself (as you point out). This is sooo frustrating. Thanks to you, Tony and Thomas for all your help.
> Date: 18 Mar 2004 23:52:47 +0100 > From: Peter Dalgaard <p.dalgaard at biostat.ku.dk> > To: <ggrothendieck at myway.com> > Cc: <p.dalgaard at biostat.ku.dk>, <tlumley at u.washington.edu>, <tplate at blackmesacapital.com>, <R-help at stat.math.ethz.ch> > Subject: Re: [R] substitute question > > > "Gabor Grothendieck" <ggrothendieck at myway.com> writes: > > > From: Peter Dalgaard <p.dalgaard at biostat.ku.dk> > > > (The real pain in these examples is that substitute autoquotes its > > > expr argument. Therefore, when you want to modify an expression that > > > is already stored in a variable, you need an extra outer layer of > > > eval(substitute(...)) to poke the content of the variable into the > > > inner substitute. An "esub" function with standard evaluation > > > semantics would make this much easier.) > > > > That is one of the frustrations of using substitute. > > > > The other is that even if you do perform two levels of substitute, > > as I have been trying, you still can't count on it working for > > an arbitrary unevaluated expression, as my examples show. > > Er, I don't think so. All I have seen is a couple of cases where you > tried to pass something that was not a language object (e.g. a > function as opposed to an expression or call generating a function.) > >The parse/deparse one definitely is an expression:> z <- parse(text=deparse(f)) > class(z);mode(z);typeof(z)[1] "expression" [1] "expression" [1] "expression" In the other one, it is not an expression in the inner substitute but should be by the time it gets to the outer one since I explicitly added expression(...). It is expanded which shows it did do that part right.
I don't think I expressed myself very well on that. Looking at what we get from the example:> z <- substitute(substitute(expression(f),list(a=quote(b))),list(f=f))> zsubstitute(expression(function () { a + 1 }), list(a = quote(b)))> class(z);mode(z);typeof(z)[1] "call" [1] "call" [1] "language" we see that the function seems to be expanded correctly and the statement does produce a call object. However, applying eval one, two or three times does not give what you would think if you looked at z above. In all cases, a is still there. Some posters have claimed that the problem is that list(f=f) cannot hold a function and must be an expression but that's not what the help page says, the examples are not all of class expression or call and even if that were true then my parse/deparse example should work since it is unquestionably an expression. Date: Thu, 18 Mar 2004 16:41:31 -0700 From: Tony Plate <tplate at blackmesacapital.com> To: <ggrothendieck at myway.com>, <p.dalgaard at biostat.ku.dk> Cc: <tlumley at u.washington.edu>, <R-help at stat.math.ethz.ch> Subject: Re: [R] substitute question Gabor, you might have less frustration if you work with Peter's esub() suggestion (use it instead of substitute()): esub <- function(expr, subst.list) do.call("substitute", list(expr, subst.list)) When you say "In the other one, it is not an expression in the inner substitute but should be by the time it gets to the outer one since I explicitly added expression(...). It is expanded which shows it did do that part right." something to bear in mind is the "natural order" of evaluation is reversed with nested substitute()'s (because the inner substitute() is quoted by the outer substitute()) (but I haven't carefully reasoned through this being an explanation for your particular frustration here.) hope this might help, Tony Plate At Thursday 04:06 PM 3/18/2004, Gabor Grothendieck wrote:> > Date: 18 Mar 2004 23:52:47 +0100 > > From: Peter Dalgaard <p.dalgaard at biostat.ku.dk> > > To: <ggrothendieck at myway.com> > > Cc: <p.dalgaard at biostat.ku.dk>, <tlumley at u.washington.edu>, > <tplate at blackmesacapital.com>, <R-help at stat.math.ethz.ch> > > Subject: Re: [R] substitute question > > > > > > "Gabor Grothendieck" <ggrothendieck at myway.com> writes: > > > > > From: Peter Dalgaard <p.dalgaard at biostat.ku.dk> > > > > (The real pain in these examples is that substitute autoquotes its > > > > expr argument. Therefore, when you want to modify an expression that > > > > is already stored in a variable, you need an extra outer layer of > > > > eval(substitute(...)) to poke the content of the variable into the > > > > inner substitute. An "esub" function with standard evaluation > > > > semantics would make this much easier.) > > > > > > That is one of the frustrations of using substitute. > > > > > > The other is that even if you do perform two levels of substitute, > > > as I have been trying, you still can't count on it working for > > > an arbitrary unevaluated expression, as my examples show. > > > > Er, I don't think so. All I have seen is a couple of cases where you > > tried to pass something that was not a language object (e.g. a > > function as opposed to an expression or call generating a function.) > > > > > >The parse/deparse one definitely is an expression: > > > z <- parse(text=deparse(f)) > > class(z);mode(z);typeof(z) >[1] "expression" >[1] "expression" >[1] "expression" > >In the other one, it is not an expression in the inner substitute >but should be by the time it gets to the outer one since I >explicitly added expression(...). It is expanded which shows >it did do that part right. >
Date: Thu, 18 Mar 2004 15:56:59 -0800 (PST) From: Thomas Lumley <tlumley at u.washington.edu>> > On Thu, 18 Mar 2004, Gabor Grothendieck wrote: > > (1) One can either pick apart the function using body, or > > Actually, I think this is fairly natural -- it is only body(f) > that is an > expression. Certainly substitute() could have been written to > operate on > functions as well, but it wasn't.In the context of R, natural is performing operations on whole objects at once. Its the same difference as indexing vs. vector operations. I am not sure where this came from about not operating on functions. Are you sure? I know just about everyone is saying this but the help page does not refer to that and the example I gave where we do use list(f=f) for f a function does seem to expand properly (see my last posting) although the subsitution never appears to take effect.
[This appears to have bounced so I am sending it again. Apologies
if it gets there twice.]
Thanks. Thus it seems that there are two types of expressions and calls:
1. fully expanded
2. partially expanded
and that fully expanded ones are a prerequisite for substitution.
body() and quote() produce such fully expanded expressions.
Using a small utility function we can investigate this:
recurse <- function( x, idx = NULL )
if ( length( x ) > 0 ) {
for( i in seq( along = x ) )
if (length(x[[i]])>1)
Recall( x[[i]], c(idx, i))
else {
if (length(idx)) cat(idx,"")
cat( i, class(x[[i]]), ":" )
cat( rep("\t",length(idx) + 2) )
print( x[[i]] )
}
}
f <- function(){a+1}
eb <- body(f)
class(eb)
recurse(eb)
eq <- quote(function(){a+1})
class(eq)
recurse(eq)
ep <- parse(text=deparse(f))
class(ep)
recurse(ep)
The output that the above is shown below. It shows that
body() and quote() produce fully expanded expression style objects
although body's is of class { and quote is of class call.
However, parse(text=deparse(f)) also produces a fully expanded
expression style object of class expression yet substitution
does not occur with that. Thus full vs. partial expansion is likely
a necessary but not a sufficient condition. There is something
else but I don't know what it is.
> f <- function(){a+1}
>
> eb <- body(f)
> class(eb)
[1] "{"> recurse(eb)
1 name : `{`
2 1 name : `+`
2 2 name : a
2 3 numeric : [1] 1>
> eq <- quote(function(){a+1})
> class(eq)
[1] "call"> recurse(eq) # lines begin with list indices and class name
1 name : `function`
2 NULL : NULL
3 1 name : `{`
3 2 1 name : `+`
3 2 2 name : a
3 2 3 numeric : [1] 1
4 NULL : NULL>
> ep <- parse(text=deparse(f))
> class(ep)
[1] "expression"> recurse(ep)
1 1 name : `function`
1 2 NULL : NULL
1 3 1 name : `{`
1 3 2 1 name : `+`
1 3 2 2 name : a
1 3 2 3 numeric : [1] 1
1 4 NULL : NULL
Date: Thu, 18 Mar 2004 17:27:20 -0800 (PST)
From: Thomas Lumley <tlumley at u.washington.edu>
To: Gabor Grothendieck <ggrothendieck at myway.com>
Cc: <tplate at blackmesacapital.com>, <R-help at stat.math.ethz.ch>
Subject: Re: [R] substitute question
On Thu, 18 Mar 2004, Gabor Grothendieck wrote:
>
>
> I don't think I expressed myself very well on that.
>
> Looking at what we get from the example:
>
> > z <-
substitute(substitute(expression(f),list(a=quote(b))),list(f=f))
>
> > z
> substitute(expression(function ()
> {
> a + 1
> }), list(a = quote(b)))
>
> > class(z);mode(z);typeof(z)
> [1] "call"
> [1] "call"
> [1] "language"
>
>
> we see that the function seems to be expanded correctly and
> the statement does produce a call object. However,
> applying eval one, two or three times does not give what
> you would think if you looked at z above.
Maybe we didn't express ourselves well enough.
Looking at z above isn't enough. z is a call to substitute().
Its first operand is an expression. The expression contains a single term,
which is a function.
If you typed
notz<- quote(substitute(expression(function ()
{
a + 1
}), list(a = quote(b))))
you would obtain something that deparsed the same as z, and so looked the
same, but was actually different. In notz the first operand of substitute
is an expression containing multiple terms, which if evaluated would
return a function.
substitute() goes though this expression and checks each term to see if it
is `a`. In z there is only one term and it isn't `a`. In notz there is
(after sufficient recursion) an `a` and it gets replaced.
So
> z[[2]][[2]]
function ()
{
a + 1
}> notz[[2]][[2]]
function() {
a + 1
}
are the respective operands, and they still look the same. But
> mode(z[[2]][[2]])
[1] "function"> mode(notz[[2]][[2]])
[1] "call"> length(z[[2]][[2]])
[1] 1> length(notz[[2]][[2]])
[1] 4
and if we try to find the actual `a` in there> notz[[2]][[2]][[3]][[2]][[2]]
a> z[[2]][[2]][[3]][[2]][[2]]
Error in z[[2]][[2]][[3]] : object is not subsettable>
-thomas
I don't think it is correct that "there are two types of expressions
and
calls". There is only one type of these things. I believe the relevant
distinction here is between 'call' objects (which can be informally
thought
of as the parse trees of unevaluated R code, and which formally have mode
'call' in R) and other things like objects of mode 'function',
etc.
However, this is all does pretty confusing when using substitute(),
because, substitute() does go inside objects that have mode 'call', but
doesn't go inside of objects that have mode 'function' or
'expression'. What makes it more confusing is that sometimes
'call'
objects can be wrapped up in expression objects. Note that parse(text=)
returns a 'call' object wrapped in an 'expression' object,
whereas quote()
returns a 'call' object -- I believe that in general it is true that
parse(text="XXX")[[1]] === quote(XXX).
Earlier in this discussion, Peter Dalgard stated "you can only do
substitutions on language objects" and then used the function is.language()
in an example, which I took at that time to imply that substitute() would
go inside objects for which is.language() returned true. However, from
experimenting, it seems that is.call() rather than is.language() is the
appropriate test.
Here are some simple examples.
> esub <- function(expr, sublist) do.call("substitute",
list(expr, sublist))
> e1 <- parse(text="a + 1")
> e2 <- quote(a + 1)
> e1
expression(a + 1)
> e2
a + 1
> mode(e1)
[1] "expression"
> mode(e2)
[1] "call"
> identical(e1[[1]], e2)
[1] TRUE
>
> # substitute() doesn't go inside e1, even though is.language(e1) is
TRUE
> c(is.language(e1), is.call(e1))
[1] TRUE FALSE
> esub(e1, list(a=as.name('b')))
expression(a + 1)
>
> c(is.language(e2), is.call(e2))
[1] TRUE TRUE
> esub(e2, list(a=as.name('b')))
b + 1
>
> c(is.language(e1[[1]]), is.call(e1[[1]]))
[1] TRUE TRUE
> esub(e1[[1]], list(a=as.name('b')))
b + 1
> identical(e2, e1[[1]])
[1] TRUE
>
> ef <- Quote(function() a + 1)
> f <- function() a + 1
> c(is.language(ef), is.call(ef))
[1] TRUE TRUE
> esub(ef, list(a=as.name('b')))
function() b + 1
> c(is.language(f), is.call(f))
[1] FALSE FALSE
> esub(f, list(a=as.name('b')))
function ()
a + 1
> c(is.language(body(f)), is.call(body(f)))
[1] TRUE TRUE
> esub(body(f), list(a=as.name('b')))
b + 1
>
>
I also see that in S-plus 6.2, substitute() behaves differently -- it does
go inside objects of mode 'call' and 'expression' and
substitutes 'b' for
'a' in every case above. To run the above code in S-plus, first do:
> body <- function(f) f[[1]]
> quote <- Quote
Although there isn't much to guide one in the documentation ?substitute,
the "R Language manual" does have some discussion of substitute() and
'expression' objects.
-- Tony Plate
At Thursday 10:58 PM 3/18/2004, Gabor Grothendieck
wrote:>
>Thanks. Thus it seems that there are two types of expressions and calls:
>
>1. fully expanded
>2. partially expanded
>
>and that fully expanded ones are a prerequisite for substitution.
>body() and quote() produce such fully expanded expressions.
>
>Using a small utility function we can investigate this:
>
>recurse <- function( x, idx = NULL )
> if ( length( x ) > 0 ) {
> for( i in seq( along = x ) )
> if (length(x[[i]])>1)
> Recall( x[[i]], c(idx, i))
> else {
> if (length(idx)) cat(idx,"")
> cat( i, class(x[[i]]), ":" )
> cat( rep("\t",length(idx) + 2) )
> print( x[[i]] )
> }
> }
>
>f <- function(){a+1}
>
>eb <- body(f)
>class(eb)
>recurse(eb)
>
>eq <- quote(function(){a+1})
>class(eq)
>recurse(eq)
>
>ep <- parse(text=deparse(f))
>class(ep)
>recurse(ep)
>
>
>The output that the above is shown below. It shows that
>body() and quote() produce fully expanded expression style objects
>although body's is of class { and quote is of class call.
>
>However, parse(text=deparse(f)) also produces a fully expanded
>expression style object of class expression yet substitution
>does not occur with that. Thus full vs. partial expansion is likely
>a necessary but not a sufficient condition. There is something
>else but I don't know what it is.
>
>
> > f <- function(){a+1}
> >
> > eb <- body(f)
> > class(eb)
>[1] "{"
> > recurse(eb)
>1 name : `{`
>2 1 name : `+`
>2 2 name : a
>2 3 numeric : [1] 1
> >
> > eq <- quote(function(){a+1})
> > class(eq)
>[1] "call"
> > recurse(eq) # lines begin with list indices and class name
>1 name : `function`
>2 NULL : NULL
>3 1 name : `{`
>3 2 1 name : `+`
>3 2 2 name : a
>3 2 3 numeric : [1] 1
>4 NULL : NULL
> >
> > ep <- parse(text=deparse(f))
> > class(ep)
>[1] "expression"
> > recurse(ep)
>1 1 name : `function`
>1 2 NULL : NULL
>1 3 1 name : `{`
>1 3 2 1 name : `+`
>1 3 2 2 name : a
>1 3 2 3 numeric : [1] 1
>1 4 NULL : NULL
>
>
>Date: Thu, 18 Mar 2004 17:27:20 -0800 (PST)
>From: Thomas Lumley <tlumley at u.washington.edu>
>To: Gabor Grothendieck <ggrothendieck at myway.com>
>Cc: <tplate at blackmesacapital.com>, <R-help at
stat.math.ethz.ch>
>Subject: Re: [R] substitute question
>
>
>On Thu, 18 Mar 2004, Gabor Grothendieck wrote:
>
> >
> >
> > I don't think I expressed myself very well on that.
> >
> > Looking at what we get from the example:
> >
> > > z <-
substitute(substitute(expression(f),list(a=quote(b))),list(f=f))
> >
> > > z
> > substitute(expression(function ()
> > {
> > a + 1
> > }), list(a = quote(b)))
> >
> > > class(z);mode(z);typeof(z)
> > [1] "call"
> > [1] "call"
> > [1] "language"
> >
> >
> > we see that the function seems to be expanded correctly and
> > the statement does produce a call object. However,
> > applying eval one, two or three times does not give what
> > you would think if you looked at z above.
>
>Maybe we didn't express ourselves well enough.
>
>Looking at z above isn't enough. z is a call to substitute().
>Its first operand is an expression. The expression contains a single term,
>which is a function.
>
>If you typed
>notz<- quote(substitute(expression(function ()
>{
>a + 1
>}), list(a = quote(b))))
>
>you would obtain something that deparsed the same as z, and so looked the
>same, but was actually different. In notz the first operand of substitute
>is an expression containing multiple terms, which if evaluated would
>return a function.
>
>substitute() goes though this expression and checks each term to see if it
>is `a`. In z there is only one term and it isn't `a`. In notz there is
>(after sufficient recursion) an `a` and it gets replaced.
>
>So
>
> > z[[2]][[2]]
>function ()
>{
>a + 1
>}
> > notz[[2]][[2]]
>function() {
>a + 1
>}
>
>are the respective operands, and they still look the same. But
>
> > mode(z[[2]][[2]])
>[1] "function"
> > mode(notz[[2]][[2]])
>[1] "call"
> > length(z[[2]][[2]])
>[1] 1
> > length(notz[[2]][[2]])
>[1] 4
>
>and if we try to find the actual `a` in there
> > notz[[2]][[2]][[3]][[2]][[2]]
>a
> > z[[2]][[2]][[3]][[2]][[2]]
>Error in z[[2]][[2]][[3]] : object is not subsettable
> >
>
>
> -thomas
>
>
>
>
>
>_______________________________________________
>No banners. No pop-ups. No kidding.
>Introducing My Way - http://www.myway.com
Great detective work, Tony! Thanks. Date: Fri, 19 Mar 2004 17:51:27 -0700 From: Tony Plate <tplate at blackmesacapital.com> To: <ggrothendieck at myway.com>, <tlumley at u.washington.edu> Cc: <R-help at stat.math.ethz.ch> Subject: Re: [R] substitute question I don't think it is correct that "there are two types of expressions and calls". There is only one type of these things. I believe the relevant distinction here is between 'call' objects (which can be informally thought of as the parse trees of unevaluated R code, and which formally have mode 'call' in R) and other things like objects of mode 'function', etc. However, this is all does pretty confusing when using substitute(), because, substitute() does go inside objects that have mode 'call', but doesn't go inside of objects that have mode 'function' or 'expression'. What makes it more confusing is that sometimes 'call' objects can be wrapped up in expression objects. Note that parse(text=) returns a 'call' object wrapped in an 'expression' object, whereas quote() returns a 'call' object -- I believe that in general it is true that parse(text="XXX")[[1]] === quote(XXX). Earlier in this discussion, Peter Dalgard stated "you can only do substitutions on language objects" and then used the function is.language() in an example, which I took at that time to imply that substitute() would go inside objects for which is.language() returned true. However, from experimenting, it seems that is.call() rather than is.language() is the appropriate test. Here are some simple examples.> esub <- function(expr, sublist) do.call("substitute", list(expr, sublist)) > e1 <- parse(text="a + 1") > e2 <- quote(a + 1) > e1expression(a + 1)> e2a + 1> mode(e1)[1] "expression"> mode(e2)[1] "call"> identical(e1[[1]], e2)[1] TRUE> > # substitute() doesn't go inside e1, even though is.language(e1) is TRUE > c(is.language(e1), is.call(e1))[1] TRUE FALSE> esub(e1, list(a=as.name('b')))expression(a + 1)> > c(is.language(e2), is.call(e2))[1] TRUE TRUE> esub(e2, list(a=as.name('b')))b + 1> > c(is.language(e1[[1]]), is.call(e1[[1]]))[1] TRUE TRUE> esub(e1[[1]], list(a=as.name('b')))b + 1> identical(e2, e1[[1]])[1] TRUE> > ef <- Quote(function() a + 1) > f <- function() a + 1 > c(is.language(ef), is.call(ef))[1] TRUE TRUE> esub(ef, list(a=as.name('b')))function() b + 1> c(is.language(f), is.call(f))[1] FALSE FALSE> esub(f, list(a=as.name('b')))function () a + 1> c(is.language(body(f)), is.call(body(f)))[1] TRUE TRUE> esub(body(f), list(a=as.name('b')))b + 1> >I also see that in S-plus 6.2, substitute() behaves differently -- it does go inside objects of mode 'call' and 'expression' and substitutes 'b' for 'a' in every case above. To run the above code in S-plus, first do:> body <- function(f) f[[1]] > quote <- QuoteAlthough there isn't much to guide one in the documentation ?substitute, the "R Language manual" does have some discussion of substitute() and 'expression' objects. -- Tony Plate At Thursday 10:58 PM 3/18/2004, Gabor Grothendieck wrote:> >Thanks. Thus it seems that there are two types of expressions and calls: > >1. fully expanded >2. partially expanded > >and that fully expanded ones are a prerequisite for substitution. >body() and quote() produce such fully expanded expressions. > >Using a small utility function we can investigate this: > >recurse <- function( x, idx = NULL ) > if ( length( x ) > 0 ) { > for( i in seq( along = x ) ) > if (length(x[[i]])>1) > Recall( x[[i]], c(idx, i)) > else { > if (length(idx)) cat(idx,"") > cat( i, class(x[[i]]), ":" ) > cat( rep("\t",length(idx) + 2) ) > print( x[[i]] ) > } > } > >f <- function(){a+1} > >eb <- body(f) >class(eb) >recurse(eb) > >eq <- quote(function(){a+1}) >class(eq) >recurse(eq) > >ep <- parse(text=deparse(f)) >class(ep) >recurse(ep) > > >The output that the above is shown below. It shows that >body() and quote() produce fully expanded expression style objects >although body's is of class { and quote is of class call. > >However, parse(text=deparse(f)) also produces a fully expanded >expression style object of class expression yet substitution >does not occur with that. Thus full vs. partial expansion is likely >a necessary but not a sufficient condition. There is something >else but I don't know what it is. > > > > f <- function(){a+1} > > > > eb <- body(f) > > class(eb) >[1] "{" > > recurse(eb) >1 name : `{` >2 1 name : `+` >2 2 name : a >2 3 numeric : [1] 1 > > > > eq <- quote(function(){a+1}) > > class(eq) >[1] "call" > > recurse(eq) # lines begin with list indices and class name >1 name : `function` >2 NULL : NULL >3 1 name : `{` >3 2 1 name : `+` >3 2 2 name : a >3 2 3 numeric : [1] 1 >4 NULL : NULL > > > > ep <- parse(text=deparse(f)) > > class(ep) >[1] "expression" > > recurse(ep) >1 1 name : `function` >1 2 NULL : NULL >1 3 1 name : `{` >1 3 2 1 name : `+` >1 3 2 2 name : a >1 3 2 3 numeric : [1] 1 >1 4 NULL : NULL > > >Date: Thu, 18 Mar 2004 17:27:20 -0800 (PST) >From: Thomas Lumley <tlumley at u.washington.edu> >To: Gabor Grothendieck <ggrothendieck at myway.com> >Cc: <tplate at blackmesacapital.com>, <R-help at stat.math.ethz.ch> >Subject: Re: [R] substitute question > > >On Thu, 18 Mar 2004, Gabor Grothendieck wrote: > > > > > > > I don't think I expressed myself very well on that. > > > > Looking at what we get from the example: > > > > > z <- substitute(substitute(expression(f),list(a=quote(b))),list(f=f)) > > > > > z > > substitute(expression(function () > > { > > a + 1 > > }), list(a = quote(b))) > > > > > class(z);mode(z);typeof(z) > > [1] "call" > > [1] "call" > > [1] "language" > > > > > > we see that the function seems to be expanded correctly and > > the statement does produce a call object. However, > > applying eval one, two or three times does not give what > > you would think if you looked at z above. > >Maybe we didn't express ourselves well enough. > >Looking at z above isn't enough. z is a call to substitute(). >Its first operand is an expression. The expression contains a single term, >which is a function. > >If you typed >notz<- quote(substitute(expression(function () >{ >a + 1 >}), list(a = quote(b)))) > >you would obtain something that deparsed the same as z, and so looked the >same, but was actually different. In notz the first operand of substitute >is an expression containing multiple terms, which if evaluated would >return a function. > >substitute() goes though this expression and checks each term to see if it >is `a`. In z there is only one term and it isn't `a`. In notz there is >(after sufficient recursion) an `a` and it gets replaced. > >So > > > z[[2]][[2]] >function () >{ >a + 1 >} > > notz[[2]][[2]] >function() { >a + 1 >} > >are the respective operands, and they still look the same. But > > > mode(z[[2]][[2]]) >[1] "function" > > mode(notz[[2]][[2]]) >[1] "call" > > length(z[[2]][[2]]) >[1] 1 > > length(notz[[2]][[2]]) >[1] 4 > >and if we try to find the actual `a` in there > > notz[[2]][[2]][[3]][[2]][[2]] >a > > z[[2]][[2]][[3]][[2]][[2]] >Error in z[[2]][[2]][[3]] : object is not subsettable > > > > > -thomas > > > > > >_______________________________________________ >No banners. No pop-ups. No kidding. >Introducing My Way - http://www.myway.com