R devel - May 2013 - Substitute / delayedAssign (was: Substitute unaware when promise objects are evaluated)

Duncan, Thank you for the clarification on how delayedAssign works. Should
R-level interfaces to promise objects ever become available, I expect they would
at time come in handy.

On the subject of substitute and delayedAssign, I do have a follow-up question
for the list. I'm trying to convert a named list of expression objects into
an environment of promise objects. After conversion, each expression in the list
will be automatically evaluated when the variable with the same name is accessed
in the environment. Effectively, I'm trying to create a hash table of
promise objects.

Here's the code I wrote that works just fine.

x <- list(a=3, b=expression(a+2), sleep=expression(Sys.sleep(2)))
env <- new.env()
for (i in seq(x)) {
	key <- names(x)[i]
	.Internal(delayedAssign(key,
				   eval(substitute(x[[i]], list(x=x, i=i)))[[1]],
				   eval.env=env, assign.env=env))
}	
env$b     # 3+2
[1] 5
env$sleep # Sleeps for 2 seconds
NULL  

The "problem" is that R CMD check complains that I shouldn't be
using .Internal() to access the delayedAssign function. However, if I don't
use .Internal(), then delayedAssign puts another substitute around my call that
prevents the 'i' iterator variable from being evaluated at the correct
time, which causes all variables to get the value x[[i]] for the very last value
of 'i'.

Can I safely ignore this R CMD check warning about .Internal, or is there a
better way to write this code?

Thanks, Robert



-----Original Message-----
From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
Sent: Wednesday, May 15, 2013 6:04 PM
To: McGehee, Robert
Cc: R-Devel (r-devel at r-project.org)
Subject: Re: [Rd] Substitute unaware when promise objects are evaluated

On 13-05-15 11:54 AM, McGehee, Robert wrote:
> R-devel,
> I used the 'substitute' function to create labels for objects
inside an environment, without actually evaluating the objects, as the objects
might be promises.
>
> However, I was surprised to see that 'substitute' returns the
expression slot of the original promise even after the promise has been forcibly
evaluated. (Doesn't the promise go away after evaluation?) This behavior
probably falls under the "...no guarantee that the resulting expression
makes any sense" clause of the ?substitute documentation, but in case
there's something actually wrong here, I thought I'd send an example.
I think you misunderstand promises.

A promise has two (or three, depending how you count) parts:  an 
expression with an associated environment, and a value.  The value isn't 
filled in until the expression is evaluated, but the expression doesn't 
go away then.  You can still see it until you change the variable that 
holds the promise.

> Here's an example showing how the evaluated expression returned by
substitute does not match the actual variable value:
>
>> env <- new.env()
>> z <- 0
>> delayedAssign("var", z+2, assign.env=env)
>> substitute(var, env=env)
> z + 2
The documentation for substitute may not be clear on this, but for a 
promise, the env argument will be ignored.  It was the eval.env argument 
to delayedAssign that set the promise's environment.
>> force(env$var)
> [1] 2
>> z <- 10
>> substitute(var, env=env)
> z + 2
>> eval(substitute(var, env=env))
> [1] 12
>> force(env$var)
> [1] 2
>
> Is there any obvious way to code around this behavior, e.g. can I
explicitly check if an object in an environment is an unevaluated promise?
Not at R level. In C code you could, but you probably shouldn't.  Think 
of promises as values where you can look up the expression that gave the 
value, and sometimes delay the calculation until you need it.

Duncan Murdoch

On 16/05/2013 9:06 AM, McGehee, Robert wrote:
> Duncan, Thank you for the clarification on how delayedAssign works. Should
R-level interfaces to promise objects ever become available, I expect they would
at time come in handy.
>
> On the subject of substitute and delayedAssign, I do have a follow-up
question for the list. I'm trying to convert a named list of expression
objects into an environment of promise objects. After conversion, each
expression in the list will be automatically evaluated when the variable with
the same name is accessed in the environment. Effectively, I'm trying to
create a hash table of promise objects.
>
> Here's the code I wrote that works just fine.
>
> x <- list(a=3, b=expression(a+2), sleep=expression(Sys.sleep(2)))
> env <- new.env()
> for (i in seq(x)) {
> 	key <- names(x)[i]
> 	.Internal(delayedAssign(key,
> 				   eval(substitute(x[[i]], list(x=x, i=i)))[[1]],
> 				   eval.env=env, assign.env=env))
> }	
> env$b     # 3+2
> [1] 5
> env$sleep # Sleeps for 2 seconds
> NULL
>
> The "problem" is that R CMD check complains that I shouldn't
be using .Internal() to access the delayedAssign function. However, if I
don't use .Internal(), then delayedAssign puts another substitute around my
call that prevents the 'i' iterator variable from being evaluated at the
correct time, which causes all variables to get the value x[[i]] for the very
last value of 'i'.
>
> Can I safely ignore this R CMD check warning about .Internal, or is there a
better way to write this code?
You should never call .Internal.  Arguments to internal functions may 
change without notice.

Here's one way to write your example without it.

x <- list(a=3, b=expression(a+2), sleep=expression(Sys.sleep(2)))
env <- new.env()

mydelay <- function(i) {
   expr <- x[[i]]
   name <- names(x)[i]
   do.call(delayedAssign, list(x=name, value=substitute(eval(expr), 
list(expr=expr)),
           eval.env=env, assign.env=env))
}

for (i in seq(x)) mydelay(i)

Duncan Murdoch

>
> Thanks, Robert
>
>
>
> -----Original Message-----
> From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com]
> Sent: Wednesday, May 15, 2013 6:04 PM
> To: McGehee, Robert
> Cc: R-Devel (r-devel at r-project.org)
> Subject: Re: [Rd] Substitute unaware when promise objects are evaluated
>
> On 13-05-15 11:54 AM, McGehee, Robert wrote:
> > R-devel,
> > I used the 'substitute' function to create labels for objects
inside an environment, without actually evaluating the objects, as the objects
might be promises.
> >
> > However, I was surprised to see that 'substitute' returns the
expression slot of the original promise even after the promise has been forcibly
evaluated. (Doesn't the promise go away after evaluation?) This behavior
probably falls under the "...no guarantee that the resulting expression
makes any sense" clause of the ?substitute documentation, but in case
there's something actually wrong here, I thought I'd send an example.
>
> I think you misunderstand promises.
>
> A promise has two (or three, depending how you count) parts:  an
> expression with an associated environment, and a value.  The value
isn't
> filled in until the expression is evaluated, but the expression doesn't
> go away then.  You can still see it until you change the variable that
> holds the promise.
>
>
> > Here's an example showing how the evaluated expression returned by
substitute does not match the actual variable value:
> >
> >> env <- new.env()
> >> z <- 0
> >> delayedAssign("var", z+2, assign.env=env)
> >> substitute(var, env=env)
> > z + 2
>
> The documentation for substitute may not be clear on this, but for a
> promise, the env argument will be ignored.  It was the eval.env argument
> to delayedAssign that set the promise's environment.
>
> >> force(env$var)
> > [1] 2
> >> z <- 10
> >> substitute(var, env=env)
> > z + 2
> >> eval(substitute(var, env=env))
> > [1] 12
> >> force(env$var)
> > [1] 2
> >
> > Is there any obvious way to code around this behavior, e.g. can I
explicitly check if an object in an environment is an unevaluated promise?
>
> Not at R level. In C code you could, but you probably shouldn't.  Think
> of promises as values where you can look up the expression that gave the
> value, and sometimes delay the calculation until you need it.
>
> Duncan Murdoch

On Thu, May 16, 2013 at 6:06 AM, McGehee, Robert <
Robert.McGehee@geodecapital.com> wrote:
> Duncan, Thank you for the clarification on how delayedAssign works. Should
> R-level interfaces to promise objects ever become available, I expect they
> would at time come in handy.
>
> On the subject of substitute and delayedAssign, I do have a follow-up
> question for the list. I'm trying to convert a named list of expression
> objects into an environment of promise objects. After conversion, each
> expression in the list will be automatically evaluated when the variable
> with the same name is accessed in the environment. Effectively, I'm
trying
> to create a hash table of promise objects.
>
Populating a new environment with promises happens to be what "calling a
function" in R does anyway, so an elegant way to accomplish this goal is:

makePromiseEnv <- function(expressions, parent=parent.frame()) {
    f <- function() environment()
    formals(f) <- as.pairlist(expressions)
    environment(f) <- parent
    f()
}
> e <- makePromiseEnv(alist(a = {print("hello"); 4}, b =
{print("again");
6}))
> e$a
[1] "hello"
[1] 4
> e$a
[1] 4
> e$b
[1] "again"
[1] 6
> e$b
[1] 6

	[[alternative HTML version deleted]]

On May 16, 2013, at 15:06 , McGehee, Robert wrote:
> Duncan, Thank you for the clarification on how delayedAssign works. Should
R-level interfaces to promise objects ever become available, I expect they would
at time come in handy.
> 
> On the subject of substitute and delayedAssign, I do have a follow-up
question for the list. I'm trying to convert a named list of expression
objects into an environment of promise objects. After conversion, each
expression in the list will be automatically evaluated when the variable with
the same name is accessed in the environment. Effectively, I'm trying to
create a hash table of promise objects.
> 
> Here's the code I wrote that works just fine.
> 
> x <- list(a=3, b=expression(a+2), sleep=expression(Sys.sleep(2)))
> env <- new.env()
> for (i in seq(x)) {
> 	key <- names(x)[i]
> 	.Internal(delayedAssign(key,
> 				   eval(substitute(x[[i]], list(x=x, i=i)))[[1]],
> 				   eval.env=env, assign.env=env))
> }	
> env$b     # 3+2
> [1] 5
> env$sleep # Sleeps for 2 seconds
> NULL  
> 
> The "problem" is that R CMD check complains that I shouldn't
be using .Internal() to access the delayedAssign function. However, if I
don't use .Internal(), then delayedAssign puts another substitute around my
call that prevents the 'i' iterator variable from being evaluated at the
correct time, which causes all variables to get the value x[[i]] for the very
last value of 'i'.
> 
> Can I safely ignore this R CMD check warning about .Internal, or is there a
better way to write this code?
These things are slippery, but the usual way out is to figure out exactly which
expression you want to call, compute the expression unevaluated, and evaulate
it.

Something like

e <- bquote(delayedAssign( .(names(x)[i]), .(x[[i]]), eval.env=env,
assign.env=env))
print(e)
eval(e)

(of course remove the print(e) once you're sure that it is doing the right
thing)


> 
> Thanks, Robert
> 
> 
> 
> -----Original Message-----
> From: Duncan Murdoch [mailto:murdoch.duncan at gmail.com] 
> Sent: Wednesday, May 15, 2013 6:04 PM
> To: McGehee, Robert
> Cc: R-Devel (r-devel at r-project.org)
> Subject: Re: [Rd] Substitute unaware when promise objects are evaluated
> 
> On 13-05-15 11:54 AM, McGehee, Robert wrote:
>> R-devel,
>> I used the 'substitute' function to create labels for objects
inside an environment, without actually evaluating the objects, as the objects
might be promises.
>> 
>> However, I was surprised to see that 'substitute' returns the
expression slot of the original promise even after the promise has been forcibly
evaluated. (Doesn't the promise go away after evaluation?) This behavior
probably falls under the "...no guarantee that the resulting expression
makes any sense" clause of the ?substitute documentation, but in case
there's something actually wrong here, I thought I'd send an example.
> 
> I think you misunderstand promises.
> 
> A promise has two (or three, depending how you count) parts:  an 
> expression with an associated environment, and a value.  The value
isn't
> filled in until the expression is evaluated, but the expression doesn't
> go away then.  You can still see it until you change the variable that 
> holds the promise.
> 
> 
>> Here's an example showing how the evaluated expression returned by
substitute does not match the actual variable value:
>> 
>>> env <- new.env()
>>> z <- 0
>>> delayedAssign("var", z+2, assign.env=env)
>>> substitute(var, env=env)
>> z + 2
> 
> The documentation for substitute may not be clear on this, but for a 
> promise, the env argument will be ignored.  It was the eval.env argument 
> to delayedAssign that set the promise's environment.
> 
>>> force(env$var)
>> [1] 2
>>> z <- 10
>>> substitute(var, env=env)
>> z + 2
>>> eval(substitute(var, env=env))
>> [1] 12
>>> force(env$var)
>> [1] 2
>> 
>> Is there any obvious way to code around this behavior, e.g. can I
explicitly check if an object in an environment is an unevaluated promise?
> 
> Not at R level. In C code you could, but you probably shouldn't.  Think
> of promises as values where you can look up the expression that gave the 
> value, and sometimes delay the calculation until you need it.
> 
> Duncan Murdoch
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
-- 
Peter Dalgaard, Professor
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk  Priv: PDalgd at gmail.com