R help - Aug 2016 - Continuation-parsing / trampoline / infinite recursion problem

That did the trick! 

I was so focused on not evaluating the continuation that I completely forgot
that the thunk could hold an unevaluated value? now it seems to be working for
all the various implementations I have been playing around with.

I think I still need to wrap my head around *why* the forced evaluation is
necessary there, but I will figure that out when my tired brain has had a little
rest.

Thanks a lot!

	Thomas

> On 10 Aug 2016, at 19:04, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:
> 
> On 10/08/2016 12:53 PM, Thomas Mailund wrote:
>> > On 10 Aug 2016, at 13:56, Thomas Mailund <mailund at
birc.au.dk> wrote:
>> >
>> > make_thunk <- function(f, ...) f(...)
>> 
>> Doh!  It is of course this one:
>> 
>> make_thunk <- function(f, ...) function() f(?)
>> 
>> It just binds a function call into a thunk so I can delay its
evaluation.
> 
> I haven't looked closely at the full set of functions, but this
comment:
> 
> force(continuation) # if I remove this line I get an error
> 
> makes it sound as though you're being caught by lazy evaluation. The
"make_thunk" doesn't appear to evaluate ..., so its value can
change between the time you make the thunk and the time you evaluate it.  I
think you could force the evaluation within make_thunk by changing it to
> 
> make_thunk <- function(f, ...) { list(...); function() f(?) }
> 
> and then would be able to skip the force() in your thunk_factorial
function.
> 
> Duncan Murdoch
> 
>

But wait, how is it actually changing? And how did calling `cat` make the
problem go away?

Ok, I will go think about it?

Thanks anyway, it seems to do the trick.

> On 10 Aug 2016, at 19:10, Thomas Mailund <mailund at birc.au.dk>
wrote:
> 
> 
> That did the trick! 
> 
> I was so focused on not evaluating the continuation that I completely
forgot that the thunk could hold an unevaluated value? now it seems to be
working for all the various implementations I have been playing around with.
> 
> I think I still need to wrap my head around *why* the forced evaluation is
necessary there, but I will figure that out when my tired brain has had a little
rest.
> 
> Thanks a lot!
> 
> 	Thomas
> 
> 
>> On 10 Aug 2016, at 19:04, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:
>> 
>> On 10/08/2016 12:53 PM, Thomas Mailund wrote:
>>>> On 10 Aug 2016, at 13:56, Thomas Mailund <mailund at
birc.au.dk> wrote:
>>>> 
>>>> make_thunk <- function(f, ...) f(...)
>>> 
>>> Doh!  It is of course this one:
>>> 
>>> make_thunk <- function(f, ...) function() f(?)
>>> 
>>> It just binds a function call into a thunk so I can delay its
evaluation.
>> 
>> I haven't looked closely at the full set of functions, but this
comment:
>> 
>> force(continuation) # if I remove this line I get an error
>> 
>> makes it sound as though you're being caught by lazy evaluation.
The "make_thunk" doesn't appear to evaluate ..., so its value can
change between the time you make the thunk and the time you evaluate it.  I
think you could force the evaluation within make_thunk by changing it to
>> 
>> make_thunk <- function(f, ...) { list(...); function() f(?) }
>> 
>> and then would be able to skip the force() in your thunk_factorial
function.
>> 
>> Duncan Murdoch
>> 
>> 
> 
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

On 10/08/2016 1:10 PM, Thomas Mailund wrote:
> That did the trick!
>
> I was so focused on not evaluating the continuation that I completely
forgot that the thunk could hold an unevaluated value? now it seems to be
working for all the various implementations I have been playing around with.
>
> I think I still need to wrap my head around *why* the forced evaluation is
necessary there, but I will figure that out when my tired brain has had a little
rest.
The original version

make_thunk <- function(f, ...) function() f(?)

says to construct a new function whose body evaluates the expression 
f(...).  It never evaluates f nor ... , so they don't get evaluated 
until the first time you evaluate that new function.

My version containing list(...) forces evaluation of ... .  It would 
have been even better to use

make_thunk <- function(f, ...) { list(f, ...); function() f(?) }

because that forces evaluation of both arguments.

I suspect you would have problems with

make_thunk <- function(f, ...) function() do.call(f, list(...))

for exactly the same reasons as the original; I'm surprised that you 
found it appears to work.

Duncan Murdoch
>
> Thanks a lot!
>
> 	Thomas
>
>
> > On 10 Aug 2016, at 19:04, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:
> >
> > On 10/08/2016 12:53 PM, Thomas Mailund wrote:
> >> > On 10 Aug 2016, at 13:56, Thomas Mailund <mailund at
birc.au.dk> wrote:
> >> >
> >> > make_thunk <- function(f, ...) f(...)
> >>
> >> Doh!  It is of course this one:
> >>
> >> make_thunk <- function(f, ...) function() f(?)
> >>
> >> It just binds a function call into a thunk so I can delay its
evaluation.
> >
> > I haven't looked closely at the full set of functions, but this
comment:
> >
> > force(continuation) # if I remove this line I get an error
> >
> > makes it sound as though you're being caught by lazy evaluation.
The "make_thunk" doesn't appear to evaluate ..., so its value can
change between the time you make the thunk and the time you evaluate it.  I
think you could force the evaluation within make_thunk by changing it to
> >
> > make_thunk <- function(f, ...) { list(...); function() f(?) }
> >
> > and then would be able to skip the force() in your thunk_factorial
function.
> >
> > Duncan Murdoch
> >
> >
>

?
I am not sure I can see exactly how the parameters are changing at all,
regardless of which of the versions I am using. Nowhere in the code do I ever
modify assign to a variable (except for defining the global-level functions).

I think my problem is that I don?t really understand ... here.

I would expect these two cases, with and without a thunk, to give me the same
output, but they clearly do not.

x <- function(...) eval(substitute(alist(...)))
x(a = 2, b = 3)
x(c = 4, d = 5)

xx <- function(...) function() eval(substitute(alist(...)))
xx(a = 2, b = 3)()
xx(c = 4, d = 5)()

The first gives me the parameters and the second just ? back.

How is the thunk actually seeing ... and why does it work with do.call and not
with direct call?

library(pryr)
xxx <- function(...) function() do.call(eval %.% substitute %.% alist,
list(...))
xxx(a = 2, b = 3)()
xxx(c = 4, d = 5)()

gives me the same results as the xx case, so it is not the do.call that does it,
even though that works in my examples.

With

xxxx <- function(...) { list(...) ; function() eval(substitute(alist(...))) }
xxxx(a = 2, b = 3)()
xxxx(c = 4, d = 5)()

it is the same.


Explicitly naming the parameters, of course works fine

y <- function( ...) { params <- list(...) ; function() params }
y(a = 2, b = 3)()
y(c = 4, d = 5)()

Here I get the expected lists out.

I guess I just shouldn?t be using ... in an inner function that refers to the
parameters in an outer function. I?m not even sure what that should be expected
to do and I certainly do not understand what is happening :)

Explicitly remembering the parameters seems to work fine, though.

Cheers
	Thomas




On 10 August 2016 at 19:28:43, Duncan Murdoch (murdoch.duncan at
gmail.com(mailto:murdoch.duncan at gmail.com)) wrote:
> On 10/08/2016 1:10 PM, Thomas Mailund wrote:
> > That did the trick!
> >
> > I was so focused on not evaluating the continuation that I completely
forgot that the thunk could hold an unevaluated value? now it seems to be
working for all the various implementations I have been playing around with.
> >
> > I think I still need to wrap my head around *why* the forced
evaluation is necessary there, but I will figure that out when my tired brain
has had a little rest.
>  
> The original version
>  
> make_thunk <- function(f, ...) function() f(?)
>  
> says to construct a new function whose body evaluates the expression
> f(...). It never evaluates f nor ... , so they don't get evaluated
> until the first time you evaluate that new function.
>  
> My version containing list(...) forces evaluation of ... . It would
> have been even better to use
>  
> make_thunk <- function(f, ...) { list(f, ...); function() f(?) }
>  
> because that forces evaluation of both arguments.
>  
> I suspect you would have problems with
>  
> make_thunk <- function(f, ...) function() do.call(f, list(...))
>  
> for exactly the same reasons as the original; I'm surprised that you
> found it appears to work.
>  
> Duncan Murdoch
>  
> >
> > Thanks a lot!
> >
> > Thomas
> >
> >
> > > On 10 Aug 2016, at 19:04, Duncan Murdoch wrote:
> > >
> > > On 10/08/2016 12:53 PM, Thomas Mailund wrote:
> > >> > On 10 Aug 2016, at 13:56, Thomas Mailund wrote:
> > >> >
> > >> > make_thunk <- function(f, ...) f(...)
> > >>
> > >> Doh! It is of course this one:
> > >>
> > >> make_thunk <- function(f, ...) function() f(?)
> > >>
> > >> It just binds a function call into a thunk so I can delay its
evaluation.
> > >
> > > I haven't looked closely at the full set of functions, but
this comment:
> > >
> > > force(continuation) # if I remove this line I get an error
> > >
> > > makes it sound as though you're being caught by lazy
evaluation. The "make_thunk" doesn't appear to evaluate ..., so
its value can change between the time you make the thunk and the time you
evaluate it. I think you could force the evaluation within make_thunk by
changing it to
> > >
> > > make_thunk <- function(f, ...) { list(...); function() f(?) }
> > >
> > > and then would be able to skip the force() in your
thunk_factorial function.
> > >
> > > Duncan Murdoch
> > >
> > >
> >
>

?
Ok, I think maybe I am beginning to see what is going wrong...

Explicitly remembering the thunk parameters in a list works fine, as far as I
can see.

make_thunk <- function(f, ...) {
? remembered <- list(...)
? function(...) do.call(f, as.list(remembered))
}

thunk_factorial <- function(n, continuation = identity) {
? if (n == 1) {
? ? continuation(1)
? } else {
? ? new_continuation <- function(result) {
? ? ? make_thunk(continuation, n * result)
? ? }
? ? make_thunk(thunk_factorial, n - 1, new_continuation)
? }
}

trampoline <- function(thunk) {
? while (is.function(thunk)) thunk <- thunk()
? thunk
}

trampoline(thunk_factorial(100))


But if I delay the evaluation of the parameters to thunk I get an error

make_thunk <- function(f, ...) {
? remembered <- eval(substitute(alist(...))) # not evaluating parameters yet
? function(...) do.call(f, as.list(remembered))
}

thunk_factorial <- function(n, continuation = identity) {
? if (n == 1) {
? ? continuation(1)
? } else {
? ? new_continuation <- function(result) {
? ? ? make_thunk(continuation, n * result)
? ? }
? ? make_thunk(thunk_factorial, n - 1, new_continuation)
? }
}

trampoline(thunk_factorial(100))

Running this version I am told, when applying the function, that it doesn?t see
variable `n`.


As far as I can see, the thunk remembers the parameters just fine. At least this
gives me the parameters I made it remember

x <- 1
f <- make_thunk(list, a = 1 * x, b = 2 * x)
g <- make_thunk(list, c = 3 * x)
f()
g()

Here I just get the parameters back in a list because the wrapped function is
`list`. (The reason I have `x` as a global variable and use it in the arguments
is so I get call objects that needs to be evaluated lazily instead of just
values).

These values contain the expressions I gave the `make_thunk` function, of
course, and they are not evaluated. So in the factorial function the missing `n`
is because I give it the expression `n - 1` that it of course cannot evaluate in
the thunk.

So I cannot really delay evaluation.

Does this sound roughly correct?

Now why I can still get it to work when I call `cat` remains a mystery?

Cheers
	Thomas



On 10 August 2016 at 19:12:41, Thomas Mailund (mailund at
birc.au.dk(mailto:mailund at birc.au.dk)) wrote:
>  
> That did the trick!
>  
> I was so focused on not evaluating the continuation that I completely
forgot that the thunk could hold an unevaluated value? now it seems to be
working for all the various implementations I have been playing around with.
>  
> I think I still need to wrap my head around *why* the forced evaluation is
necessary there, but I will figure that out when my tired brain has had a little
rest.
>  
> Thanks a lot!
>  
> Thomas
>  
>  
> > On 10 Aug 2016, at 19:04, Duncan Murdoch wrote:
> >
> > On 10/08/2016 12:53 PM, Thomas Mailund wrote:
> >> > On 10 Aug 2016, at 13:56, Thomas Mailund wrote:
> >> >
> >> > make_thunk <- function(f, ...) f(...)
> >>
> >> Doh! It is of course this one:
> >>
> >> make_thunk <- function(f, ...) function() f(?)
> >>
> >> It just binds a function call into a thunk so I can delay its
evaluation.
> >
> > I haven't looked closely at the full set of functions, but this
comment:
> >
> > force(continuation) # if I remove this line I get an error
> >
> > makes it sound as though you're being caught by lazy evaluation.
The "make_thunk" doesn't appear to evaluate ..., so its value can
change between the time you make the thunk and the time you evaluate it. I think
you could force the evaluation within make_thunk by changing it to
> >
> > make_thunk <- function(f, ...) { list(...); function() f(?) }
> >
> > and then would be able to skip the force() in your thunk_factorial
function.
> >
> > Duncan Murdoch
> >
> >
>  
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

On 10/08/2016 2:39 PM, Thomas Mailund wrote:
>
> Ok, I think maybe I am beginning to see what is going wrong...
>
> Explicitly remembering the thunk parameters in a list works fine, as far as
I can see.
>
> make_thunk <- function(f, ...) {
>   remembered <- list(...)
>   function(...) do.call(f, as.list(remembered))
> }
Where that will fail is in a situation like this:

thunklist <- list(thunk_factorial, thunk_somethingelse)
for (i in seq_along(thunklist))
   thunklist[[i]] <- make_thunk(thunklist[[i]])

The problem is that the first time thunklist[[1]] is evaluated, it will 
call the function thunklist[[2]] (or something else if i has been 
modified in the meantime), and things will go bad.  That's why it's 
important to force both f and ... in make_thunk.

Duncan Murdoch
>
> thunk_factorial <- function(n, continuation = identity) {
>   if (n == 1) {
>     continuation(1)
>   } else {
>     new_continuation <- function(result) {
>       make_thunk(continuation, n * result)
>     }
>     make_thunk(thunk_factorial, n - 1, new_continuation)
>   }
> }
>
> trampoline <- function(thunk) {
>   while (is.function(thunk)) thunk <- thunk()
>   thunk
> }
>
> trampoline(thunk_factorial(100))
>
>
> But if I delay the evaluation of the parameters to thunk I get an error
>
> make_thunk <- function(f, ...) {
>   remembered <- eval(substitute(alist(...))) # not evaluating parameters
yet
>   function(...) do.call(f, as.list(remembered))
> }
>
> thunk_factorial <- function(n, continuation = identity) {
>   if (n == 1) {
>     continuation(1)
>   } else {
>     new_continuation <- function(result) {
>       make_thunk(continuation, n * result)
>     }
>     make_thunk(thunk_factorial, n - 1, new_continuation)
>   }
> }
>
> trampoline(thunk_factorial(100))
>
> Running this version I am told, when applying the function, that it doesn?t
see variable `n`.
>
>
> As far as I can see, the thunk remembers the parameters just fine. At least
this gives me the parameters I made it remember
>
> x <- 1
> f <- make_thunk(list, a = 1 * x, b = 2 * x)
> g <- make_thunk(list, c = 3 * x)
> f()
> g()
>
> Here I just get the parameters back in a list because the wrapped function
is `list`. (The reason I have `x` as a global variable and use it in the
arguments is so I get call objects that needs to be evaluated lazily instead of
just values).
>
> These values contain the expressions I gave the `make_thunk` function, of
course, and they are not evaluated. So in the factorial function the missing `n`
is because I give it the expression `n - 1` that it of course cannot evaluate in
the thunk.
>
> So I cannot really delay evaluation.
>
> Does this sound roughly correct?
>
> Now why I can still get it to work when I call `cat` remains a mystery?
>
> Cheers
> 	Thomas
>
>
>
> On 10 August 2016 at 19:12:41, Thomas Mailund (mailund at
birc.au.dk(mailto:mailund at birc.au.dk)) wrote:
>
>>
>> That did the trick!
>>
>> I was so focused on not evaluating the continuation that I completely
forgot that the thunk could hold an unevaluated value? now it seems to be
working for all the various implementations I have been playing around with.
>>
>> I think I still need to wrap my head around *why* the forced evaluation
is necessary there, but I will figure that out when my tired brain has had a
little rest.
>>
>> Thanks a lot!
>>
>> Thomas
>>
>>
>>> On 10 Aug 2016, at 19:04, Duncan Murdoch wrote:
>>>
>>> On 10/08/2016 12:53 PM, Thomas Mailund wrote:
>>>>> On 10 Aug 2016, at 13:56, Thomas Mailund wrote:
>>>>>
>>>>> make_thunk <- function(f, ...) f(...)
>>>>
>>>> Doh! It is of course this one:
>>>>
>>>> make_thunk <- function(f, ...) function() f(?)
>>>>
>>>> It just binds a function call into a thunk so I can delay its
evaluation.
>>>
>>> I haven't looked closely at the full set of functions, but this
comment:
>>>
>>> force(continuation) # if I remove this line I get an error
>>>
>>> makes it sound as though you're being caught by lazy
evaluation. The "make_thunk" doesn't appear to evaluate ..., so
its value can change between the time you make the thunk and the time you
evaluate it. I think you could force the evaluation within make_thunk by
changing it to
>>>
>>> make_thunk <- function(f, ...) { list(...); function() f(?) }
>>>
>>> and then would be able to skip the force() in your thunk_factorial
function.
>>>
>>> Duncan Murdoch
>>>
>>>
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

On 10/08/2016 1:28 PM, Duncan Murdoch wrote:
> On 10/08/2016 1:10 PM, Thomas Mailund wrote:
>> That did the trick!
>>
>> I was so focused on not evaluating the continuation that I completely
forgot that the thunk could hold an unevaluated value? now it seems to be
working for all the various implementations I have been playing around with.
>>
>> I think I still need to wrap my head around *why* the forced evaluation
is necessary there, but I will figure that out when my tired brain has had a
little rest.
>
> The original version
>
> make_thunk <- function(f, ...) function() f(?)
>
> says to construct a new function whose body evaluates the expression
> f(...).  It never evaluates f nor ... , so they don't get evaluated
> until the first time you evaluate that new function.
>
> My version containing list(...) forces evaluation of ... .  It would
> have been even better to use
>
> make_thunk <- function(f, ...) { list(f, ...); function() f(?) }
>
> because that forces evaluation of both arguments.
>
> I suspect you would have problems with
>
> make_thunk <- function(f, ...) function() do.call(f, list(...))
>
> for exactly the same reasons as the original; I'm surprised that you
> found it appears to work.
I have done some experimentation, and am unable to reproduce the 
behaviour you described.  Using do.call() doesn't affect things.

Duncan Murdoch