Would introducing the new frame, with the call to local(), cause problems
when you use frame counting instead of <<- to modify variables outside the
scope of lapply's FUN, I think the frame counts may have to change. E.g.,
here is code from actuar::simul() that might be affected:
x <- unlist(lapply(nodes[[i]], seq))
lapply(nodes[(i + 1):(nlevels - 1)],
function(v) assign("x", rep.int(x, v), envir
parent.frame(2)))
m[, i] <- x
(I think the parent.frame(2) might have to be changed to parent.frame(8)
for that to work. Such code looks pretty ugly to me but seems to be rare.)
It also seems to cause problems with some built-in functions:
newlapply <- function (X, FUN, ...)
{
FUN <- match.fun(FUN)
if (!is.list(X))
X <- as.list(X)
rval <- vector("list", length(X))
for (i in seq(along = X)) {
rval[i] <- list(local({
i <- i
FUN(X[[i]], ...)
}))
}
names(rval) <- names(X)
return(rval)
}
newlapply(1:2,log)
#Error in FUN(X[[i]], ...) : non-numeric argument to mathematical function
newlapply(1:2,function(x)log(x))
#[[1]]
#[1] 0
#
#[[2]]
#[1] 0.6931472
Bill Dunlap
TIBCO Software
wdunlap tibco.com
On Tue, Feb 24, 2015 at 7:50 AM, <luke-tierney at uiowa.edu> wrote:
> The documentation is not specific enough on the indented semantics in
> this situation to consider this a bug. The original R-level
> implementation of lapply was
>
> lapply <- function(X, FUN, ...) {
> FUN <- match.fun(FUN)
> if (!is.list(X))
> X <- as.list(X)
> rval <- vector("list", length(X))
> for(i in seq(along = X))
> rval[i] <- list(FUN(X[[i]], ...))
> names(rval) <- names(X) # keep `names' !
> return(rval)
> }
>
> and the current internal implementation is consistent with this. With
> a loop like this lazy evaluation and binding assignment interact in
> this way; the force() function was introduced to help with this.
>
> That said, the expression FUN(X[[i]], ...) could be replaced by
>
> local({
> i <- i
> list(FUN(X[[i]], ...)
> })
>
> which would produce the more desirable result
>
> > sapply(test, function(myfn) myfn(2))
> [1] 2 4 6 8
>
> The C implementation could use this approach, or could rebuild the
> expression being evaluated at each call to get almost the same semantics.
> Both would add a little overhead. Some code optimization might reduce
> the overhead in some instances (e.g. if FUN is a BUILTIN), but it's
> not clear that would be worth while.
>
> Variants of this issue arise in a couple of places so it may be worth
> looking into.
>
> Best,
>
> luke
>
>
> On Tue, 24 Feb 2015, Radford Neal wrote:
>
> From: Daniel Kaschek <daniel.kaschek at physik.uni-freiburg.de>
>>
>>> ... When I evaluate this list of functions by
>>> another lapply/sapply, I get an unexpected result: all values
coincide.
>>> However, when I uncomment the print(), it works as expected. Is
this a
>>> bug or a feature?
>>>
>>> conditions <- 1:4
>>> test <- lapply(conditions, function(mycondition){
>>> #print(mycondition)
>>> myfn <- function(i) mycondition*i
>>> return(myfn)
>>> })
>>>
>>> sapply(test, function(myfn) myfn(2))
>>>
>>
>> From: Jeroen Ooms <jeroenooms at gmail.com>
>>
>>> I think it is a bug. If we use substitute to inspect the promise,
it
>>> appears the index number is always equal to its last value:
>>>
>>
>> From: Duncan Temple Lang <dtemplelang at ucdavis.edu>
>>
>>> Not a bug, but does surprise people. It is lazy evaluation.
>>>
>>
>>
>> I think it is indeed a bug. The lapply code saves a bit of time by
>> reusing the same storage for the scalar index number every iteration.
>> This amounts to modifying the R code that was used for the previous
>> function call. There's no justification for doing this in the
>> documentation for lapply. It is certainly not desired behaviour,
>> except in so far as it allows a slight savings in time (which is
>> minor, given the time that the function call itself will take).
>>
>> Radford Neal
>>
>> ______________________________________________
>> R-devel at r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
> --
> Luke Tierney
> Ralph E. Wareham Professor of Mathematical Sciences
> University of Iowa Phone: 319-335-3386
> Department of Statistics and Fax: 319-335-3017
> Actuarial Science
> 241 Schaeffer Hall email: luke-tierney at uiowa.edu
> Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
>
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
[[alternative HTML version deleted]]
Actually using local() might create some issues, though probably not many. For the C implementation of lapply I would probably create a new environment with a frame containing the binding for i and use that in an eval call. That wouldn't add another call frame, but it would change the environment which could still bite something. I would want to run any change like this over at least CRAN, maybe also BIOC, tests to see if there are any issues before committing. There are a few other places where the internal C code does calls to R functions in a less that ideal way. apply() is also currently written as a loop along the lines of the original lapply I showed. The parallel constructs from snow all use lapply or apply, so any changes there would be inherited; the mc functions are a bit more complicated and may need a more careful look. Overall it looks like we could use a new utility at both R and C level for calling a function with already evaluated arguments and use this in all relevant places (maybe called funcall or .Funcall or something like that). I'll try look into this in the next few weeks. Best, luke On Thu, 26 Feb 2015, William Dunlap wrote:> Would introducing the new frame, with the call to local(), cause problems > when you use frame counting instead of <<- to modify variables outside the > scope of lapply's FUN, I think the frame counts may have to change.? E.g., > here is code from actuar::simul() that might be affected: > ? ? ? ? x <- unlist(lapply(nodes[[i]], seq)) > ? ? ? ? lapply(nodes[(i + 1):(nlevels - 1)], > ? ? ? ? ? ? ? ?function(v) assign("x", rep.int(x, v), envir > parent.frame(2))) > ? ? ? ? m[, i] <- x > > (I think the parent.frame(2) might have to be changed to parent.frame(8) for > that to work.? Such code looks pretty ugly to me but seems to be rare.) > > It also seems to cause problems with some built-in functions: > newlapply <- function (X, FUN, ...)? > { > ? ? FUN <- match.fun(FUN) > ? ? if (!is.list(X))? > ? ? ? ? X <- as.list(X) > ? ? rval <- vector("list", length(X)) > ? ? for (i in seq(along = X)) { > ? ? ? ? rval[i] <- list(local({ > ? ? ? ? ? ? i <- i > ? ? ? ? ? ? FUN(X[[i]], ...) > ? ? ? ? })) > ? ? } > ? ? names(rval) <- names(X) > ? ? return(rval) > } > newlapply(1:2,log) > #Error in FUN(X[[i]], ...) : non-numeric argument to mathematical function > newlapply(1:2,function(x)log(x)) > #[[1]] > #[1] 0 > # > #[[2]] > #[1] 0.6931472 > > > > Bill Dunlap > TIBCO Software > wdunlap tibco.com > > On Tue, Feb 24, 2015 at 7:50 AM, <luke-tierney at uiowa.edu> wrote: > The documentation is not specific enough on the indented > semantics in > this situation to consider this a bug. The original R-level > implementation of lapply was > > ? ? lapply <- function(X, FUN, ...) { > ? ? ? ? FUN <- match.fun(FUN) > ? ? ? ? if (!is.list(X)) > ? ? ? ? X <- as.list(X) > ? ? ? ? rval <- vector("list", length(X)) > ? ? ? ? for(i in seq(along = X)) > ? ? ? ? rval[i] <- list(FUN(X[[i]], ...)) > ? ? ? ? names(rval) <- names(X)? ? ? ? ? ?# keep `names' ! > ? ? ? ? return(rval) > ? ? } > > and the current internal implementation is consistent with this. > With > a loop like this lazy evaluation and binding assignment interact > in > this way; the force() function was introduced to help with this. > > That said, the expression FUN(X[[i]], ...) could be replaced by > > ? ? local({ > ? ? ? ? i <- i > ? ? ? ? list(FUN(X[[i]], ...) > ? ? }) > > which would produce the more desirable result > > ? ? > sapply(test, function(myfn) myfn(2)) > ? ? [1] 2 4 6 8 > > The C implementation could use this approach, or could rebuild > the > expression being evaluated at each call to get almost the same > semantics. > Both would add a little overhead. Some code optimization might > reduce > the overhead in some instances (e.g. if FUN is a BUILTIN), but > it's > not clear that would be worth while. > > Variants of this issue arise in a couple of places so it may be > worth > looking into. > > Best, > > luke > > On Tue, 24 Feb 2015, Radford Neal wrote: > > From: Daniel Kaschek > <daniel.kaschek at physik.uni-freiburg.de> > ... When I evaluate this list of > functions by > another lapply/sapply, I get an > unexpected result: all values coincide. > However, when I uncomment the print(), > it works as expected. Is this a > bug or a feature? > > conditions <- 1:4 > test <- lapply(conditions, > function(mycondition){ > ? #print(mycondition) > ? myfn <- function(i) mycondition*i > ? return(myfn) > }) > > sapply(test, function(myfn) myfn(2)) > > > From: Jeroen Ooms <jeroenooms at gmail.com> > I think it is a bug. If we use > substitute to inspect the promise, it > appears the index number is always equal > to its last value: > > > From: Duncan Temple Lang <dtemplelang at ucdavis.edu> > Not a bug, but does surprise people. It > is lazy evaluation. > > > > I think it is indeed a bug.? The lapply code saves a > bit of time by > reusing the same storage for the scalar index number > every iteration. > This amounts to modifying the R code that was used > for the previous > function call.? There's no justification for doing > this in the > documentation for lapply.? It is certainly not > desired behaviour, > except in so far as it allows a slight savings in > time (which is > minor, given the time that the function call itself > will take). > > ? Radford Neal > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > > > -- > Luke Tierney > Ralph E. Wareham Professor of Mathematical Sciences > University of Iowa? ? ? ? ? ? ? ? ? Phone:? ? ? ? ? ? ?319-335-3386 > Department of Statistics and? ? ? ? Fax:? ? ? ? ? ? ? ?319-335-3017 > ? ?Actuarial Science > 241 Schaeffer Hall? ? ? ? ? ? ? ? ? email:? ?luke-tierney at uiowa.edu > Iowa City, IA 52242? ? ? ? ? ? ? ? ?WWW:? http://www.stat.uiowa.edu > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > > > >-- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tierney at uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
On Thu, 26 Feb 2015, William Dunlap wrote:> ... > It also seems to cause problems with some built-in functions: > newlapply <- function (X, FUN, ...)? > { > ? ? FUN <- match.fun(FUN) > ? ? if (!is.list(X))? > ? ? ? ? X <- as.list(X) > ? ? rval <- vector("list", length(X)) > ? ? for (i in seq(along = X)) { > ? ? ? ? rval[i] <- list(local({ > ? ? ? ? ? ? i <- i > ? ? ? ? ? ? FUN(X[[i]], ...) > ? ? ? ? })) > ? ? } > ? ? names(rval) <- names(X) > ? ? return(rval) > } > newlapply(1:2,log) > #Error in FUN(X[[i]], ...) : non-numeric argument to mathematical functionThis seems to be a bug in log() -- this takes local() out of the issue:> f <- function(x, ...) {+ g <- function() + log(x, ...) + g() + }> f(1)Error in log(x, ...) : non-numeric argument to mathematical function It's not following the ... properly for some reason. Best, luke -- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tierney at uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
On Sun, 1 Mar 2015, luke-tierney at uiowa.edu wrote:> On Thu, 26 Feb 2015, William Dunlap wrote: > >> ... >> It also seems to cause problems with some built-in functions: >> newlapply <- function (X, FUN, ...)? >> { >> ? ? FUN <- match.fun(FUN) >> ? ? if (!is.list(X))? >> ? ? ? ? X <- as.list(X) >> ? ? rval <- vector("list", length(X)) >> ? ? for (i in seq(along = X)) { >> ? ? ? ? rval[i] <- list(local({ >> ? ? ? ? ? ? i <- i >> ? ? ? ? ? ? FUN(X[[i]], ...) >> ? ? ? ? })) >> ? ? } >> ? ? names(rval) <- names(X) >> ? ? return(rval) >> } >> newlapply(1:2,log) >> #Error in FUN(X[[i]], ...) : non-numeric argument to mathematical function > > This seems to be a bug in log() -- this takes local() out of the issue: > >> f <- function(x, ...) { > + g <- function() > + log(x, ...) > + g() > + } >> f(1) > Error in log(x, ...) : non-numeric argument to mathematical function > > It's not following the ... properly for some reason.But no longer a problem in R-devel -- maybe there is a change worth back-porting to R-patched. Best, luke> luke > >-- Luke Tierney Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: luke-tierney at uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu