R devel - Oct 2010 - scoping goes wrong when some functions are used within others.

Dear,

I'm following the r tag on stackoverflow.com, and couldn't but notice
there are quite some questions popping up that deal with scoping in
relation to custom functions. I grinded my teeth on it already, and I
have absolutely no clue what goes wrong. The general pattern is as
follows :

ff <- function(x){
    y <- some_value
    some_function(y)
}
> ff(x)
Error in eval(expr, envir, enclos) : object 'y' not found

I tried to report this as a bug earlier, but got the message I used
the wrong channel. I also don't know how to formalize it into a bug
report on the report site. That's why I bring it to your attention
this way, and want to ask you whether this is by design and we're all
doing something wrong, whether these are problems within certain
packages/situations, ...

I solve these problems now by adding an environment to my global
environment, and delete it after the function finished running. But
this can't be the correct way.

The problem is described here :
http://stackoverflow.com/questions/3840769/scoping-and-functions-in-r-2-11-1-whats-going-wrong

Links to different reports, all having that same pattern but with
different functions :

http://stackoverflow.com/questions/3742415/r-statistical-scoping-error-using-transformby-part-of-the-doby-package
http://stackoverflow.com/questions/3768417/how-to-use-acast-reshape2-within-a-function-in-r
http://stackoverflow.com/questions/3661500/why-cant-i-pass-a-dataset-to-a-function
http://stackoverflow.com/questions/3574858/values-not-being-copied-to-the-next-local-environment
http://stackoverflow.com/questions/2646402/using-functions-and-environments


-- 
Joris Meys
Statistical consultant

Ghent University
Faculty of Bioscience Engineering
Department of Applied mathematics, biometrics and process control

tel : +32 9 264 59 87
Joris.Meys at Ugent.be
-------------------------------
Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php

Joris Meys wrote:
> Dear,
> 
> I'm following the r tag on stackoverflow.com, and couldn't but
notice
> there are quite some questions popping up that deal with scoping in
> relation to custom functions. I grinded my teeth on it already, and I
> have absolutely no clue what goes wrong. The general pattern is as
> follows :
> 
> ff <- function(x){
>     y <- some_value
>     some_function(y)
> }
> 
>> ff(x)
> Error in eval(expr, envir, enclos) : object 'y' not found
I get:

Error in ff(x) : object 'some_value' not found

which is entirely consistent, since it has not been bound to a value.


> 
> I tried to report this as a bug earlier, but got the message I used
> the wrong channel. I also don't know how to formalize it into a bug
> report on the report site. That's why I bring it to your attention
> this way, and want to ask you whether this is by design and we're all
> doing something wrong, whether these are problems within certain
> packages/situations, ...
> 
> I solve these problems now by adding an environment to my global
> environment, and delete it after the function finished running. But
> this can't be the correct way.
> 
> The problem is described here :
>
http://stackoverflow.com/questions/3840769/scoping-and-functions-in-r-2-11-1-whats-going-wrong
> 
> Links to different reports, all having that same pattern but with
> different functions :
> 
>
http://stackoverflow.com/questions/3742415/r-statistical-scoping-error-using-transformby-part-of-the-doby-package
>
http://stackoverflow.com/questions/3768417/how-to-use-acast-reshape2-within-a-function-in-r
>
http://stackoverflow.com/questions/3661500/why-cant-i-pass-a-dataset-to-a-function
>
http://stackoverflow.com/questions/3574858/values-not-being-copied-to-the-next-local-environment
> http://stackoverflow.com/questions/2646402/using-functions-and-environments
> 
>

On 01/10/2010 12:00 PM, Joris Meys wrote:
> Dear,
>
> I'm following the r tag on stackoverflow.com, and couldn't but
notice
> there are quite some questions popping up that deal with scoping in
> relation to custom functions. I grinded my teeth on it already, and I
> have absolutely no clue what goes wrong. The general pattern is as
> follows :
I think each of the reports is really a separate bug, mostly in the 
implementation of "some_function". As far as I could see, only the
last one

|y<- new.env()
with(y, x<- 1)
f<- function(env,z) {
     with(env, x+z)
}
f(y,z=1)
|

involves base R functions, and there I think the problem is with your 
reading of the documentation, not with the function. The documentation 
may suggest that should work by saying "The environment has the
caller's
environment as its parent", but there's no way it possibly could. 
Environments only have one parent. If you read carefully you'll see that 
this is documented correctly in "(Note: if ?data? is already an 
environment then this is used with its existing parent.)"

Duncan Murdoch
> ff<- function(x){
>      y<- some_value
>      some_function(y)
> }
>
> >  ff(x)
> Error in eval(expr, envir, enclos) : object 'y' not found
>
> I tried to report this as a bug earlier, but got the message I used
> the wrong channel. I also don't know how to formalize it into a bug
> report on the report site. That's why I bring it to your attention
> this way, and want to ask you whether this is by design and we're all
> doing something wrong, whether these are problems within certain
> packages/situations, ...
>
> I solve these problems now by adding an environment to my global
> environment, and delete it after the function finished running. But
> this can't be the correct way.
>
> The problem is described here :
>
http://stackoverflow.com/questions/3840769/scoping-and-functions-in-r-2-11-1-whats-going-wrong
>
> Links to different reports, all having that same pattern but with
> different functions :
>
>
http://stackoverflow.com/questions/3742415/r-statistical-scoping-error-using-transformby-part-of-the-doby-package
>
http://stackoverflow.com/questions/3768417/how-to-use-acast-reshape2-within-a-function-in-r
>
http://stackoverflow.com/questions/3661500/why-cant-i-pass-a-dataset-to-a-function
>
http://stackoverflow.com/questions/3574858/values-not-being-copied-to-the-next-local-environment
> http://stackoverflow.com/questions/2646402/using-functions-and-environments
>
>

One of the complaints in your stackoverlow references
concerned doBy::transformBy and the error message came
from eval().  Using eval() means you are throwing the
usual scoping rules out the window and making up some
new ones.  Using a non-core package means you are at
the mercy of its writer.

In transformBy I believe there is an error in the use
of parent.frame() that can be fixed by changing the
current

  > head(doBy::transformBy)
                                                                  
  1 function (formula, data, ...)                                 
  2 {                                                             
  3     transform2 <- function(data, ...) {                       
  4         e <- eval(substitute(list(...)), data, parent.frame())
  5         tags <- names(e)                                      
  6         inx <- match(tags, names(data))

to
                       
  > head(transformBy)
                                                              
  1 function (formula, data, ...)                               
  2 {                                                           
  3     PARENT.FRAME <- parent.frame()                          
  4     transform2 <- function(data, ...) {                     
  5         e <- eval(substitute(list(...)), data, PARENT.FRAME)
  6         tags <- names(e)             

transform2's parent.frame will always be transformBy
and we really want to look for variables in the expressions
in the .. list in transformBy's parent frame, not in
transformBy itself.

Using parent.frame() often causes problems because you don't
always know how many frames come between your user's code and
your function's code.  (E.g., using methods may insert extra
frames.)  In this case I think it was always looking in the
wrong place.  The fix doesn't guarentee that it will always
look in the right place, but it should be right more often.

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com  
> -----Original Message-----
> From: r-devel-bounces at r-project.org 
> [mailto:r-devel-bounces at r-project.org] On Behalf Of Joris Meys
> Sent: Friday, October 01, 2010 9:01 AM
> To: r-devel at r-project.org
> Subject: [Rd] scoping goes wrong when some functions are used 
> within others.
> 
> Dear,
> 
> I'm following the r tag on stackoverflow.com, and couldn't but
notice
> there are quite some questions popping up that deal with scoping in
> relation to custom functions. I grinded my teeth on it already, and I
> have absolutely no clue what goes wrong. The general pattern is as
> follows :
> 
> ff <- function(x){
>     y <- some_value
>     some_function(y)
> }
> 
> > ff(x)
> Error in eval(expr, envir, enclos) : object 'y' not found
> 
> I tried to report this as a bug earlier, but got the message I used
> the wrong channel. I also don't know how to formalize it into a bug
> report on the report site. That's why I bring it to your attention
> this way, and want to ask you whether this is by design and we're all
> doing something wrong, whether these are problems within certain
> packages/situations, ...
> 
> I solve these problems now by adding an environment to my global
> environment, and delete it after the function finished running. But
> this can't be the correct way.
> 
> The problem is described here :
> http://stackoverflow.com/questions/3840769/scoping-and-functio
ns-in-r-2-11-1-whats-going-wrong
> 
> Links to different reports, all having that same pattern but with
> different functions :
> 
> http://stackoverflow.com/questions/3742415/r-statistical-scopi
ng-error-using-transformby-part-of-the-doby-package
> http://stackoverflow.com/questions/3768417/how-to-use-acast-re
shape2-within-a-function-in-r
> http://stackoverflow.com/questions/3661500/why-cant-i-pass-a-d
ataset-to-a-function
> http://stackoverflow.com/questions/3574858/values-not-being-co
pied-to-the-next-local-environment
> http://stackoverflow.com/questions/2646402/using-functions-and
-environments
> 
> 
> -- 
> Joris Meys
> Statistical consultant
> 
> Ghent University
> Faculty of Bioscience Engineering
> Department of Applied mathematics, biometrics and process control
> 
> tel : +32 9 264 59 87
> Joris.Meys at Ugent.be
> -------------------------------
> Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>