Actually, R's scope rules are seriously weird. I set out to write an R compiler, wow, >20 years ago. Figured out how to handle optional and keyword parameters efficiently, figured out a lot of other things, but choked on the scope rules. Consider> x <- 1 > f <- function () {+ a <- x + x <- 2 + b <- x + c(a=a, b=b) + }> f()a b 1 2> x[1] 1 It's really not clear what is going on here. However, ?assign can introduce new variables into an environment, and from something like with(df, x*2-y) it is impossible for a compiler to tell which, if either, of x and y is to be obtained from df and which from outside. And of course ?with is just a function:> df <- data.frame(y=24) > w <- with > w(df, x*2-y)[1] -22 So you cannot in general tell *which* function can twist the environment in which its arguments will be evaluated. I got very tired of trying to explore a twisty maze of documentation and trying to infer a specification from examples. I would come up with an ingenious mechanism for making the common case tolerable and the rare cases possible, and then I'd discover a bear trap I hadn't seen. I love R, but I try really hard not to be clever with it. So while R's scoping is *like* lexical scoping, it is *dynamic* lexical scoping, to coin a phrase. On Thu, 26 Sep 2019 at 23:56, Martin M?ller Skarbiniks Pedersen <traxplayer at gmail.com> wrote:> > On Wed, 25 Sep 2019 at 11:03, Francesco Ariis <fa-ml at ariis.it> wrote: > > > > Dear R users/developers, > > while ploughing through "An Introduction to R" [1], I found the > > expression "static scope" (in contraposition to "lexical scope"). > > > > I was a bit puzzled by the difference (since e.g. Wikipedia conflates the > > two) until I found this document [2]. > > > I sometimes teach a little R, and they might ask about static/lexical scope. > My short answer is normally that S uses static scoping and R uses > lexical scoping. > And most all modern languages uses lexical scoping. > So if they know Java, C, C# etc. then the scoping rules for R are the same. > > I finally says that it is not a full answer but enough for most. > > Regards > Martin > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
On 26/09/2019 9:44 a.m., Richard O'Keefe wrote:> Actually, R's scope rules are seriously weird. > I set out to write an R compiler, wow, >20 years ago. > Figured out how to handle optional and keyword parameters efficiently, > figured out a lot of other things, but choked on the scope rules. > Consider > >> x <- 1 >> f <- function () { > + a <- x > + x <- 2 > + b <- x > + c(a=a, b=b) > + } >> f() > a b > 1 2 >> x > [1] 1 > > It's really not clear what is going on here.This is all pretty clear: in the first assignment, x is found in the global environment, because it does not exist in the evaluation frame. In the second assignment, a new variable is created in the evaluation frame. In the third assignment, that new variable is used to set the value of b.> However, ?assign can introduce new variables into an environment, > and from something like > with(df, x*2-y) > it is impossible for a compiler to tell which, if either, of x and y is to > be obtained from df and which from outside. And of course ?with > is just a function: > >> df <- data.frame(y=24) >> w <- with >> w(df, x*2-y) > [1] -22 > > So you cannot in general tell *which* function can twist the environment > in which its arguments will be evaluated.It's definitely hard to compile R because of the scoping rules, but that doesn't make the scoping rules unclear.> I got very tired of trying to explore a twisty maze of documentation and > trying to infer a specification from examples. I would come up with an > ingenious mechanism for making the common case tolerable and the > rare cases possible, and then I'd discover a bear trap I hadn't seen. > I love R, but I try really hard not to be clever with it.I think the specification is really pretty simple. I'm not sure it is well documented anywhere, but I think I understand it pretty well, and it doesn't seem overly complicated to me.> So while R's scoping is *like* lexical scoping, it is *dynamic* lexical > scoping, to coin a phrase.I'd say it is regular lexical scoping but with dynamic variable creation. Call that dynamic lexical scoping if you want, but it's not really a mystery. Duncan Murdoch> > On Thu, 26 Sep 2019 at 23:56, Martin M?ller Skarbiniks Pedersen > <traxplayer at gmail.com> wrote: >> >> On Wed, 25 Sep 2019 at 11:03, Francesco Ariis <fa-ml at ariis.it> wrote: >>> >>> Dear R users/developers, >>> while ploughing through "An Introduction to R" [1], I found the >>> expression "static scope" (in contraposition to "lexical scope"). >>> >>> I was a bit puzzled by the difference (since e.g. Wikipedia conflates the >>> two) until I found this document [2]. >> >> >> I sometimes teach a little R, and they might ask about static/lexical scope. >> My short answer is normally that S uses static scoping and R uses >> lexical scoping. >> And most all modern languages uses lexical scoping. >> So if they know Java, C, C# etc. then the scoping rules for R are the same. >> >> I finally says that it is not a full answer but enough for most. >> >> Regards >> Martin >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
The best summary I've read on the subject of R's scoping rules (in particular how they compare to scoping rules in S-PLUS) is Dr. John Fox's "Frames, Environments, and Scope in R and S-PLUS", written as an Appendix to the first edition of his book, An R and S-PLUS Companion to Applied Regression (2002). In this document Dr. Fox refers to "lexical" scoping primarily, however where "static" scoping is mentioned, it is defined as equivalent to "lexical" scoping. The Appendix is available as a PDF from: https://socialsciences.mcmaster.ca/jfox/Books/Companion-1E/appendix-scope.pdf HTH, Bill. W. Michels, Ph.D. On Thu, Sep 26, 2019 at 7:59 AM Duncan Murdoch <murdoch.duncan at gmail.com> wrote:> > On 26/09/2019 9:44 a.m., Richard O'Keefe wrote: > > Actually, R's scope rules are seriously weird. > > I set out to write an R compiler, wow, >20 years ago. > > Figured out how to handle optional and keyword parameters efficiently, > > figured out a lot of other things, but choked on the scope rules. > > Consider > > > >> x <- 1 > >> f <- function () { > > + a <- x > > + x <- 2 > > + b <- x > > + c(a=a, b=b) > > + } > >> f() > > a b > > 1 2 > >> x > > [1] 1 > > > > It's really not clear what is going on here. > > This is all pretty clear: in the first assignment, x is found in the > global environment, because it does not exist in the evaluation frame. > In the second assignment, a new variable is created in the evaluation > frame. In the third assignment, that new variable is used to set the > value of b. > > > However, ?assign can introduce new variables into an environment, > > and from something like > > with(df, x*2-y) > > it is impossible for a compiler to tell which, if either, of x and y is to > > be obtained from df and which from outside. And of course ?with > > is just a function: > > > >> df <- data.frame(y=24) > >> w <- with > >> w(df, x*2-y) > > [1] -22 > > > > So you cannot in general tell *which* function can twist the environment > > in which its arguments will be evaluated. > > It's definitely hard to compile R because of the scoping rules, but that > doesn't make the scoping rules unclear. > > > I got very tired of trying to explore a twisty maze of documentation and > > trying to infer a specification from examples. I would come up with an > > ingenious mechanism for making the common case tolerable and the > > rare cases possible, and then I'd discover a bear trap I hadn't seen. > > I love R, but I try really hard not to be clever with it. > > I think the specification is really pretty simple. I'm not sure it is > well documented anywhere, but I think I understand it pretty well, and > it doesn't seem overly complicated to me. > > > So while R's scoping is *like* lexical scoping, it is *dynamic* lexical > > scoping, to coin a phrase. > > I'd say it is regular lexical scoping but with dynamic variable > creation. Call that dynamic lexical scoping if you want, but it's not > really a mystery. > > Duncan Murdoch > > > > > On Thu, 26 Sep 2019 at 23:56, Martin M?ller Skarbiniks Pedersen > > <traxplayer at gmail.com> wrote: > >> > >> On Wed, 25 Sep 2019 at 11:03, Francesco Ariis <fa-ml at ariis.it> wrote: > >>> > >>> Dear R users/developers, > >>> while ploughing through "An Introduction to R" [1], I found the > >>> expression "static scope" (in contraposition to "lexical scope"). > >>> > >>> I was a bit puzzled by the difference (since e.g. Wikipedia conflates the > >>> two) until I found this document [2]. > >> > >> > >> I sometimes teach a little R, and they might ask about static/lexical scope. > >> My short answer is normally that S uses static scoping and R uses > >> lexical scoping. > >> And most all modern languages uses lexical scoping. > >> So if they know Java, C, C# etc. then the scoping rules for R are the same. > >> > >> I finally says that it is not a full answer but enough for most. > >> > >> Regards > >> Martin > >> > >> ______________________________________________ > >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > > > > ______________________________________________ > > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > > https://stat.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > > and provide commented, minimal, self-contained, reproducible code. > > > > ______________________________________________ > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
I found this confusing until I learned about environments. The current state of
the environment that was active at the time the function was defined is
searched, not a frozen copy of the enclosing environment as it existed at the
time the function was defined.
x <- 1 # as it was when f was created
f <- function () {
a <- x # x not yet in current environment
x <- 2 # always modifies current environment
b <- x # finds the definition nearest along the enclosing environments list
c(a=a, b=b)
}
x <- 3 # change after f was created
f()
## a b
## 3 2
x # changes within function don't affect enclosing or calling environments
## [1] 3
Duncan refers to "evaluation frame" (a term described in the R
Language Definition) but I tend to think of a "current environment"
and two linked lists of supplementary environments: the search path of
`parent.env`()s (a.k.a. enclosing environments built from the current
environments that were active when functions were defined) and the call chain of
`parent.frame`()s (function environments that were active at the point functions
were called). You can almost forget about the parent.frame chain unless you are
creating your own non-standard evaluation functions (like `with` or `subset`)...
Environments mutate over time. Specifically, the list of enclosing environments
doesn't change but the variables in them do.
(Credit to Hadley Wickham's Advanced R, criticisms to me.)
On September 26, 2019 7:58:57 AM PDT, Duncan Murdoch <murdoch.duncan at
gmail.com> wrote:>On 26/09/2019 9:44 a.m., Richard O'Keefe wrote:
>> Actually, R's scope rules are seriously weird.
>> I set out to write an R compiler, wow, >20 years ago.
>> Figured out how to handle optional and keyword parameters
>efficiently,
>> figured out a lot of other things, but choked on the scope rules.
>> Consider
>>
>>> x <- 1
>>> f <- function () {
>> + a <- x
>> + x <- 2
>> + b <- x
>> + c(a=a, b=b)
>> + }
>>> f()
>> a b
>> 1 2
>>> x
>> [1] 1
>>
>> It's really not clear what is going on here.
>
>This is all pretty clear: in the first assignment, x is found in the
>global environment, because it does not exist in the evaluation frame.
>In the second assignment, a new variable is created in the evaluation
>frame. In the third assignment, that new variable is used to set the
>value of b.
>
>> However, ?assign can introduce new variables into an environment,
>> and from something like
>> with(df, x*2-y)
>> it is impossible for a compiler to tell which, if either, of x and y
>is to
>> be obtained from df and which from outside. And of course ?with
>> is just a function:
>>
>>> df <- data.frame(y=24)
>>> w <- with
>>> w(df, x*2-y)
>> [1] -22
>>
>> So you cannot in general tell *which* function can twist the
>environment
>> in which its arguments will be evaluated.
>
>It's definitely hard to compile R because of the scoping rules, but
>that
>doesn't make the scoping rules unclear.
>
>> I got very tired of trying to explore a twisty maze of documentation
>and
>> trying to infer a specification from examples. I would come up with
>an
>> ingenious mechanism for making the common case tolerable and the
>> rare cases possible, and then I'd discover a bear trap I hadn't
seen.
>> I love R, but I try really hard not to be clever with it.
>
>I think the specification is really pretty simple. I'm not sure it is
>well documented anywhere, but I think I understand it pretty well, and
>it doesn't seem overly complicated to me.
>
>> So while R's scoping is *like* lexical scoping, it is *dynamic*
>lexical
>> scoping, to coin a phrase.
>
>I'd say it is regular lexical scoping but with dynamic variable
>creation. Call that dynamic lexical scoping if you want, but it's not
>really a mystery.
>
>Duncan Murdoch
>
>>
>> On Thu, 26 Sep 2019 at 23:56, Martin M?ller Skarbiniks Pedersen
>> <traxplayer at gmail.com> wrote:
>>>
>>> On Wed, 25 Sep 2019 at 11:03, Francesco Ariis <fa-ml at
ariis.it>
>wrote:
>>>>
>>>> Dear R users/developers,
>>>> while ploughing through "An Introduction to R" [1], I
found the
>>>> expression "static scope" (in contraposition to
"lexical scope").
>>>>
>>>> I was a bit puzzled by the difference (since e.g. Wikipedia
>conflates the
>>>> two) until I found this document [2].
>>>
>>>
>>> I sometimes teach a little R, and they might ask about
>static/lexical scope.
>>> My short answer is normally that S uses static scoping and R uses
>>> lexical scoping.
>>> And most all modern languages uses lexical scoping.
>>> So if they know Java, C, C# etc. then the scoping rules for R are
>the same.
>>>
>>> I finally says that it is not a full answer but enough for most.
>>>
>>> Regards
>>> Martin
>>>
>>> ______________________________________________
>>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>> ______________________________________________
>> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
--
Sent from my phone. Please excuse my brevity.
I didn't say R's rules were "a mystery" or "overly
complicated", but
that they are "weird".
When I started trying to compile R, with() did not exist, if I
remember correctly.
Half the point of compiling is to do variable lookups at compile time.
In an example like
function (...) {
use x
x <- ...
use x
}
it's easy enough to tell one x from the other. Now change it slightly:
function (...) {
use x
if (...) x <- ...
use x
}
Now we cannot tell whether the second use refers to an inner x or the outer one.
Consider next> f <- function () {
+ g <- function () { x }
+ x <- 2
+ g()'
+ }> x <- 1
> f()
Here's the snag: the x in g does not refer to the x that is visible at
the time when
g is *defined* (as in every other language with lexical scope I've
used) but to the
x that is visible when g is *called*. In other languages, you can
'trim' the
lexical environment of a closure to just the variables that the
closure mentions.
In R you cannot.
It gets nastier. A function can *remove* a variable from another
function's frame.
(See ?rm and note the 'envir' argument and then read ?sys.frame.)
The best I was able to come up with was a scheme where each statically visible
variable had a slot for its value and a reserved object.
Mention of x =>
t := static slot for x
if t is the reserved object: t := full lookup("x")
Assignment to x =>
static slot for := t
That doesn't work with "with", and then of course there are
"active
bindings" nowadays,
see ?bindenv.
On Fri, 27 Sep 2019 at 02:59, Duncan Murdoch <murdoch.duncan at gmail.com>
wrote:>
> On 26/09/2019 9:44 a.m., Richard O'Keefe wrote:
> > Actually, R's scope rules are seriously weird.
> > I set out to write an R compiler, wow, >20 years ago.
> > Figured out how to handle optional and keyword parameters efficiently,
> > figured out a lot of other things, but choked on the scope rules.
> > Consider
> >
> >> x <- 1
> >> f <- function () {
> > + a <- x
> > + x <- 2
> > + b <- x
> > + c(a=a, b=b)
> > + }
> >> f()
> > a b
> > 1 2
> >> x
> > [1] 1
> >
> > It's really not clear what is going on here.
>
> This is all pretty clear: in the first assignment, x is found in the
> global environment, because it does not exist in the evaluation frame.
> In the second assignment, a new variable is created in the evaluation
> frame. In the third assignment, that new variable is used to set the
> value of b.
>
> > However, ?assign can introduce new variables into an environment,
> > and from something like
> > with(df, x*2-y)
> > it is impossible for a compiler to tell which, if either, of x and y
is to
> > be obtained from df and which from outside. And of course ?with
> > is just a function:
> >
> >> df <- data.frame(y=24)
> >> w <- with
> >> w(df, x*2-y)
> > [1] -22
> >
> > So you cannot in general tell *which* function can twist the
environment
> > in which its arguments will be evaluated.
>
> It's definitely hard to compile R because of the scoping rules, but
that
> doesn't make the scoping rules unclear.
>
> > I got very tired of trying to explore a twisty maze of documentation
and
> > trying to infer a specification from examples. I would come up with
an
> > ingenious mechanism for making the common case tolerable and the
> > rare cases possible, and then I'd discover a bear trap I
hadn't seen.
> > I love R, but I try really hard not to be clever with it.
>
> I think the specification is really pretty simple. I'm not sure it is
> well documented anywhere, but I think I understand it pretty well, and
> it doesn't seem overly complicated to me.
>
> > So while R's scoping is *like* lexical scoping, it is *dynamic*
lexical
> > scoping, to coin a phrase.
>
> I'd say it is regular lexical scoping but with dynamic variable
> creation. Call that dynamic lexical scoping if you want, but it's not
> really a mystery.
>
> Duncan Murdoch
>
> >
> > On Thu, 26 Sep 2019 at 23:56, Martin M?ller Skarbiniks Pedersen
> > <traxplayer at gmail.com> wrote:
> >>
> >> On Wed, 25 Sep 2019 at 11:03, Francesco Ariis <fa-ml at
ariis.it> wrote:
> >>>
> >>> Dear R users/developers,
> >>> while ploughing through "An Introduction to R" [1],
I found the
> >>> expression "static scope" (in contraposition to
"lexical scope").
> >>>
> >>> I was a bit puzzled by the difference (since e.g. Wikipedia
conflates the
> >>> two) until I found this document [2].
> >>
> >>
> >> I sometimes teach a little R, and they might ask about
static/lexical scope.
> >> My short answer is normally that S uses static scoping and R uses
> >> lexical scoping.
> >> And most all modern languages uses lexical scoping.
> >> So if they know Java, C, C# etc. then the scoping rules for R are
the same.
> >>
> >> I finally says that it is not a full answer but enough for most.
> >>
> >> Regards
> >> Martin
> >>
> >> ______________________________________________
> >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >
> > ______________________________________________
> > R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>