R devel - Mar 2017 - RFC: (in-principle) native unquoting for standard evaluation

(please be gentle, it's my first time)

I am interested in discussions (possibly reiterating past threads --
searching didn't turn up much) on the possibility of supporting standard
evaluation unquoting at the language level. This has been brought up in a
recent similar thread here [1] and on Twitter [2] where I proposed the
following desired (in-principle) syntax

    f <- function(col1, col2, new_col_name) {
        mtcars %>% mutate(@new_col_name = @col1 + @col2)
    }

or closer to home

    x <- 1:10; y <- "x"
    data.frame(z = @y)

where @ would be defined as a unary prefix operator which substitutes the
quoted variable name in-place, to allow more flexibility of NSE functions
within a programming context. This mechanism exists within MySQL [3] (and
likely other languages) and could potentially be extremely useful. Several
alternatives have been incorporated into packages (most recently work
on tidyeval) none of which appear to fully match the simplicity of the
above, and some of which cut a forceful path through the syntax tree.

The exact syntax isn't my concern at the moment (@ vs unquote() or other,
though the first requires user-supplied native prefix support within the
language, as per [1]) and neither is the exact way in which this would be
achieved (well above my pay grade). The practicality of @ being on the LHS
of `=` is also of a lesser concern (likely greater complexity) than the RHS.

I hear there exists (justified) reluctance to add new syntax to the
language, but I think this has sufficient merit (and a growing number of
workarounds) to warrant continued discussion.

With kindest regards,

- Jonathan.

[1] https://stat.ethz.ch/pipermail/r-devel/2017-March/073894.html
[2] https://twitter.com/carroll_jono/status/842142292253196290
[3] https://dev.mysql.com/doc/refman/5.7/en/user-variables.html

	[[alternative HTML version deleted]]

Interesting idea. Lazy and non-standard evaluation is going to happen; the
language needs a way to contain it.

I'll extend the proposal so that prefixing a formal argument with @ in
function() marks the argument as auto-quoting, so it arrives as a language
object without use of substitute(). Kind of like how '*' in C declares a
pointer and dereferences one.

subset <- function(x, @subset, ...) { }

This should make it easier to implement such functions, simplify
compilation, and allow detection of potential quoting errors through static
analysis.

Michael

On Thu, Mar 16, 2017 at 5:03 PM, Jonathan Carroll <jono at
jcarroll.com.au>
wrote:
> (please be gentle, it's my first time)
>
> I am interested in discussions (possibly reiterating past threads --
> searching didn't turn up much) on the possibility of supporting
standard
> evaluation unquoting at the language level. This has been brought up in a
> recent similar thread here [1] and on Twitter [2] where I proposed the
> following desired (in-principle) syntax
>
>     f <- function(col1, col2, new_col_name) {
>         mtcars %>% mutate(@new_col_name = @col1 + @col2)
>     }
>
> or closer to home
>
>     x <- 1:10; y <- "x"
>     data.frame(z = @y)
>
> where @ would be defined as a unary prefix operator which substitutes the
> quoted variable name in-place, to allow more flexibility of NSE functions
> within a programming context. This mechanism exists within MySQL [3] (and
> likely other languages) and could potentially be extremely useful. Several
> alternatives have been incorporated into packages (most recently work
> on tidyeval) none of which appear to fully match the simplicity of the
> above, and some of which cut a forceful path through the syntax tree.
>
> The exact syntax isn't my concern at the moment (@ vs unquote() or
other,
> though the first requires user-supplied native prefix support within the
> language, as per [1]) and neither is the exact way in which this would be
> achieved (well above my pay grade). The practicality of @ being on the LHS
> of `=` is also of a lesser concern (likely greater complexity) than the
> RHS.
>
> I hear there exists (justified) reluctance to add new syntax to the
> language, but I think this has sufficient merit (and a growing number of
> workarounds) to warrant continued discussion.
>
> With kindest regards,
>
> - Jonathan.
>
> [1] https://stat.ethz.ch/pipermail/r-devel/2017-March/073894.html
> [2] https://twitter.com/carroll_jono/status/842142292253196290
> [3] https://dev.mysql.com/doc/refman/5.7/en/user-variables.html
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]

I love the pointer analogy. Presumably the additional complication of scope
breaks this however. * itself would have been a nice operator for this were
it not prone to ambiguity (`a * *b` vs `a**b`, from which @ does not
suffer).

Would this extension require that function authors explicitly enable
auto-quoting support? I somewhat envisioned functions seeing the resolved
unquoted object (within their calling scope) so that they could retain
their standard defintions when not using @. In my mutate example, mutate
itself could simply be the NSE version, so

    mutate(mtcars, z = mpg)

would work as normal, but

    x = "mpg"
    mutate(mtcars, z = @x)

would produce the same result (x may be changing within a loop or be
defined through a formal argument). Here, @x would resolve to `mpg` and
mutate would retain the duty of resolving that to mtcars$mpg as per normal.

A seperate SE version would not be required (as arguments could be set
programatically), but an additional flexibility could be @ acting on a
string rather than an object for direct unquoting

    mutate(mtcars, z = @"mpg")

for when the name is known but NSE isn't desired (which would also assist
with the whole utils::globalVariables() vs CRAN checks concern).

Having a formal argument forcefully auto-unquote would prevent standard
usage unless there was a way to also disable it. Unless I'm missing an
angle (which I very likely am) wouldn't it be better to have the user
supply an @-prefixed argument and retain the connection to the calling
scope?

Apologies if I have any of that confused or there are better approaches. I
merely have a desire for this to work and am learning as much as possible
about "how" as I go.

Your comments are greatly appreciated.

- Jonathan.

On Fri, 17 Mar 2017 at 21:00, Michael Lawrence <lawrence.michael at
gene.com>
wrote:

Interesting idea. Lazy and non-standard evaluation is going to happen; the
language needs a way to contain it.

I'll extend the proposal so that prefixing a formal argument with @ in
function() marks the argument as auto-quoting, so it arrives as a language
object without use of substitute(). Kind of like how '*' in C declares a
pointer and dereferences one.

subset <- function(x, @subset, ...) { }

This should make it easier to implement such functions, simplify
compilation, and allow detection of potential quoting errors through static
analysis.

Michael

On Thu, Mar 16, 2017 at 5:03 PM, Jonathan Carroll <jono at
jcarroll.com.au>
wrote:

(please be gentle, it's my first time)

I am interested in discussions (possibly reiterating past threads --
searching didn't turn up much) on the possibility of supporting standard
evaluation unquoting at the language level. This has been brought up in a
recent similar thread here [1] and on Twitter [2] where I proposed the
following desired (in-principle) syntax

    f <- function(col1, col2, new_col_name) {
        mtcars %>% mutate(@new_col_name = @col1 + @col2)
    }

or closer to home

    x <- 1:10; y <- "x"
    data.frame(z = @y)

where @ would be defined as a unary prefix operator which substitutes the
quoted variable name in-place, to allow more flexibility of NSE functions
within a programming context. This mechanism exists within MySQL [3] (and
likely other languages) and could potentially be extremely useful. Several
alternatives have been incorporated into packages (most recently work
on tidyeval) none of which appear to fully match the simplicity of the
above, and some of which cut a forceful path through the syntax tree.

The exact syntax isn't my concern at the moment (@ vs unquote() or other,
though the first requires user-supplied native prefix support within the
language, as per [1]) and neither is the exact way in which this would be
achieved (well above my pay grade). The practicality of @ being on the LHS
of `=` is also of a lesser concern (likely greater complexity) than the RHS.

I hear there exists (justified) reluctance to add new syntax to the
language, but I think this has sufficient merit (and a growing number of
workarounds) to warrant continued discussion.

With kindest regards,

- Jonathan.

[1] https://stat.ethz.ch/pipermail/r-devel/2017-March/073894.html
[2] https://twitter.com/carroll_jono/status/842142292253196290
[3] https://dev.mysql.com/doc/refman/5.7/en/user-variables.html

        [[alternative HTML version deleted]]

______________________________________________
R-devel at r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

	[[alternative HTML version deleted]]

What would you propose for the unquote-splice operator?

Hadley

On Friday, March 17, 2017, Jonathan Carroll <jono at jcarroll.com.au>
wrote:
> (please be gentle, it's my first time)
>
> I am interested in discussions (possibly reiterating past threads --
> searching didn't turn up much) on the possibility of supporting
standard
> evaluation unquoting at the language level. This has been brought up in a
> recent similar thread here [1] and on Twitter [2] where I proposed the
> following desired (in-principle) syntax
>
>     f <- function(col1, col2, new_col_name) {
>         mtcars %>% mutate(@new_col_name = @col1 + @col2)
>     }
>
> or closer to home
>
>     x <- 1:10; y <- "x"
>     data.frame(z = @y)
>
> where @ would be defined as a unary prefix operator which substitutes the
> quoted variable name in-place, to allow more flexibility of NSE functions
> within a programming context. This mechanism exists within MySQL [3] (and
> likely other languages) and could potentially be extremely useful. Several
> alternatives have been incorporated into packages (most recently work
> on tidyeval) none of which appear to fully match the simplicity of the
> above, and some of which cut a forceful path through the syntax tree.
>
> The exact syntax isn't my concern at the moment (@ vs unquote() or
other,
> though the first requires user-supplied native prefix support within the
> language, as per [1]) and neither is the exact way in which this would be
> achieved (well above my pay grade). The practicality of @ being on the LHS
> of `=` is also of a lesser concern (likely greater complexity) than the
> RHS.
>
> I hear there exists (justified) reluctance to add new syntax to the
> language, but I think this has sufficient merit (and a growing number of
> workarounds) to warrant continued discussion.
>
> With kindest regards,
>
> - Jonathan.
>
> [1] https://stat.ethz.ch/pipermail/r-devel/2017-March/073894.html
> [2] https://twitter.com/carroll_jono/status/842142292253196290
> [3] https://dev.mysql.com/doc/refman/5.7/en/user-variables.html
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org <javascript:;> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
http://hadley.nz

	[[alternative HTML version deleted]]

Would this return a quosure? (i.e. a single sided formula that captures
both expression and environment). That's the data structure we've
adopted
in tidyeval as it already has some built in support.

Hadley

On Friday, March 17, 2017, Michael Lawrence <lawrence.michael at gene.com>
wrote:
> Interesting idea. Lazy and non-standard evaluation is going to happen; the
> language needs a way to contain it.
>
> I'll extend the proposal so that prefixing a formal argument with @ in
> function() marks the argument as auto-quoting, so it arrives as a language
> object without use of substitute(). Kind of like how '*' in C
declares a
> pointer and dereferences one.
>
> subset <- function(x, @subset, ...) { }
>
> This should make it easier to implement such functions, simplify
> compilation, and allow detection of potential quoting errors through static
> analysis.
>
> Michael
>
> On Thu, Mar 16, 2017 at 5:03 PM, Jonathan Carroll <jono at
jcarroll.com.au
> <javascript:;>>
> wrote:
>
> > (please be gentle, it's my first time)
> >
> > I am interested in discussions (possibly reiterating past threads --
> > searching didn't turn up much) on the possibility of supporting
standard
> > evaluation unquoting at the language level. This has been brought up
in a
> > recent similar thread here [1] and on Twitter [2] where I proposed the
> > following desired (in-principle) syntax
> >
> >     f <- function(col1, col2, new_col_name) {
> >         mtcars %>% mutate(@new_col_name = @col1 + @col2)
> >     }
> >
> > or closer to home
> >
> >     x <- 1:10; y <- "x"
> >     data.frame(z = @y)
> >
> > where @ would be defined as a unary prefix operator which substitutes
the
> > quoted variable name in-place, to allow more flexibility of NSE
functions
> > within a programming context. This mechanism exists within MySQL [3]
(and
> > likely other languages) and could potentially be extremely useful.
> Several
> > alternatives have been incorporated into packages (most recently work
> > on tidyeval) none of which appear to fully match the simplicity of the
> > above, and some of which cut a forceful path through the syntax tree.
> >
> > The exact syntax isn't my concern at the moment (@ vs unquote() or
other,
> > though the first requires user-supplied native prefix support within
the
> > language, as per [1]) and neither is the exact way in which this would
be
> > achieved (well above my pay grade). The practicality of @ being on the
> LHS
> > of `=` is also of a lesser concern (likely greater complexity) than
the
> > RHS.
> >
> > I hear there exists (justified) reluctance to add new syntax to the
> > language, but I think this has sufficient merit (and a growing number
of
> > workarounds) to warrant continued discussion.
> >
> > With kindest regards,
> >
> > - Jonathan.
> >
> > [1] https://stat.ethz.ch/pipermail/r-devel/2017-March/073894.html
> > [2] https://twitter.com/carroll_jono/status/842142292253196290
> > [3] https://dev.mysql.com/doc/refman/5.7/en/user-variables.html
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > R-devel at r-project.org <javascript:;> mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org <javascript:;> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

-- 
http://hadley.nz

	[[alternative HTML version deleted]]