R devel - Mar 2017 - RFC: (in-principle) native unquoting for standard evaluation

Michael Lawrence (as last in long series of posters)...
> Yes, it would bind the language object to the environment, like an
> R-level promise (but "promise" of course refers specifically to
just
> _lazy_ evaluation).
> 
> For the uqs() thing, expanding calls like that is somewhat orthogonal
> to NSE. It would be nice in general to be able to write something like
> mean(x, extra_args...) without resorting to do.call(mean, c(list(x),
> extra_args)). If we had that then uqs() would just be the combination
> of unquote and expansion, i.e., mean(x, @extra_args...). The
"..."
> postfix would not work since it's still a valid symbol name, but we
> could come up with something.

I've been trying to follow this proposal, though without tracking down
all the tweets, etc. that are referenced.  I suspect I'm not the only
reader who isn't clear exactly what is being proposed.  I think a
detailed, self-contained proposal would be useful.

One thing I'm not clear on is whether the proposal would add anything
semantically beyond what the present "eval" and "substitute"
functions
can do fairly easily.  If not, is there really any need for a slightly
more concise syntax?  Is it expected that the new syntax would be used
lots by ordinary users, or is it only for the convenience of people
who are writing fairly esoteric functions (which might then be used by
many)?  If the later, it seems undesirable to me.

There is an opportunity cost to grabbing the presently-unused unary @
operator for this, in that it might otherwise be used for some other
extension.  For example, see the last five slides in my talk at
http://www.cs.utoronto.ca/~radford/ftp/R-lang-ext.pdf for a different
proposal for a new unary @ operator.  I'm not necessarily advocating
that particular use (my ideas in this respect are still undergoing
revisions), but the overall point is that there may well be several
good uses of a unary @ operator (and there aren't many other good
characters to use for a unary operator besides @).  It is unclear to
me that the current proposal is the highest-value use of @.

   Radford Neal

On Mon, Mar 20, 2017 at 7:36 AM, Radford Neal <radford at cs.toronto.edu>
wrote:
> Michael Lawrence (as last in long series of posters)...
>
>> Yes, it would bind the language object to the environment, like an
>> R-level promise (but "promise" of course refers specifically
to just
>> _lazy_ evaluation).
>>
>> For the uqs() thing, expanding calls like that is somewhat orthogonal
>> to NSE. It would be nice in general to be able to write something like
>> mean(x, extra_args...) without resorting to do.call(mean, c(list(x),
>> extra_args)). If we had that then uqs() would just be the combination
>> of unquote and expansion, i.e., mean(x, @extra_args...). The
"..."
>> postfix would not work since it's still a valid symbol name, but we
>> could come up with something.
>
>
> I've been trying to follow this proposal, though without tracking down
> all the tweets, etc. that are referenced.  I suspect I'm not the only
> reader who isn't clear exactly what is being proposed.  I think a
> detailed, self-contained proposal would be useful.
We have a working implementation (which I'm calling tidyeval) in
https://github.com/hadley/rlang, but we have yet to write it up. We'll
spend some time documenting since it seems to be of broader interest.
> One thing I'm not clear on is whether the proposal would add anything
> semantically beyond what the present "eval" and
"substitute" functions
> can do fairly easily.  If not, is there really any need for a slightly
> more concise syntax?  Is it expected that the new syntax would be used
> lots by ordinary users, or is it only for the convenience of people
> who are writing fairly esoteric functions (which might then be used by
> many)?  If the later, it seems undesirable to me.
I accidentally responded off list to Michael, but I think there are
three legs to "tidy" style of NSE:

1) capturing a quosure from a promise

2) quasiquotation (unquote + unquote-splice)

3) pronouns, so you can be explicit about where a variable should be
looked up (.data vs .end)

These are largely orthogonal, but I don't think you can solve the most
important NSE problems without all three. Just having 1) in base R
would be a big step forward.
> There is an opportunity cost to grabbing the presently-unused unary @
> operator for this, in that it might otherwise be used for some other
> extension.  For example, see the last five slides in my talk at
> http://www.cs.utoronto.ca/~radford/ftp/R-lang-ext.pdf for a different
> proposal for a new unary @ operator.  I'm not necessarily advocating
> that particular use (my ideas in this respect are still undergoing
> revisions), but the overall point is that there may well be several
> good uses of a unary @ operator (and there aren't many other good
> characters to use for a unary operator besides @).  It is unclear to
> me that the current proposal is the highest-value use of @.
A further extension would be to allow binary @ in function argument
names; then the LHS could be an arbitrary string used as an extension
mechanism.

Hadley

-- 
http://hadley.nz

On Mon, Mar 20, 2017 at 8:00 AM, Hadley Wickham <h.wickham at gmail.com>
wrote:
> On Mon, Mar 20, 2017 at 7:36 AM, Radford Neal <radford at
cs.toronto.edu> wrote:
>> Michael Lawrence (as last in long series of posters)...
>>
>>> Yes, it would bind the language object to the environment, like an
>>> R-level promise (but "promise" of course refers
specifically to just
>>> _lazy_ evaluation).
>>>
>>> For the uqs() thing, expanding calls like that is somewhat
orthogonal
>>> to NSE. It would be nice in general to be able to write something
like
>>> mean(x, extra_args...) without resorting to do.call(mean,
c(list(x),
>>> extra_args)). If we had that then uqs() would just be the
combination
>>> of unquote and expansion, i.e., mean(x, @extra_args...). The
"..."
>>> postfix would not work since it's still a valid symbol name,
but we
>>> could come up with something.
>>
>>
>> I've been trying to follow this proposal, though without tracking
down
>> all the tweets, etc. that are referenced.  I suspect I'm not the
only
>> reader who isn't clear exactly what is being proposed.  I think a
>> detailed, self-contained proposal would be useful.
>
> We have a working implementation (which I'm calling tidyeval) in
> https://github.com/hadley/rlang, but we have yet to write it up. We'll
> spend some time documenting since it seems to be of broader interest.
First pass at programming dplyr vignette (including details about tidyeval) at
http://rpubs.com/hadley/dplyr-programming

Hadley

-- 
http://hadley.nz

RN> There is an opportunity cost to grabbing the presently-unused unary @
RN> operator for this

I don't think this is the case because the parser has to interpret `@`
in formal argument lists in a different way than in function calls.
Besides, it'd make sense to set up these annotations with a binary
`@`. There are already two main ways of passing arguments in R: by
value and by expression. Providing an explicit annotation for passing
by expression would standardise the semantics of these functions and,
as Michael suggests, would help static analysis. So passing by name
could be just another argument-passing method:

    function(expr@ x, value@ y = 10L, name@ z = rnorm(1)) {
      list(x, y, z, z)
    }

The parser would record the argument metadata in the formals list.
This metadata could be consulted by static analysis tools and a
selected subset of those tags (`expr` and `name`) would have an effect
on the evaluation mechanism.


RN> One thing I'm not clear on is whether the proposal would add anything
RN> semantically beyond what the present "eval" and
"substitute" functions
RN> can do fairly easily.

Quasiquotation makes it possible to program with functions that take
arguments by expression. There is no easy way to do that with
eval() and substitute() alone. R has always been an interface language
and as such, its main advantage is to provide DSLs for data analysis
tasks. Specification of statistical models with a formula, overscoping
data frame columns with subset() and transform(), etc. This is why
providing an easier means of programming with these functions seems to
have more value than call-by-name semantics. Unquoting will be used
extensively in ggplot2 and dplyr, two popular R packages.  Please see
the vignette posted by Hadley for some introductory examples.

In any case, the unquoting notation would be orthogonal to function
arguments annotations because actuals and formals are parsed
differently. While formals annotations would be recorded by the parser
in the formals list, `@` in an actual argument list would be parsed as
a function call, like the rest of R operators.

Lionel