R devel - Oct 2019 - [External] Re: should base R have a piping operator ?

> On 7 Oct 2019, at 17:04, Tierney, Luke <luke-tierney at uiowa.edu>
wrote:
> 
>  Think about what happens if an
> argument in a pipe stage contains a pipe. (Not completely
> unreasonable, e.g. for a left_join).
It should work exactly as it does in a local environment.

```
`%foo%` <- function(x, y) {
  env <- parent.frame()

  # Use `:=` to avoid partial matching on .env/.frame
  rlang::scoped_bindings(. := x, .env = env)

  eval(substitute(y), env)
}

"A" %foo% {
  print(.)
  "B" %foo% print(.)
  print(.)
}
#> [1] "A"
#> [1] "B"
#> [1] "A"

print(.)
#> Error in print(.) : object '.' not found

```

The advantage is that side effects (such as assigning variables or calling
`return()`) will occur in the expected environment. I don't see it causing
problems except in artificial cases. Am I missing something?

I agree that restraining the pipe to a single placeholder (to avoid
double evaluation) would be a good design too.

I can't access https://gitlab.com/luke-tierney/pipes, it appears to be 404.

Best,
Lionel

On Mon, 7 Oct 2019, Lionel Henry wrote:
>
>
>> On 7 Oct 2019, at 17:04, Tierney, Luke <luke-tierney at
uiowa.edu> wrote:
>>
>>  Think about what happens if an
>> argument in a pipe stage contains a pipe. (Not completely
>> unreasonable, e.g. for a left_join).
>
> It should work exactly as it does in a local environment.
>
> ```
> `%foo%` <- function(x, y) {
>  env <- parent.frame()
>
>  # Use `:=` to avoid partial matching on .env/.frame
>  rlang::scoped_bindings(. := x, .env = env)
>
>  eval(substitute(y), env)
> }
>
> "A" %foo% {
>  print(.)
>  "B" %foo% print(.)
>  print(.)
> }
> #> [1] "A"
> #> [1] "B"
> #> [1] "A"
>
> print(.)
> #> Error in print(.) : object '.' not found
>
> ```
>
> The advantage is that side effects (such as assigning variables or calling
> `return()`) will occur in the expected environment.
You get the assignment behavior with the nested call approach. (Not
that doing this is necessarily a good idea).
> I don't see it causing
> problems except in artificial cases. Am I missing something?
Here is a stylized example:

f <- function(x, y) {
     assign("xx", x, parent.frame())
     on.exit(rm(xx, envir = parent.frame()))
     y
     get("xx") + 1
}

## This is fine:
> f(1, 2) 
[1] 2

## This is not:
> f(1, f(1, 2))
Error in get("xx") : object 'xx' not found

If you play these games whether you get the result you want, or an
obvious error, or just the wrong answer depends on argument evaluation
order and the like. You really don't want to go there. Not to mention
that you would be telling users they are not allowed to use '.' as a
variable name for their own purposes or you would be polluting their
environment with some other artificial symbol that they would see in
debugging. Just don't.

Anything going in base needs to worry even about artificial cases.
Yes, there are things in base that don't meet that standard. No, that
is not a reason to add more.
> I agree that restraining the pipe to a single placeholder (to avoid
> double evaluation) would be a good design too.
>
> I can't access https://gitlab.com/luke-tierney/pipes, it appears to be
404.
Should be able to get there now. Needed to change the visibility ---
still learning my way around gitlab.

Best,

luke
> Best,
> Lionel
>
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu

IMHO, if base R were to include a pipe operator, I think it should be much
simpler than the magrittr pipe. It should satisfy the property that:

    x |> f(...)   is equivalent to   f(x, ...)

Except, perhaps, in terms of when the promise for 'x' gets forced. We
shouldn't need to mess with bindings in environments to make that work.

My understanding is that the '.' placeholder is used so that the
magrittr
pipe can be adapted to functions that aren't endomorphic or otherwise
easily pipeable. I would argue that:

1. Users could just create their own pipable wrapper functions if so
required, or
2. Users could use magrittr to get some of the 'extensions' to the pipe
operator (with the noted caveats).

Best,
Kevin

	[[alternative HTML version deleted]]

On 7 Oct 2019, at 18:17, Tierney, Luke <luke-tierney at uiowa.edu> wrote:
> Here is a stylized example:
The previous value of the binding should only be restored if it
existed:

g <- function(x, y) {
  rlang::scoped_bindings(xx = x, .env = parent.frame())
  y
  get("xx") + 10
}

# Good
g(1, 2)
#> [1] 11

# Still good?
g(1, g(1, 2))
#> [1] 11

> If you play these games whether you get the result you want, or an
> obvious error, or just the wrong answer depends on argument evaluation
> order and the like.
I think the surprises are limited because the pattern has stack-like
semantics. We get in a new context where `.` gains a new meaning, and
when we exit the previous meaning is restored.

One example where this could lead to unexpected behaviour is trying to
capture the value of the placeholder in a closure:

f <- function(x) {
  x %>% {
    identity(function() .)
  }
}

# This makes sense:
f("A")()
#> Error: object '.' not found

# This doesn't:
"B" %>% { f("A")() }
#> [1] "B"

> Not to mention that you would be telling users they are not allowed
> to use '.' as a variable name for their own purposes or you would
be
> polluting their environment with some other artificial symbol that
> they would see in debugging.
That's a good point. Debugging allows to move up the call stack before
the context is exited, so you'd see the last value of `.` in examples
of nested pipes like `foo %>% bar( f %>% g() )`. That could be confusing.

> Anything going in base needs to worry even about artificial cases.
> Yes, there are things in base that don't meet that standard. No, that
> is not a reason to add more.
Agreed. What I meant by artificial cases is functions making
questionable assumptions after peeking into foreign contexts etc.

I'm worried about what happens with important language constructs like
`<-` and `return()` when code is evaluated in a local context. That
said, I think binding pipe values to `.` is more important than these
particular semantics because the placeholder is an obvious binding to
inspect while debug-stepping through a pipeline. So evaluating in a
child is probably preferable to giving up the placeholder altogether.

Best,
Lionel

Just for the record, and not that using return() calls like this is
necessarily a good idea, it is possible to make a nested-call-based
pipe that handles return() calls the way you want using delayedAssign.
I've added it to the end of the file on gitlab.

Time to move on to the stuff I've been avoiding ...

Best,

luke

On Mon, 7 Oct 2019, Tierney, Luke wrote:
> On Mon, 7 Oct 2019, Lionel Henry wrote:
>
>>
>>
>>> On 7 Oct 2019, at 17:04, Tierney, Luke <luke-tierney at
uiowa.edu> wrote:
>>>
>>>  Think about what happens if an
>>> argument in a pipe stage contains a pipe. (Not completely
>>> unreasonable, e.g. for a left_join).
>>
>> It should work exactly as it does in a local environment.
>>
>> ```
>> `%foo%` <- function(x, y) {
>>  env <- parent.frame()
>>
>>  # Use `:=` to avoid partial matching on .env/.frame
>>  rlang::scoped_bindings(. := x, .env = env)
>>
>>  eval(substitute(y), env)
>> }
>>
>> "A" %foo% {
>>  print(.)
>>  "B" %foo% print(.)
>>  print(.)
>> }
>> #> [1] "A"
>> #> [1] "B"
>> #> [1] "A"
>>
>> print(.)
>> #> Error in print(.) : object '.' not found
>>
>> ```
>>
>> The advantage is that side effects (such as assigning variables or
calling
>> `return()`) will occur in the expected environment.
>
> You get the assignment behavior with the nested call approach. (Not
> that doing this is necessarily a good idea).
>
>> I don't see it causing
>> problems except in artificial cases. Am I missing something?
>
> Here is a stylized example:
>
> f <- function(x, y) {
>     assign("xx", x, parent.frame())
>     on.exit(rm(xx, envir = parent.frame()))
>     y
>     get("xx") + 1
> }
>
> ## This is fine:
>> f(1, 2)
> [1] 2
>
> ## This is not:
>> f(1, f(1, 2))
> Error in get("xx") : object 'xx' not found
>
> If you play these games whether you get the result you want, or an
> obvious error, or just the wrong answer depends on argument evaluation
> order and the like. You really don't want to go there. Not to mention
> that you would be telling users they are not allowed to use '.' as
a
> variable name for their own purposes or you would be polluting their
> environment with some other artificial symbol that they would see in
> debugging. Just don't.
>
> Anything going in base needs to worry even about artificial cases.
> Yes, there are things in base that don't meet that standard. No, that
> is not a reason to add more.
>
>> I agree that restraining the pipe to a single placeholder (to avoid
>> double evaluation) would be a good design too.
>>
>> I can't access https://gitlab.com/luke-tierney/pipes, it appears to
be 404.
>
> Should be able to get there now. Needed to change the visibility ---
> still learning my way around gitlab.
>
> Best,
>
> luke
>
>> Best,
>> Lionel
>>
>>
>
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu