R devel - Nov 2018 - Unexpected argument-matching when some are missing

On Thu, Nov 29, 2018 at 10:51 AM S Ellison <S.Ellison at lgcgroup.com>
wrote:
>
> > When trying out some variations with `[.data.frame` I noticed some (to
me)
> > odd behaviour,
>
> Not just in 'myfun' ...
>
> plot(x=1:10, y=)
> plot(x=1:10, y=, 10:1)
>
> In both cases, 'y=' is ignored. In the first, the plot is for
y=NULL (so not 'missing' y)
> In the second case, 10:1 is positionally matched to y despite the
intervening 'missing' 'y='
>
> So it isn't just 'missing'; it's 'not there at all'
What exactly is the difference between "missing" and "not there
at all"?

--Ista
>
> Steve E
>
> > -----Original Message-----
> > From: R-devel [mailto:r-devel-bounces at r-project.org] On Behalf Of
Emil
> > Bode
> > Sent: 29 November 2018 10:09
> > To: r-devel at r-project.org
> > Subject: [Rd] Unexpected argument-matching when some are missing
> >
> > When trying out some variations with `[.data.frame` I noticed some (to
me)
> > odd behaviour, which I found out has nothing to do with
`[.data.frame`, but
> > rather with the way arguments are matched, when mixing named/unnamed
> > and missing/non-missing arguments. Consider the following example:
> >
> >
> >
> > myfun <- function(x,y,z) {
> >
> >   print(match.call())
> >
> >   cat('x=',if(missing(x)) 'missing' else x,
'\n')
> >
> >   cat('y=',if(missing(y)) 'missing' else y,
'\n')
> >
> >   cat('z=',if(missing(z)) 'missing' else z,
'\n')
> >
> > }
> >
> > myfun(x=, y=, "z's value")
> >
> >
> >
> > gives:
> >
> >
> >
> > # myfun(x = "z's value")
> >
> > # x= z's value
> >
> > # y= missing
> >
> > # z= missing
> >
> >
> >
> > This seems very counterintuitive to me, I expect the arguments x and y
to be
> > missing, and z to get ?z?s value?.
> >
> > When I call myfun(,y=,"z's value"), x is missing, and y
gets ?z?s value?.
> >
> > Are my expectations wrong or is this a bug? And if my expectations are
> > wrong, where can I find more information on argument-matching?
> >
> > My gut-feeling says to call this a bug, but then I?m surprised no-one
else has
> > encountered it before.
> >
> >
> >
> > And I don?t have multiple installations to work from, so could
somebody else
> > confirm this (if it?s not my expectations that are wrong) for
R-devel/other R-
> > versions/other platforms?
> >
> > My setup: R 3.5.1, MacOS 10.13.6, both Rstudio 1.1.453 and R --vanilla
from
> > Bash
> >
> >
> >
> > Best regards,
> >
> > Emil Bode
>
>
>
> *******************************************************************
> This email and any attachments are confidential. Any u...{{dropped:11}}

> > plot(x=1:10, y=)
> > plot(x=1:10, y=, 10:1)
> >
> > In both cases, 'y=' is ignored. In the first, the plot is for
y=NULL (so not
> 'missing' y)
> > In the second case, 10:1 is positionally matched to y despite the
intervening
> 'missing' 'y='
> >
> > So it isn't just 'missing'; it's 'not there at
all'
> 
> What exactly is the difference between "missing" and "not
there at all"?
A "missing argument" in R means that an argument with no default value
was omitted from the call, and that is what I meant by "missing".
But that is not what is happening here. I was talking about "y="
apparently being treated as not present in the call, rather than the argument y
being treated as a missing argument.

In these examples, plot.default has a default value for y (NULL) so y can never
be "missing" in the sense of the 'missing argument' error
(compare what happens with plot(y=1:10), which reports x as 'missing').
In the first example, y was (from the plot behaviour) taken as NULL - the
default - so was not considered a missing argument. In the second, it was taken
as 10:1 - again, non-missing, despite 10:1 being in the normal position for the
(character) argument "type".
But neither call did anything at all with "y=". Instead, the behaviour
is consistent with what would have happened if 'y=' were "not
present at all" when counting position or named argument list, rather than
if 'y' were an absent required argument.
It _looks_ as if the initial call parsing silently ignored the malformed
expression "y=" before any argument matching - positional or by name -
takes place.

But I'm thinking that it'll take an R-core guru to explain what's
going on here, so I was going to wait and see.

Steve Ellison



*******************************************************************
This email and any attachments are confidential. Any use, copying or
disclosure other than by the intended recipient is unauthorised. If 
you have received this message in error, please notify the sender 
immediately via +44(0)20 8943 7000 or notify postmaster at lgcgroup.com 
and delete this message and any copies from your computer and network. 
LGC Limited. Registered in England 2991879. 
Registered office: Queens Road, Teddington, Middlesex, TW11 0LY, UK

On Thu, Nov 29, 2018 at 1:10 PM S Ellison <S.Ellison at lgcgroup.com>
wrote:
>
>
> > > plot(x=1:10, y=)
> > > plot(x=1:10, y=, 10:1)
> > >
> > > In both cases, 'y=' is ignored. In the first, the plot is
for y=NULL (so not
> > 'missing' y)
> > > In the second case, 10:1 is positionally matched to y despite the
intervening
> > 'missing' 'y='
> > >
> > > So it isn't just 'missing'; it's 'not there
at all'
> >
> > What exactly is the difference between "missing" and
"not there at all"?
>
> A "missing argument" in R means that an argument with no default
value was omitted from the call, and that is what I meant by
"missing".
> But that is not what is happening here. I was talking about "y="
apparently being treated as not present in the call, rather than the argument y
being treated as a missing argument.
>
> In these examples, plot.default has a default value for y (NULL) so y can
never be "missing" in the sense of the 'missing argument'
error (compare what happens with plot(y=1:10), which reports x as
'missing').
> In the first example, y was (from the plot behaviour) taken as NULL - the
default - so was not considered a missing argument. In the second, it was taken
as 10:1 - again, non-missing, despite 10:1 being in the normal position for the
(character) argument "type".
> But neither call did anything at all with "y=". Instead, the
behaviour is consistent with what would have happened if 'y=' were
"not present at all" when counting position or named argument list,
rather than if 'y' were an absent required argument.
> It _looks_ as if the initial call parsing silently ignored the malformed
expression "y=" before any argument matching - positional or by name -
takes place.
Yes, I think all of that is correct. But y _is_ missing in this sense:
> debug(plot)
> plot(1:10, y=)
debugging in: plot(1:10, y = )
debug: UseMethod("plot")
Browse[2]> missing(y)
[1] TRUE

though this does not explain the behavior since
> plot( , , "l")
debugging in: plot(, , "l")
debug: UseMethod("plot")
Browse[2]> missing(y)
[1] TRUE

--Ista
>
> But I'm thinking that it'll take an R-core guru to explain
what's going on here, so I was going to wait and see.
>
> Steve Ellison
>
>
>
> *******************************************************************
> This email and any attachments are confidential. Any u...{{dropped:8}}

It looks like you're right that somewhere in (presumably) match.call, the
named, empty arguments are removed, such that the call plot(x=1:10, y=, 10:1) is
translated to plot(x=1:10, 10:1).
But I would have expected it to be the same as plot(x=1:10, , 10:1) (note the
", ,"), which gives an error (10:1 is not a valid plot-type). In this
case you get an error straightaway, I find this more
interesting:
> options(warn=-1)
> plot(x=1, y=, 'p', ylim=c(0,10))
> plot(x=1, , 'p', ylim=c(0,10))
Both valid (no errors), albeit strange calls, but I'd say the first call is
better code, it's clearer you intend to not give any value for y. But
exactly this one gives unexpected results: it tries to plot at position (1,
'p'), or (1, NA).

And the behaviour as it is gives rise to some strange inconsistencies. I have
gathered some examples below (at the very bottom of the thread, as it got quite
extensive), where some variations are surprisingly different from each other.
There are also some issues when using data.frame(...)[i=, j=,...], but at least
here you are warned about naming i and j.
But basically, it means any function where arguments like fun(,,) are a valid
possibility should throw the same warning, e.g. any R-code replacement of
[.matrix or [.array, or as in my examples, for data.table (and related
structures)

?On 29/11/2018, 19:10, "S Ellison" <S.Ellison at LGCGroup.com>
wrote:

    
    > > plot(x=1:10, y=)
    > > plot(x=1:10, y=, 10:1)
    > >
    > > In both cases, 'y=' is ignored. In the first, the plot is
for y=NULL (so not
    > 'missing' y)
    > > In the second case, 10:1 is positionally matched to y despite the
intervening
    > 'missing' 'y='
    > >
    > > So it isn't just 'missing'; it's 'not there at
all'
    > 
    > What exactly is the difference between "missing" and
"not there at all"?
    
    A "missing argument" in R means that an argument with no default
value was omitted from the call, and that is what I meant by
"missing".
    But that is not what is happening here. I was talking about "y="
apparently being treated as not present in the call, rather than the argument y
being treated as a missing argument.
    
    In these examples, plot.default has a default value for y (NULL) so y can
never be "missing" in the sense of the 'missing argument'
error (compare what happens with plot(y=1:10), which reports x as
'missing').
    In the first example, y was (from the plot behaviour) taken as NULL - the
default - so was not considered a missing argument. In the second, it was taken
as 10:1 - again, non-missing, despite 10:1 being in the normal position for the
(character) argument "type".
    But neither call did anything at all with "y=". Instead, the
behaviour is consistent with what would have happened if 'y=' were
"not present at all" when counting position or named argument list,
rather than if 'y' were an absent required argument.
    It _looks_ as if the initial call parsing silently ignored the malformed
expression "y=" before any argument matching - positional or by name -
takes place.
    
    But I'm thinking that it'll take an R-core guru to explain
what's going on here, so I was going to wait and see.
    
    Steve Ellison
    

Exampled if what I (Emil) found odd:
---------------------------------------------------------------------------------------------------------------------------------------
> library(data.table)
> options(warn=1) # Or 2
> data.table(a=1:2, b=3:4)[1] # As expected
   a b
1: 1 3
> data.table(a=1:2, b=3:4)[, 1] # As expected
   a
1: 1
2: 2
> data.table(a=1:2, b=3:4)[i=, 1] # Huh? We get the first row
   a b
1: 1 3
> data.table(a=1:2, b=3:4)[, 1, 'a'] # As expected
   a V1
1: 1  1
2: 2  1
> data.table(a=1:2, b=3:4)[i=, 1, 'a'] # I would have expected the
same result, and definitely more than 1 value
   a
1: 1
> data.table(a=1:2, b=3:4)[i=, 1, by='a'] # And this doesn't
work?
Error in `[.data.table`(data.table(a = 1:2, b = 3:4), i = , 1, by =
"a") :
  'by' or 'keyby' is supplied but not j
> myfun <- function(x,y,z) {
+   print(match.call())
+   cat('nargs: ', nargs(), '\n')
+   cat('x=',if(missing(x)) 'missing' else x, '\n')
+   cat('y=',if(missing(y)) 'missing' else y, '\n')
+   cat('z=',if(missing(z)) 'missing' else z, '\n')
+ }
> myfun(x=, y=, , , "z's value") # 5 arguments??
myfun(z = "z's value")
nargs:  5 
x= missing 
y= missing 
z= z's value 
> myfun(x=, y=, , , "z's value", , ) # But any more are not
allowed
Error in myfun(x = , y = , , , "z's value", , ) : 
  unused arguments (alist(, ))
> myfun(x2=, y=, "z's value") # And named arguments are
ignored, but the names have to be to existing argument-names
Error in myfun(x2 = , y = , "z's value") : unused argument
(alist(x2 = ))
> myfun(x=, x=, , "z's value") # And naming it multiple times
also gives an error
Error in myfun(x = , x = , , "z's value") : 
  formal argument "x" matched by multiple actual
arguments
> myfun(y=, , "z's value", x=3) # Having fun with obfuscation,
is this call backwards and forwards compatible?
myfun(x = 3, z = "z's value")
nargs:  4 
x= 3 
y= missing 
z= z's value 
> myfun(y=rlang::missing_arg(), , "z's value", x=3)
Error in myfun(y = rlang::missing_arg(), , "z's value", x = 3) : 
  unused argument ("z's value")
> myfun(y=rlang::missing_arg(), z=, "z's value", x=3) # Now
with a named empty argument
myfun(x = 3, y = rlang::missing_arg(), z = "z's value")
nargs:  4 
x= 3 
y=  
z= z's value 
> myfun(y=rlang::missing_arg(), z= "z's value", x=3) # Just a
comma removed: same match.call(), different nargs()
myfun(x = 3, y = rlang::missing_arg(), z = "z's value")
nargs:  3 
x= 3 
y=  
z= z's value