R devel - Sep 2008 - typo in cov()? var() fails on NA in R 2.7.2 but not R 2.6.1

I recently started using R 2.7.2, and noticed a surprising change in
the behavior of var() on NA data:

R 2.6.1 (Patched), 2007-11-26, svn.rev 43541, x86_64-unknown-linux-gnu:

  > stdev(rep(NA,3), na.rm=F)
  [1] NA
  > stdev(rep(NA,3), na.rm=T)
  [1] NA
  > var(rep(NA,3), na.rm=T, use="complete.obs")
  [1] NA

R 2.7.2 (Patched), 2008-09-02, svn.rev 46491, x86_64-unknown-linux-gnu:

  > stdev(rep(NA,3), na.rm=F)
  [1] NA

  > stdev(rep(NA,3), na.rm=T)
  Error in var(x, na.rm = na.rm) : no complete element pairs

  Enter a frame number, or 0 to exit
  1: stdev(rep(NA, 3), na.rm = T)
  2: var(x, na.rm = na.rm)
  Selection: 0

  > var(rep(NA,3), na.rm=T, use="complete.obs")
  Error in var(rep(NA, 3), na.rm = T, use = "complete.obs") :
    no complete element pairs

Is this change intentional?  Also, what is causing it?  Looking, I see
no changes in var() at all, so the new behavior must be due to a
change in what this call does:

  .Internal(cov(x, y, na.method, FALSE))

The R 2.7.2 cov() also has this weird line:

  else if (na.method != 3L) {
 
Note the "L" in the "3L".  A typo?  The older 2.6.1 cov()
just has "3"
on that line, no "L".

Interactively redefining cov() to remove the "L" makes no difference
in my var() calls, but that could b

The original source file seems to be:

  src/library/stats/R/cor.R 

svn annotate says that 3L line was last changed nearly a year ago, way
back in rev 43302:

  ------------------------------------------------------------------------
  r43302 | ripley | 2007-10-29 14:50:18 -0400 (Mon, 29 Oct 2007) | 2 lines
  make cor/cov a little less inconsistent

The strange 3L line occurs twice in that file, in both cor() and cov():

  $ grep -n 3L cor.R  
  36:    else if (na.method != 3L) { 
  118:    else if (na.method != 3L) { 

That line might not be the cause of my "no complete element pairs"
problem (I'm not at all sure), but it does look suspicious.

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/

This has been changed in 2.7.1

BUG FIXES

    o	co[rv](use = "complete.obs") now always gives an error if there
	are no complete cases: they used to give NA if
	method = "pearson" but an error for the other two methods.
	(Note that this is pretty arbitrary, but zero-length vectors
	always give an error so it is at least consistent.)

	Since sd(na.rm=TRUE) and var(na.rm=TRUE) both call cov(use 
"complete.obs"), this applies also to them.

	cor(use="pair") used to give diagonal 1 even if the variable
	was completely missing for the rank methods but NA for the
	Pearson method: it now gives NA in all cases.

	cor(use="pair") for the rank methods gave a matrix result with
	dimensions > 0 even if one of the inputs had 0 columns.

Regards,

Simone




On Sat, Sep 6, 2008 at 12:38 AM, Andrew Piskorski <atp@piskorski.com>
wrote:
> I recently started using R 2.7.2, and noticed a surprising change in
> the behavior of var() on NA data:
>
> R 2.6.1 (Patched), 2007-11-26, svn.rev 43541, x86_64-unknown-linux-gnu:
>
>  > stdev(rep(NA,3), na.rm=F)
>  [1] NA
>  > stdev(rep(NA,3), na.rm=T)
>  [1] NA
>  > var(rep(NA,3), na.rm=T, use="complete.obs")
>  [1] NA
>
> R 2.7.2 (Patched), 2008-09-02, svn.rev 46491, x86_64-unknown-linux-gnu:
>
>  > stdev(rep(NA,3), na.rm=F)
>  [1] NA
>
>  > stdev(rep(NA,3), na.rm=T)
>  Error in var(x, na.rm = na.rm) : no complete element pairs
>
>  Enter a frame number, or 0 to exit
>  1: stdev(rep(NA, 3), na.rm = T)
>  2: var(x, na.rm = na.rm)
>  Selection: 0
>
>  > var(rep(NA,3), na.rm=T, use="complete.obs")
>  Error in var(rep(NA, 3), na.rm = T, use = "complete.obs") :
>    no complete element pairs
>
> Is this change intentional?  Also, what is causing it?  Looking, I see
> no changes in var() at all, so the new behavior must be due to a
> change in what this call does:
>
>  .Internal(cov(x, y, na.method, FALSE))
>
> The R 2.7.2 cov() also has this weird line:
>
>  else if (na.method != 3L) {
>
> Note the "L" in the "3L".  A typo?  The older 2.6.1
cov() just has "3"
> on that line, no "L".
>
> Interactively redefining cov() to remove the "L" makes no
difference in my
> var() calls, but that could b
>
> The original source file seems to be:
>
>  src/library/stats/R/cor.R
>
> svn annotate says that 3L line was last changed nearly a year ago, way
> back in rev 43302:
>
>  ------------------------------------------------------------------------
>  r43302 | ripley | 2007-10-29 14:50:18 -0400 (Mon, 29 Oct 2007) | 2 lines
>  make cor/cov a little less inconsistent
>
> The strange 3L line occurs twice in that file, in both cor() and cov():
>
>  $ grep -n 3L cor.R
>  36:    else if (na.method != 3L) {
>  118:    else if (na.method != 3L) {
>
> That line might not be the cause of my "no complete element
pairs"
> problem (I'm not at all sure), but it does look suspicious.
>
> --
> Andrew Piskorski <atp@piskorski.com>
> http://www.piskorski.com/
>
> ______________________________________________
> R-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>


-- 
______________________________________________________

Simone Giannerini
Dipartimento di Scienze Statistiche "Paolo Fortunati"
Universita' di Bologna
Via delle belle arti 41 - 40126 Bologna, ITALY
Tel: +39 051 2098262 Fax: +39 051 232153
http://www2.stat.unibo.it/giannerini/
______________________________________________________



-- 
______________________________________________________

Simone Giannerini
Dipartimento di Scienze Statistiche "Paolo Fortunati"
Universita' di Bologna
Via delle belle arti 41 - 40126 Bologna, ITALY
Tel: +39 051 2098262 Fax: +39 051 232153
http://www2.stat.unibo.it/giannerini/
______________________________________________________

	[[alternative HTML version deleted]]

Andrew Piskorski wrote:
> I recently started using R 2.7.2, and noticed a surprising change in
> the behavior of var() on NA data:
> 
> R 2.6.1 (Patched), 2007-11-26, svn.rev 43541, x86_64-unknown-linux-gnu:
> 
>   > stdev(rep(NA,3), na.rm=F)
>   [1] NA
>   > stdev(rep(NA,3), na.rm=T)
>   [1] NA
>   > var(rep(NA,3), na.rm=T, use="complete.obs")
>   [1] NA
> 
> R 2.7.2 (Patched), 2008-09-02, svn.rev 46491, x86_64-unknown-linux-gnu:
> 
>   > stdev(rep(NA,3), na.rm=F)
>   [1] NA
> 
>   > stdev(rep(NA,3), na.rm=T)
>   Error in var(x, na.rm = na.rm) : no complete element pairs
> 
>   Enter a frame number, or 0 to exit
>   1: stdev(rep(NA, 3), na.rm = T)
>   2: var(x, na.rm = na.rm)
>   Selection: 0
> 
>   > var(rep(NA,3), na.rm=T, use="complete.obs")
>   Error in var(rep(NA, 3), na.rm = T, use = "complete.obs") :
>     no complete element pairs
> 
> Is this change intentional?  Also, what is causing it?  Looking, I see
> no changes in var() at all, so the new behavior must be due to a
> change in what this call does:
> 
>   .Internal(cov(x, y, na.method, FALSE))
> 
> The R 2.7.2 cov() also has this weird line:
> 
>   else if (na.method != 3L) {
>  
> Note the "L" in the "3L".  A typo?  The older 2.6.1
cov() just has "3"
> on that line, no "L".
Not a typo! See R-2.5.0's NEWS:

NEW FEATURES

     o	Introduced the suffix L for integer literals to create
	integer rather than numeric values, e.g. 100L, 0x10L, 1e2L.

So,

 > class(3)
[1] "numeric"
 > class(3L)
[1] "integer"


HTH,
Henric


> 
> Interactively redefining cov() to remove the "L" makes no
difference in my var() calls, but that could b
> 
> The original source file seems to be:
> 
>   src/library/stats/R/cor.R 
> 
> svn annotate says that 3L line was last changed nearly a year ago, way
> back in rev 43302:
> 
>   ------------------------------------------------------------------------
>   r43302 | ripley | 2007-10-29 14:50:18 -0400 (Mon, 29 Oct 2007) | 2 lines
>   make cor/cov a little less inconsistent
> 
> The strange 3L line occurs twice in that file, in both cor() and cov():
> 
>   $ grep -n 3L cor.R  
>   36:    else if (na.method != 3L) { 
>   118:    else if (na.method != 3L) { 
> 
> That line might not be the cause of my "no complete element
pairs"
> problem (I'm not at all sure), but it does look suspicious.
>