R SIG Debian - Sep 2011 - make check reg-tests-1b.R fails with Ubuntu R

Today I built R from source on a 32-bit Ubuntu 10.04.3 LTS box, and
saw that the "make check" tests/reg-tests-1b.R failed.  From the
output at the end of my "tests/reg-tests-1b.Rout.fail" file, the
problem is appearing in the "identical(z, x %*% t(y))" test code
below.

I then tried the stock R provided by the Ubuntu r-base-core binary
package, and to my surprise, it failed the test in exactly the same
way!

So this is NOT solely a problem with my from-source build (I merely
happened to notice it first that way).  It probably indicates some
sort of system-wide bug or misconfiguration, at least on my machine,
perhaps on others' too.

Could someone please advise me:  What is the significance of this test
from reg-tests-1b.R failing, and what should I do to further track it
down and fix it?

Thanks!  Much more detail is below.

------------------------------------------------------------

# Here's the excerpt from the "make check" output where I first
# noticed the problem:

running regression tests ...
make[3]: Entering directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
running code in '../../tests/reg-tests-1.R' ... OK
running code in '../../tests/reg-tests-1b.R' ...make[3]: ***
[reg-tests-1b.Rout] Error 1
make[3]: Leaving directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
make[2]: *** [test-Reg] Error 2
make[2]: Leaving directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
make[1]: *** [test-all-basics] Error 1
make[1]: Leaving directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
make: *** [check] Error 2


# This is the Ubuntu machine where I found the error:

andy at n88:~$ lsb_release -d
Description:	Ubuntu 10.04.3 LTS
andy at n88:~$ uname -a
Linux n88 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:08:37 UTC 2011 i686
GNU/Linux
andy at n88:~$ dpkg -l r-base-core | grep r-base
ii  r-base-core   2.10.1-2   GNU R core of statistical computation and graphics
system


# Stock Ubuntu-provided R has the same failure:

andy at n88:~$ /usr/bin/R -q --vanilla
> dtk.R.version()
[1] "R 2.10.1, 2009-12-14, svn.rev 50720,
i486-pc-linux-gnu"
> x <- matrix(c(1, 0, NA, 1), 2, 2) ; y <- matrix(c(1, 0, 0, 2, 1, 0),
3, 2)
> (z <- tcrossprod(x, y))
     [,1] [,2] [,3]
[1,]   NA   NA    0
[2,]    2    1    0
> identical(z, x %*% t(y))
[1] FALSE


# R I just built from scratch, same failure as stock R:

andy at n88:~$ /usr/local/pkg/R-2.12-branch-20110914/bin/R -q
--vanilla
> dtk.R.version()
[1] "R 2.12.2 (Patched), 2011-03-18, svn.rev 57004,
i686-pc-linux-gnu"
> x <- matrix(c(1, 0, NA, 1), 2, 2) ; y <- matrix(c(1, 0, 0, 2, 1, 0),
3, 2)
> (z <- tcrossprod(x, y))
     [,1] [,2] [,3]
[1,]   NA   NA    0
[2,]    2    1    0
> identical(z, x %*% t(y))
[1] FALSE


# Apparently the correct output is supposed to look like this:

andy at dax:~$ /usr/bin/R -q --vanilla
> x <- matrix(c(1, 0, NA, 1), 2, 2) ; y <- matrix(c(1, 0, 0, 2, 1, 0),
3, 2)
> (z <- tcrossprod(x, y))
     [,1] [,2] [,3]
[1,]   NA   NA   NA
[2,]    2    1    0
> identical(z, x %*% t(y))
[1] TRUE

# Note that the correct output above was run on a different machine,
# which happens to be x86-64:

andy at dax:~$ lsb_release -d
Description:	Ubuntu 10.04.3 LTS
andy at dax:~$ uname -a
Linux dax 2.6.32.29+drm33.13-custom #1 SMP Fri Apr 8 13:42:18 EDT 2011 x86_64
GNU/Linux


# Function I used to print out the version number info above:
dtk.R.version <- function() {
   patched.str <- version$status
   if (patched.str != "")
      patched.str <- paste(" (" ,patched.str ,")"
,sep="")
   paste(paste(version$language  ," "
               ,paste(version[c("major","minor")]
,collapse=".")
               ,patched.str  ,sep="")
         ,paste(version[c("year","month","day")]
,collapse="-")
         ,paste("svn.rev" ,version$"svn rev")
         ,version$platform
         ,sep=", " ,collapse=" ")
}

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/

Andrew,

Crossprod etc all dispatch to BLAS / LAPACK.  The error appears to be that
z[1,3] is 0 whereas a NA gets expected.

I just verified this on another 32bit box where I had stock R and little
else:
> x <- matrix(c(1, 0, NA, 1), 2, 2) ; y <- matrix(c(1, 0, 0, 2, 1, 0),
3, 2)
> (z <- tcrossprod(x, y))
     [,1] [,2] [,3]
[1,]   NA   NA    0
[2,]    2    1    0
>  identical(z, x %*% t(y))
[1] FALSE
>
but all it takes is 

wajig install libatlas3gf-base

after which we get 

dd at chibud:~$ R

R version 2.13.1 (2011-07-08)
Copyright (C) 2011 The R Foundation for Statistical Computing
ISBN 3-900051-07-0
Platform: i686-pc-linux-gnu (32-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
>  x <- matrix(c(1, 0, NA, 1), 2, 2) ; y <- matrix(c(1, 0, 0, 2, 1, 0),
3, 2)
> (z <- tcrossprod(x, y))
     [,1] [,2] [,3]
[1,]   NA   NA   NA
[2,]    2    1    0
>  identical(z, x %*% t(y))
[1] TRUE
> 

So try installing 'a better BLAS'.  Maybe we can distill this into a
pure C
program exhibiting the same issue and bring it to the BLAS maintainer?

Dirk


-- 
New Rcpp master class for R and C++ integration is scheduled for 
San Francisco (Oct 8), more details / reg.info available at
http://www.revolutionanalytics.com/products/training/public/rcpp-master-class.php

On 09/14/2011 10:18 AM, Andrew Piskorski wrote:
> Today I built R from source on a 32-bit Ubuntu 10.04.3 LTS box, and
> saw that the "make check" tests/reg-tests-1b.R failed.  From the
> output at the end of my "tests/reg-tests-1b.Rout.fail" file, the
> problem is appearing in the "identical(z, x %*% t(y))" test code
> below.
>
> I then tried the stock R provided by the Ubuntu r-base-core binary
> package, and to my surprise, it failed the test in exactly the same
> way!
>
> So this is NOT solely a problem with my from-source build (I merely
> happened to notice it first that way).  It probably indicates some
> sort of system-wide bug or misconfiguration, at least on my machine,
> perhaps on others' too.
>
> Could someone please advise me:  What is the significance of this test
> from reg-tests-1b.R failing, and what should I do to further track it
> down and fix it?
>
> Thanks!  Much more detail is below.
>
> ------------------------------------------------------------
>
> # Here's the excerpt from the "make check" output where I
first
> # noticed the problem:
>
> running regression tests ...
> make[3]: Entering directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
> running code in '../../tests/reg-tests-1.R' ... OK
> running code in '../../tests/reg-tests-1b.R' ...make[3]: ***
[reg-tests-1b.Rout] Error 1
> make[3]: Leaving directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
> make[2]: *** [test-Reg] Error 2
> make[2]: Leaving directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
> make[1]: *** [test-all-basics] Error 1
> make[1]: Leaving directory
`/net/brain/home/local-x86_32-ubuntu/nobackup/co-R/R-2-12-branch/Build-x86_32/tests'
> make: *** [check] Error 2
>
>
> # This is the Ubuntu machine where I found the error:
>
> andy at n88:~$ lsb_release -d
> Description:	Ubuntu 10.04.3 LTS
> andy at n88:~$ uname -a
> Linux n88 2.6.32-33-generic #72-Ubuntu SMP Fri Jul 29 21:08:37 UTC 2011
i686 GNU/Linux
> andy at n88:~$ dpkg -l r-base-core | grep r-base
> ii  r-base-core   2.10.1-2   GNU R core of statistical computation and
graphics system
>
>
> # Stock Ubuntu-provided R has the same failure:
>
> andy at n88:~$ /usr/bin/R -q --vanilla
>> dtk.R.version()
> [1] "R 2.10.1, 2009-12-14, svn.rev 50720, i486-pc-linux-gnu"
>> x<- matrix(c(1, 0, NA, 1), 2, 2) ; y<- matrix(c(1, 0, 0, 2, 1,
0), 3, 2)
>> (z<- tcrossprod(x, y))
>       [,1] [,2] [,3]
> [1,]   NA   NA    0
> [2,]    2    1    0
>> identical(z, x %*% t(y))
> [1] FALSE
>
>
> # R I just built from scratch, same failure as stock R:
>
> andy at n88:~$ /usr/local/pkg/R-2.12-branch-20110914/bin/R -q --vanilla
>> dtk.R.version()
> [1] "R 2.12.2 (Patched), 2011-03-18, svn.rev 57004,
i686-pc-linux-gnu"
>> x<- matrix(c(1, 0, NA, 1), 2, 2) ; y<- matrix(c(1, 0, 0, 2, 1,
0), 3, 2)
>> (z<- tcrossprod(x, y))
>       [,1] [,2] [,3]
> [1,]   NA   NA    0
> [2,]    2    1    0
>> identical(z, x %*% t(y))
> [1] FALSE
>
>
> # Apparently the correct output is supposed to look like this:
>
> andy at dax:~$ /usr/bin/R -q --vanilla
>> x<- matrix(c(1, 0, NA, 1), 2, 2) ; y<- matrix(c(1, 0, 0, 2, 1,
0), 3, 2)
>> (z<- tcrossprod(x, y))
>       [,1] [,2] [,3]
> [1,]   NA   NA   NA
> [2,]    2    1    0
>> identical(z, x %*% t(y))
> [1] TRUE
>
> # Note that the correct output above was run on a different machine,
> # which happens to be x86-64:
>
> andy at dax:~$ lsb_release -d
> Description:	Ubuntu 10.04.3 LTS
> andy at dax:~$ uname -a
> Linux dax 2.6.32.29+drm33.13-custom #1 SMP Fri Apr 8 13:42:18 EDT 2011
x86_64 GNU/Linux
>
>
> # Function I used to print out the version number info above:
> dtk.R.version<- function() {
>     patched.str<- version$status
>     if (patched.str != "")
>        patched.str<- paste(" (" ,patched.str ,")"
,sep="")
>     paste(paste(version$language  ," "
>                 ,paste(version[c("major","minor")]
,collapse=".")
>                 ,patched.str  ,sep="")
>          
,paste(version[c("year","month","day")]
,collapse="-")
>           ,paste("svn.rev" ,version$"svn rev")
>           ,version$platform
>           ,sep=", " ,collapse=" ")
> }
On my i386 machine:

~/D/R> lsb_release -d
Description:	Ubuntu 10.04.3 LTS

I get the following:

 > dtk.R.version()
[1] "R 2.13.1, 2011-07-08, svn.rev 56322, i486-pc-linux-gnu"
 > x <- matrix(c(1, 0, NA, 1), 2, 2)
 > y <- matrix(c(1, 0, 0, 2, 1, 0), 3, 2)
 > (z <- tcrossprod(x, y))
 > identical(z, x %*% t(y))
      [,1] [,2] [,3]
[1,]   NA   NA   NA
[2,]    2    1    0
[1] TRUE

This is using the R available on CRAN.  I can think of two possible causes:

1.  It is an error that was found prior to 2.13.1 and has been fixed.
2.  You have installed a non-standard, math-related library on your 
machine that is causing the error. A LAPACK library or the Revolution 
computing libraries available in Ubuntu are examples.

To test, I downgraded my R back to 2.10.1-2, the version from Ubuntu, 
not from CRAN.  Here are my results:

 > x <- matrix(c(1, 0, NA, 1), 2, 2)
 > y <- matrix(c(1, 0, 0, 2, 1, 0), 3, 2)
 > (z <- tcrossprod(x, y))
      [,1] [,2] [,3]
[1,]   NA   NA   NA
[2,]    2    1    0
 > identical(z, x %*% t(y))
[1] TRUE
 > dtk.R.version()
[1] "R 2.10.1, 2009-12-14, svn.rev 50720, i486-pc-linux-gnu"

This leads me to believe cause number 2 is the likely culprit.

Michael

-- 
Dr. Michael A. Rutter
School of Science
Penn State Erie, The Behrend College
Station Road
Erie, PA 16563
http://math.bd.psu.edu/faculty/rutter

Ah, with some different google searches, I found an older report of
this problem:

 
http://r.789695.n4.nabble.com/R-2-12-0-beta-r53110-fails-make-check-on-Ubuntu-hardy-td2954249.html
  http://comments.gmane.org/gmane.comp.lang.r.devel/25646

Further below is the configure output from my from-source build.  It
says "External libraries:  [...]  BLAS(generic)", so apparently the
Ubuntu-provided BLAS is bad?  Really?!

I also don't know how to get that sort of configure information from
an already installed R.  Can you advise please?

I suspect R is linked against libraries from Ubuntu's libblas3gf
1.2-2build1 package (which is the Fortran 77 reference implementation,
modulo any Debian tweaks), but I'm not sure, and I don't know how to
check.  (Perhaps I need some magic ar, nm, or objdump incantation?)

I think my from-source builds on older versions of Ubuntu have used
the built-in BLAS provided by R, but something has switched it on me.
I suspect the main solution for me is to figure out how to either
switch back to the built-in BLAS, or to a faster optimized BLAS like
Atlas.

Any advice on how I should best straighten this would be appreciated...
Ideally, I'd like to fix both the stock Ubuntu R and my own
from-source installs.  However of the two, making the from-source
install correct is much more important to me.

On Wed, Sep 14, 2011 at 10:18:55AM -0400, Andrew Piskorski wrote:
> # R I just built from scratch, same failure as stock R:
> andy at n88:~$ /usr/local/pkg/R-2.12-branch-20110914/bin/R -q --vanilla
> > dtk.R.version()
> [1] "R 2.12.2 (Patched), 2011-03-18, svn.rev 57004,
i686-pc-linux-gnu"
Here's the corresponding build configuration:
------------------------------------------------------------ 
R is now configured for i686-pc-linux-gnu  
 
  Source directory:          ..  
  Installation directory:    /usr/local/pkg/R-2.12-branch-20110914  
 
  C compiler:                gcc -std=gnu99  -g -O2  
  Fortran 77 compiler:       gfortran  -g -O2  
 
  C++ compiler:              g++  -g -O2  
  Fortran 90/95 compiler:    gfortran -g -O2  
  Obj-C compiler:               
 
  Interfaces supported:      X11, tcltk  
  External libraries:        readline, BLAS(generic), LAPACK(generic)  
  Additional capabilities:   PNG, JPEG, NLS  
  Options enabled:           shared R library, R profiling, Java  

-- 
Andrew Piskorski <atp at piskorski.com>
http://www.piskorski.com/