R SIG Debian - Feb 2011 - GotoBLAS2 breaks lapack

Hi,

I'm relatively new to R on Ubuntu (moving from Windows), and I'm trying
to
get GotoBLAS2 working.  I installed (from a CRAN mirror) the pre-built
binaries of R (which, as far as I know, is compiled as a shared library) on
Ubuntu 10.10 using
apt-get install r-base r-base-dev

I successfully built GotoBLAS2 from source, copied the library to /usr/lib
and created s symbolic link from libblas.so.3gf to point to the new
GotoBLAS2 library.

I think the symbolic links and everything are set up correctly, because the
BLAS itself works: I'm using Dirk Eddelbuettel's gbcd package to
benchmark
the difference, and I can still run the matrix multiplication benchmarks (I
guess crossprod doesn't depend on lapack?), and I'm getting a 15x
speedup
over the default BLAS with 8 cores used.  The problem, though, is that when
I try to run anything that depends on lapack--like qr()--I get the
following:

Error in qr.default(a, LAPACK = TRUE) : lapack routines cannot be loaded
In addition: Warning message:
In qr.default(a, LAPACK = TRUE) :
  unable to load shared object '/usr/lib64/R/modules//lapack.so':
  /usr/lib/liblapack.so.3gf: undefined symbol: ATL_chemv

If I delete the symbolic link to GotoBLAS2 and stick the default BLAS back
in there, everything works fine, and it finds lapack once again.  I guess I
don't understand the relationship between BLAS and lapack well enough to
figure out what's happening, because I don't understand how replacing
the
BLAS breaks lapack, and yet the new BLAS still works!  Nothing I've found
online suggests that I need to rebuild lapack to go with GotoBLAS2--I
thought that it should just use the system-wide version.

I've read everything in the R-sig-Debian archives on this, and the
suggestion always seems to be to use the precompiled binaries rather than
building R locally, but I've already done that.

Any help would be much appreciated.

Thanks,
Dan

	[[alternative HTML version deleted]]

On 25 February 2011 at 21:47, dks wrote:
| I'm relatively new to R on Ubuntu (moving from Windows), and I'm
trying to
| get GotoBLAS2 working.  I installed (from a CRAN mirror) the pre-built
| binaries of R (which, as far as I know, is compiled as a shared library) on
| Ubuntu 10.10 using
| apt-get install r-base r-base-dev

Good.
 
| I successfully built GotoBLAS2 from source, copied the library to /usr/lib
| and created s symbolic link from libblas.so.3gf to point to the new
| GotoBLAS2 library.

You did that wrong.  This isn't easy stuff (as understanding how the plug
&
play of all the BLAS / LAPACK alternatives is implemented is non-trivial),
but forcing it isn't the right approach.  Below you mention my gcbd
paper/package. In it, I use a rather awesome 'gotoblas2-helper' package
which
_automates_ creating a deb from the gotoblas2 sources.

I suspect your problem comes from the fact that you only took half of what
GotoBlas can give.  If I look at the deb created from the helper:

edd at max:~/atlas$ dpkg -c
/var/spool/gotoblas2-helper/archive/gotoblas2_1.13-1_amd64.deb |grep lib
drwxr-xr-x root/root         0 2010-07-05 18:57 ./usr/lib/
drwxr-xr-x root/root         0 2010-07-05 19:00 ./usr/lib/gotoblas2/
-rw-r--r-- root/root  10445138 2010-07-05 19:00 ./usr/lib/gotoblas2/libblas.a
-rw-r--r-- root/root   8672978 2010-07-05 19:00 ./usr/lib/gotoblas2/liblapack.a
-rw-r--r-- root/root   5491025 2010-07-05 19:00
./usr/lib/gotoblas2/libblas.so.3gf.0
-rw-r--r-- root/root   7427273 2010-07-05 19:00
./usr/lib/gotoblas2/liblapack.so.3gf.0
edd at max:~/atlas$ 

you see that I got libblas.so and liblapack.so.

So your crashes / lack of lapack success may well be due to Goto assuming its
own lapack, but not getting it.

Dirk

 
| I think the symbolic links and everything are set up correctly, because the
| BLAS itself works: I'm using Dirk Eddelbuettel's gbcd package to
benchmark
| the difference, and I can still run the matrix multiplication benchmarks (I
| guess crossprod doesn't depend on lapack?), and I'm getting a 15x
speedup
| over the default BLAS with 8 cores used.  The problem, though, is that when
| I try to run anything that depends on lapack--like qr()--I get the
| following:
| 
| Error in qr.default(a, LAPACK = TRUE) : lapack routines cannot be loaded
| In addition: Warning message:
| In qr.default(a, LAPACK = TRUE) :
|   unable to load shared object '/usr/lib64/R/modules//lapack.so':
|   /usr/lib/liblapack.so.3gf: undefined symbol: ATL_chemv
| 
| If I delete the symbolic link to GotoBLAS2 and stick the default BLAS back
| in there, everything works fine, and it finds lapack once again.  I guess I
| don't understand the relationship between BLAS and lapack well enough to
| figure out what's happening, because I don't understand how replacing
the
| BLAS breaks lapack, and yet the new BLAS still works!  Nothing I've found
| online suggests that I need to rebuild lapack to go with GotoBLAS2--I
| thought that it should just use the system-wide version.
| 
| I've read everything in the R-sig-Debian archives on this, and the
| suggestion always seems to be to use the precompiled binaries rather than
| building R locally, but I've already done that.
| 
| Any help would be much appreciated.
| 
| Thanks,
| Dan
| 
| 	[[alternative HTML version deleted]]
| 
| _______________________________________________
| R-SIG-Debian mailing list
| R-SIG-Debian at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-debian

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com

Thanks a lot for this, which led me to my solution.  (I'm including all this
in case others run into this problem.)

It turns out that, by default GotoBLAS2 builds a single shared library that
includes both BLAS and Lapack, rather than producing two separate libraries,
which is what Debian/Ubuntu wants.  One of the things that gotoblas2-helper
does is to separate the single shared library into separate BLAS and Lapack
libraries, which is why the deb you showed looks as it does.

I had looked at gotoblas2-helper before, but I saw that it was built before
GotoBLAS adopted the BSD license (when it still required login/password to
download source), and so I didn't think it would still work or get the new
version, but I was wrong on both counts.  It worked great and installed
GotoBLAS2 system-wide.  For others looking at this, full instructions are on
the poster linked from the gbcd paper:
http://prs.ism.ac.jp/~nakama/debian/SC09-poster.pdf

You still have to get a login/password from TACC (
http://www.tacc.utexas.edu/?id=402) and set that and the architecture target
(e.g., "NEHALEM") in the site.conf file, and gotoblas2-helper does the
rest.  I wasn't able to get the public key to download for the ISM
repository, but as long as you're willing to install it unverified, it
should work fine.

(Finally, a bad hack around this is to symbolically link both libblas and
liblapack to point to the single GotoBLAS2 library; it works for R, but I
have no idea what bad consequences might ultimately result.  There are a lot
of web guides out there for installing GotoBLAS2 in Ubuntu, and they all
suggest this strategy of forcing symbolic links, so I'm including this here
in case people google around, so they find the better approach above.)




| I successfully built GotoBLAS2 from source, copied the library to
/usr/lib
> | and created s symbolic link from libblas.so.3gf to point to the new
> | GotoBLAS2 library.
>
> You did that wrong.  This isn't easy stuff (as understanding how the
plug &
> play of all the BLAS / LAPACK alternatives is implemented is non-trivial),
> but forcing it isn't the right approach.  Below you mention my gcbd
> paper/package. In it, I use a rather awesome 'gotoblas2-helper'
package
> which
> _automates_ creating a deb from the gotoblas2 sources.
>
> I suspect your problem comes from the fact that you only took half of what
> GotoBlas can give.  If I look at the deb created from the helper:
>
> edd@max:~/atlas$ dpkg -c
> /var/spool/gotoblas2-helper/archive/gotoblas2_1.13-1_amd64.deb |grep lib
> drwxr-xr-x root/root         0 2010-07-05 18:57 ./usr/lib/
> drwxr-xr-x root/root         0 2010-07-05 19:00 ./usr/lib/gotoblas2/
> -rw-r--r-- root/root  10445138 2010-07-05 19:00
> ./usr/lib/gotoblas2/libblas.a
> -rw-r--r-- root/root   8672978 2010-07-05 19:00
> ./usr/lib/gotoblas2/liblapack.a
> -rw-r--r-- root/root   5491025 2010-07-05 19:00
> ./usr/lib/gotoblas2/libblas.so.3gf.0
> -rw-r--r-- root/root   7427273 2010-07-05 19:00
> ./usr/lib/gotoblas2/liblapack.so.3gf.0
> edd@max:~/atlas$
>
> you see that I got libblas.so and liblapack.so.
>
> So your crashes / lack of lapack success may well be due to Goto assuming
> its
> own lapack, but not getting it.
>
> Dirk
>
	[[alternative HTML version deleted]]