R SIG Debian - Mar 2012 - [R-sig-hpc] Debian/Ubuntu + threaded BLAS/ATLAS (solved)

On 14 March 2012 at 23:09, Simon Urbanek wrote:
| I thought I'll post this here since I could not find an answer anywhere I
searched: how to setup R with threaded BLAS on Debian/Ubuntu.
| 
| First, neither Debian nor Ubuntu come with optimized ATLAS binaries, simply
because by definition they need to be optimized for a particular machine. The
good news: it is easy to build:
| 
| apt-get source atlas

You could also go to newer Atlas sources at this point.

| cd atlas-3.*
| # less debian/README.Debian # read the docs - it's short
| # sudo apt-get build-dep atlas # <- you may need this if you don't have
all tools installed
| fakeroot debian/rules custom

"Yes but."  

Someone should write an atlas-helper package that deals with this and ...
 
| That will give you (after a long while) a bunch of libatlas*.deb files that
you can easily install with sudo dpkg -i
| 
| Now if you simply configure R --with-blas --with-lapack it will work, and
you'll get a nice and fast BLAS, but unfortunately it will be
single-threaded:
| 
| > m=matrix(rnorm(4e6),2e3)
| > system.time(tcrossprod(m))
|    user  system elapsed 
|   0.684   0.012   0.697 
| 
| The problem is that all those nice libraries that are handled by
update-alternatives are just the single-threaded part of ATLAS. If you want to
use the parallelized part, you'll need to add the libpt* libraries by hand.
This worked for me:

... also this.  Because if you do not deal well with update-alternatives, you
loose the plug-and-play of installing and installing BLAS providing packages.

For the gcbd package/vignette/paper I hacked together a plugin-package,
locally built and hence optmised, but following the Atlas packaging:

edd at max:~$ dpkg -f /var/spool/atlas39/libatlas39_3.9.25-1_amd64.deb 
dpkg -f /var/spool/atlas39/libatlas39_3.9.25-1_amd64.deb 
Package: libatlas39
Source: atlas39
Version: 3.9.25-1
Architecture: amd64
Maintainer: Dirk Eddelbuettel <edd at debian.org>
Installed-Size: 22656
Depends: libc6 (>= 2.4), libgcc1 (>= 1:4.1.1), libgfortran3 (>= 4.3)
Suggests: libblas3gf, liblapack3gf
Provides: libatlas.so.3gf, libblas.so.3gf, liblapack.so.3gf
Section: libs
Priority: optional
Homepage: http://math-atlas.sourceforge.net/
Description: Automatically Tuned Linear Algebra Software, version 3.9.25
 ATLAS is an approach for the automatic generation and optimization of
 numerical software. Currently ATLAS supplies optimized versions for the
 complete set of linear algebra kernels known as the Basic Linear Algebra
 Subroutines (BLAS), and a subset of the linear algebra routines in the
 LAPACK library.
 .
 The libraries in this package are built without any processor extension
 instructions, and should run on all processors of this general
 architecture, albeit less than optimally.
 .
 This package is a local package meant for run-time, built in a somewhat
 non-standard way off the Atlas 3.9.25 sources.
edd at max:~$ 

Someone with interest and a little time should get together with Sylvestre
and see ifwe could turn this into a helper package, or an experimental
package, or ...

| 
| --with-blas='/usr/lib/atlas-base/libptcblas.a
/usr/lib/atlas-base/libptf77blas.a /usr/lib/atlas-base/libatlas.a -lpthread
-lm' --with-lapack
| 
| I opted for using the static version of ATLAS, but you can equally well use
the shared one.
| 
| > m=matrix(rnorm(4e6),2e3)
| > system.time(tcrossprod(m))
|    user  system elapsed 
|   1.548   0.068   0.133 
| 
| Note that by default it will use all cores. You can add "-t N" to
the atlas configure flags in debian/rules if you want to change that (e.g. if
you want to combine it with explicit parallelization).
| 
| I'm simply sharing this in the hope that it will be helpful - please no
flames ;).

Very useful.  It all helps -- Claudia's post was very good, your's here
is
too and I am just trying to move the cart further along.

Dirk
 
| Cheers,
| Simon
| 
| _______________________________________________
| R-sig-hpc mailing list
| R-sig-hpc at r-project.org
| https://stat.ethz.ch/mailman/listinfo/r-sig-hpc

-- 
"Outside of a dog, a book is a man's best friend. Inside of a dog, it
is too
dark to read." -- Groucho Marx