R SIG Debian - Mar 2011 - Special BLAS and explcit parallel code

Hi

I just read the thread on gotoBLAS, as well as the excellent vignette of 
gcbd. I still have some confusion and would like to ask very basic 
questions, hope I am not taking too much of your time.

The point that retained my attention was the question of using implicit 
(multi-threaded blas) versus explicit (parallel code) optimisation. As I 
understood, the ideal would be to use a multi-threaded BLAS for simple 
code, and restrict it to use one core when parallel R code is used? Do 
you agree? I could not find the presentation of R Bivand on this... Any 
other references on this question?

Secondly, I am a little bit confused about how this is done on Ubuntu. 
First of all.... I am not sure of which BLAS is actually being used... a 
dpkg -l indicated me that both liblas (-dev and 3gf) and libatlas 
(3gf-base) are installed... Doing:

$ ls -n /usr/lib/R/lib/
total 2528
-rw-r--r-- 1 0 0 2583076 2011-02-26 04:21 libR.so

I don't see which one is used actually... how can I figure out?

Thanks!

Matthieu

On 15 March 2011 at 18:27, Matthieu Stigler wrote:
| Hi
| 
| I just read the thread on gotoBLAS, as well as the excellent vignette of 
| gcbd. I still have some confusion and would like to ask very basic 
| questions, hope I am not taking too much of your time.
| 
| The point that retained my attention was the question of using implicit 
| (multi-threaded blas) versus explicit (parallel code) optimisation. As I 
| understood, the ideal would be to use a multi-threaded BLAS for simple 
| code, and restrict it to use one core when parallel R code is used? Do 

Right, as fine-grained parallelism (via BLAS) can clash with coarse-grained
parallelism (via, say, multicore) where you could end up with
'overbooked'
cpus.  Being able to tell the BLAS implementation to _not_ use parallel code
is a nice feature ... which Atlas for example does not have.

| you agree? I could not find the presentation of R Bivand on this... Any 
| other references on this question?
| 
| Secondly, I am a little bit confused about how this is done on Ubuntu. 
| First of all.... I am not sure of which BLAS is actually being used... a 
| dpkg -l indicated me that both liblas (-dev and 3gf) and libatlas 
| (3gf-base) are installed... Doing:
| 
| $ ls -n /usr/lib/R/lib/
| total 2528
| -rw-r--r-- 1 0 0 2583076 2011-02-26 04:21 libR.so
| 
| I don't see which one is used actually... how can I figure out?

Use the 'ldd' command, not the 'ls' command, and run

    $ ldd /usr/lib/R/lib/libR.so
    $ ldd /usr/lib/R/modules/lapack.so

That should show you e.g. your Atlas libblas and liblapack, if you have those
packages installed.

Also note that /usr/bin/R sets more LD_LIBRARY_PATH arguments meaning so it
can potentially see more locations than the system default for ld.so
reflected in the ldd output you just saw..  That is the trick which was used
by the MKL package so that R saw those libraries but other programs did
not... (as per the wishes of Intel who gave REvo permission to distribute MKL
just for R on Ubuntu).

Makes sense?

Dirk

-- 
Dirk Eddelbuettel | edd at debian.org | http://dirk.eddelbuettel.com