R SIG Fedora - May 2020 - Changing the BLAS from openblas on a F32 box

Dear list,

What is the recommended incantation on Fedora 32 to swap out the
openblas BLAS that the packaged (rpm) version of R-core installs for
ATLAS?

I'm running into some problems with some big models I want to fit
using the mgcv package, and openblas is apparently not thread safe and
is causing problems.

I have the following installed:

$ dnf list installed | grep ^R
R-core.x86_64                               3.6.3-1.fc32
      @fedora
R-core-devel.x86_64                         3.6.3-1.fc32
      @fedora
$ dnf list installed | grep openblas-R
openblas-Rblas.x86_64                       0.3.9-2.fc32
      @fedora

And i'd like to temporarily switch to the atlas BLAS.

If anyone is interested, the issue Simon Wood reports with openblas
and *mgcv* is:

Issues:

** openblas 0.3.x x<7 is not thread safe if itself compiled for single thread
   use and then called from multiple threads (unlike the reference BLAS, say).
   0.2.20 appears to be OK. For 0.3.x x>6 make USE_THREAD=0 USE_LOCKING=1
   to make openblas ensures thread safety (currently unclear if USE_LOCKING
   will be default from 0.3.7).

and running `mgcv:::blas.thread.test()` (Simon's non-exported function
to test for problems) returns:
> mgcv:::blas.thread.test()
  |                                                                      |   0%
BLAS thread safety problem at iteration 2

On my systems.

Thanks in advance for any suggestions as to how to best swap in/out the BLAS

All the best

Gavin
-- 
Gavin Simpson, PhD [he/him/his]           [t] +1 306 337 8863
? Research Scientist                      [f] +1 306 337 2410
Institute of Environmental               [tw] @ucfagls
  Change & Society                        [w] goo.gl/Zpkdem
University of Regina         [w] www.iecs-uregina.ca
Regina, SK S4S 0A2, Canada               [iD] 0000-0002-9084-8413

? Adjunct Professor, Department of Biology, University of Regina.

Hi Gavin,

On Wed, 27 May 2020 at 01:15, Gavin Simpson <ucfagls at gmail.com>
wrote:
>
> Dear list,
>
> What is the recommended incantation on Fedora 32 to swap out the
> openblas BLAS that the packaged (rpm) version of R-core installs for
> ATLAS?
I'm afraid there is no official mechanism in place to do that yet.
There was a proposal [1], but it was never pushed forward due to some
issues and lack of time. There's a simple hack you can do though.

Since recently, R in Fedora no longer uses openblas-Rblas and links
against system openblas, as you can see here:

$ ldd /usr/lib64/R/bin/exec/R | grep blas
        libopenblas.so.0 => /lib64/libopenblas.so.0 (0x00007f8fff849000)

So one simple trick to override that is something along these lines:

$ mkdir ~/blas
$ ln -s /lib64/atlas/libsatlas.so.3 ~/blas/libopenblas.so.0
$ LD_LIBRARY_PATH=~/blas ldd /usr/lib64/R/bin/exec/R | grep blas
        libopenblas.so.0 => /home/****/blas/libopenblas.so.0
(0x00007f78b3ea2000)

As you can see, now R points to the link in my home, which in turn
points to system's atlas. Now you can define an alias in your .bashrc
to always prepend that override to R and Rscript. If you remove the
link in your home, it will default to system's openblas. If you point
it to another BLAS implementation, it will pick it up.

Hope it helps.

[1] https://fedoraproject.org/w/index.php?title=PackagingDrafts:BLAS_LAPACK
> If anyone is interested, the issue Simon Wood reports with openblas
> and *mgcv* is:
>
> Issues:
>
> ** openblas 0.3.x x<7 is not thread safe if itself compiled for single
thread
>    use and then called from multiple threads (unlike the reference BLAS,
say).
>    0.2.20 appears to be OK. For 0.3.x x>6 make USE_THREAD=0
USE_LOCKING=1
>    to make openblas ensures thread safety (currently unclear if USE_LOCKING
>    will be default from 0.3.7).
Does this mean that single-threaded openblas should be built with
USE_LOCKING=1 by default? Is there any upstream recommendation about
this?

-- 
I?aki ?car

Of course, even a simpler trick is to launch R as follows:

LD_PRELOAD=/lib64/atlas/libsatlas.so.3 R

and then the symbols in libsatlas take precedence over libopenblas. Or
a mix between both alternatives, i.e., setting

LD_PRELOAD=/path/to/some/link R

and then change that link to point to openblas, atlas... Whatever
suits you best.

I?aki

On Wed, 27 May 2020 at 11:00, I?aki Ucar <iucar at fedoraproject.org>
wrote:
>
> Hi Gavin,
>
> On Wed, 27 May 2020 at 01:15, Gavin Simpson <ucfagls at gmail.com>
wrote:
> >
> > Dear list,
> >
> > What is the recommended incantation on Fedora 32 to swap out the
> > openblas BLAS that the packaged (rpm) version of R-core installs for
> > ATLAS?
>
> I'm afraid there is no official mechanism in place to do that yet.
> There was a proposal [1], but it was never pushed forward due to some
> issues and lack of time. There's a simple hack you can do though.
>
> Since recently, R in Fedora no longer uses openblas-Rblas and links
> against system openblas, as you can see here:
>
> $ ldd /usr/lib64/R/bin/exec/R | grep blas
>         libopenblas.so.0 => /lib64/libopenblas.so.0 (0x00007f8fff849000)
>
> So one simple trick to override that is something along these lines:
>
> $ mkdir ~/blas
> $ ln -s /lib64/atlas/libsatlas.so.3 ~/blas/libopenblas.so.0
> $ LD_LIBRARY_PATH=~/blas ldd /usr/lib64/R/bin/exec/R | grep blas
>         libopenblas.so.0 => /home/****/blas/libopenblas.so.0
> (0x00007f78b3ea2000)
>
> As you can see, now R points to the link in my home, which in turn
> points to system's atlas. Now you can define an alias in your .bashrc
> to always prepend that override to R and Rscript. If you remove the
> link in your home, it will default to system's openblas. If you point
> it to another BLAS implementation, it will pick it up.
>
> Hope it helps.
>
> [1] https://fedoraproject.org/w/index.php?title=PackagingDrafts:BLAS_LAPACK
>
> > If anyone is interested, the issue Simon Wood reports with openblas
> > and *mgcv* is:
> >
> > Issues:
> >
> > ** openblas 0.3.x x<7 is not thread safe if itself compiled for
single thread
> >    use and then called from multiple threads (unlike the reference
BLAS, say).
> >    0.2.20 appears to be OK. For 0.3.x x>6 make USE_THREAD=0
USE_LOCKING=1
> >    to make openblas ensures thread safety (currently unclear if
USE_LOCKING
> >    will be default from 0.3.7).
>
> Does this mean that single-threaded openblas should be built with
> USE_LOCKING=1 by default? Is there any upstream recommendation about
> this?
>
> --
> I?aki ?car


-- 
I?aki ?car