R SIG Fedora - May 2020 - Changing the BLAS from openblas on a F32 box

Of course, even a simpler trick is to launch R as follows:

LD_PRELOAD=/lib64/atlas/libsatlas.so.3 R

and then the symbols in libsatlas take precedence over libopenblas. Or
a mix between both alternatives, i.e., setting

LD_PRELOAD=/path/to/some/link R

and then change that link to point to openblas, atlas... Whatever
suits you best.

I?aki

On Wed, 27 May 2020 at 11:00, I?aki Ucar <iucar at fedoraproject.org>
wrote:
>
> Hi Gavin,
>
> On Wed, 27 May 2020 at 01:15, Gavin Simpson <ucfagls at gmail.com>
wrote:
> >
> > Dear list,
> >
> > What is the recommended incantation on Fedora 32 to swap out the
> > openblas BLAS that the packaged (rpm) version of R-core installs for
> > ATLAS?
>
> I'm afraid there is no official mechanism in place to do that yet.
> There was a proposal [1], but it was never pushed forward due to some
> issues and lack of time. There's a simple hack you can do though.
>
> Since recently, R in Fedora no longer uses openblas-Rblas and links
> against system openblas, as you can see here:
>
> $ ldd /usr/lib64/R/bin/exec/R | grep blas
>         libopenblas.so.0 => /lib64/libopenblas.so.0 (0x00007f8fff849000)
>
> So one simple trick to override that is something along these lines:
>
> $ mkdir ~/blas
> $ ln -s /lib64/atlas/libsatlas.so.3 ~/blas/libopenblas.so.0
> $ LD_LIBRARY_PATH=~/blas ldd /usr/lib64/R/bin/exec/R | grep blas
>         libopenblas.so.0 => /home/****/blas/libopenblas.so.0
> (0x00007f78b3ea2000)
>
> As you can see, now R points to the link in my home, which in turn
> points to system's atlas. Now you can define an alias in your .bashrc
> to always prepend that override to R and Rscript. If you remove the
> link in your home, it will default to system's openblas. If you point
> it to another BLAS implementation, it will pick it up.
>
> Hope it helps.
>
> [1] https://fedoraproject.org/w/index.php?title=PackagingDrafts:BLAS_LAPACK
>
> > If anyone is interested, the issue Simon Wood reports with openblas
> > and *mgcv* is:
> >
> > Issues:
> >
> > ** openblas 0.3.x x<7 is not thread safe if itself compiled for
single thread
> >    use and then called from multiple threads (unlike the reference
BLAS, say).
> >    0.2.20 appears to be OK. For 0.3.x x>6 make USE_THREAD=0
USE_LOCKING=1
> >    to make openblas ensures thread safety (currently unclear if
USE_LOCKING
> >    will be default from 0.3.7).
>
> Does this mean that single-threaded openblas should be built with
> USE_LOCKING=1 by default? Is there any upstream recommendation about
> this?
>
> --
> I?aki ?car


-- 
I?aki ?car

Thanks I?aki, that is exactly what i was looking for, esp the last
option which I have now configured as an alias for easy remembering.

I can answer the question re USE_LOCKING=1. I think that using both
those options is required to get thread-safety even if openblas was
compiled for single thread use. I don't know to what extent Simon has
engaged with upstream on this etc.

All I know is that using the openblas shipped with Fedora for R is
currently a recipe for disaster for the large GAMs we're trying to
fit. But being able to switch to atlas temporarily is a good
alternative.

Cheers

G

On Wed, 27 May 2020 at 03:20, I?aki Ucar <iucar at fedoraproject.org>
wrote:
>
> Of course, even a simpler trick is to launch R as follows:
>
> LD_PRELOAD=/lib64/atlas/libsatlas.so.3 R
>
> and then the symbols in libsatlas take precedence over libopenblas. Or
> a mix between both alternatives, i.e., setting
>
> LD_PRELOAD=/path/to/some/link R
>
> and then change that link to point to openblas, atlas... Whatever
> suits you best.
>
> I?aki
>
> On Wed, 27 May 2020 at 11:00, I?aki Ucar <iucar at fedoraproject.org>
wrote:
> >
> > Hi Gavin,
> >
> > On Wed, 27 May 2020 at 01:15, Gavin Simpson <ucfagls at
gmail.com> wrote:
> > >
> > > Dear list,
> > >
> > > What is the recommended incantation on Fedora 32 to swap out the
> > > openblas BLAS that the packaged (rpm) version of R-core installs
for
> > > ATLAS?
> >
> > I'm afraid there is no official mechanism in place to do that yet.
> > There was a proposal [1], but it was never pushed forward due to some
> > issues and lack of time. There's a simple hack you can do though.
> >
> > Since recently, R in Fedora no longer uses openblas-Rblas and links
> > against system openblas, as you can see here:
> >
> > $ ldd /usr/lib64/R/bin/exec/R | grep blas
> >         libopenblas.so.0 => /lib64/libopenblas.so.0
(0x00007f8fff849000)
> >
> > So one simple trick to override that is something along these lines:
> >
> > $ mkdir ~/blas
> > $ ln -s /lib64/atlas/libsatlas.so.3 ~/blas/libopenblas.so.0
> > $ LD_LIBRARY_PATH=~/blas ldd /usr/lib64/R/bin/exec/R | grep blas
> >         libopenblas.so.0 => /home/****/blas/libopenblas.so.0
> > (0x00007f78b3ea2000)
> >
> > As you can see, now R points to the link in my home, which in turn
> > points to system's atlas. Now you can define an alias in your
.bashrc
> > to always prepend that override to R and Rscript. If you remove the
> > link in your home, it will default to system's openblas. If you
point
> > it to another BLAS implementation, it will pick it up.
> >
> > Hope it helps.
> >
> > [1]
https://fedoraproject.org/w/index.php?title=PackagingDrafts:BLAS_LAPACK
> >
> > > If anyone is interested, the issue Simon Wood reports with
openblas
> > > and *mgcv* is:
> > >
> > > Issues:
> > >
> > > ** openblas 0.3.x x<7 is not thread safe if itself compiled
for single thread
> > >    use and then called from multiple threads (unlike the
reference BLAS, say).
> > >    0.2.20 appears to be OK. For 0.3.x x>6 make USE_THREAD=0
USE_LOCKING=1
> > >    to make openblas ensures thread safety (currently unclear if
USE_LOCKING
> > >    will be default from 0.3.7).
> >
> > Does this mean that single-threaded openblas should be built with
> > USE_LOCKING=1 by default? Is there any upstream recommendation about
> > this?
> >
> > --
> > I?aki ?car
>
>
>
> --
> I?aki ?car


-- 
Gavin Simpson, PhD [he/him/his]           [t] +1 306 337 8863
? Research Scientist                      [f] +1 306 337 2410
Institute of Environmental               [tw] @ucfagls
  Change & Society                        [w] goo.gl/Zpkdem
University of Regina         [w] www.iecs-uregina.ca
Regina, SK S4S 0A2, Canada               [iD] 0000-0002-9084-8413

? Adjunct Professor, Department of Biology, University of Regina.

On Wed, 27 May 2020 at 21:40, Gavin Simpson <ucfagls at gmail.com>
wrote:
>
> Thanks I?aki, that is exactly what i was looking for, esp the last
> option which I have now configured as an alias for easy remembering.
>
> I can answer the question re USE_LOCKING=1. I think that using both
> those options is required to get thread-safety even if openblas was
> compiled for single thread use. I don't know to what extent Simon has
> engaged with upstream on this etc.
I've seen that there is a brief note about this in the project's wiki.
I agree that a sensible default in any distro would be to build
openblas-serial with USE_LOCKING=1. But I don't understand why there
is no recommendation upstream (or I didn't find it besides the note in
the wiki) and there's no idea about the performance penalty that we
incur doing so. I brought this topic to fedora-devel.
> All I know is that using the openblas shipped with Fedora for R is
> currently a recipe for disaster for the large GAMs we're trying to
> fit. But being able to switch to atlas temporarily is a good
> alternative.
Note that switching to openblas-openmp (libopenblaso.so) should be
thread-safe and will probably get you a better performance than Atlas.
Also, Fedora packages blis (which provides
/lib64/blisblas/libblas.so.3). It seems to be thread-safe should be
more performant than Atlas too.

-- 
I?aki ?car