R SIG Debian - Dec 2023 - custom built R will not change BLAS/LAPACK with update-alternatives

Dear All,

I am building R from source[1], following what is done in "rules" for
building Debian's R. But the R I generate, in contrast to the standard
Debian's R, will not change the BLAS and LAPACK libraries it uses when I
change them via "update-alternatives". I have no idea what I am doing
wrong (but, somehow, I've been quite capable of making the same mistake(s)
in three different machines).

I can "solve" the issue by running configure with
--with-blas="-lblas"
--with-lapack="-llapack"

but I do not understand why I need to do this, when Dirk's configure in
rules is using just
--with-blas
--with-lapack

(and this is also what I understand I should be doing from section A.3.1 of
"R Installation and Administration"
---https://cran.r-project.org/doc/manuals/r-release/R-admin.html#BLAS).

This concerns me in so far as it suggests that I am missing something rather
obvious.

Thanks,

R.


Details of the steps I am following.

1. update-alternatives: use Atlas (lower priority than OpenBLAS)

   update-alternatives --config libblas.so.3-x86_64-linux-gnu 
  There are 3 choices for the alternative libblas.so.3-x86_64-linux-gnu
(providing /usr/lib/x86_64-linux-gnu/li

  Selection    Path                                                    Priority 
Status
------------------------------------------------------------
  0            /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3   95      
auto mode
 * 1            /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3             35     
manual mode
  2            /usr/lib/x86_64-linux-gnu/blas/libblas.so.3              10      
manual mode
  3            /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3   95      
manual mode

update-alternatives --config liblapack.so.3-x86_64-linux-gnu 
There are 3 choices for the alternative liblapack.so.3-x86_64-linux-gnu
(providing /usr/lib/x86_64-linux-gnu/
3).

  Selection    Path                                                     
Priority   Status
------------------------------------------------------------
  0            /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3   95    
auto mode
 * 1            /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3             35   
manual mode
  2            /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3            10    
manual mode
  3            /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3   95    
manual mode



2. Just in case, run /sbin/ldconfig

3. No LD_LIBRARY_PATH in env.


4. Configure: ./configure --with-blas --with-lapack --enable-R-shlib
--without-recommended-packages

5. Interestingly, this picks up OpenBLAS:

 configure: searching for an external BLAS
configure: searching for BLAS in default libraries
checking for dgemm_... no
configure: searching for OpenBLAS

and

R is now configured for x86_64-pc-linux-gnu
(...)
External libraries:          pcre2, readline, BLAS(OpenBLAS), LAPACK(in blas),
curl
Options enabled:             shared R library, R profiling
(...)
Options not enabled:         shared BLAS, memory profiling


6. After make is finished, ldd:
libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007f01b1ac2000)
libatlas.so.3 => /usr/lib/x86_64-linux-gnu/libatlas.so.3 (0x00007f01b0f57000)

And no openblas in any line.

7. sessionInfo shows it is using OpenBLAS

  Matrix products: default
BLAS/LAPACK: /usr/lib/x86_64-linux-gnu/openblas-openmp/libopenblasp-r0.3.25.so; 
LAPACK version 3.11.0

8. R CMD config shows difference in the configuration
- My build:
     ./R CMD config BLAS_LIBS: -lopenblas
   - ./R CMD config LAPACK_LIBS: nothing
- Debian's R
   - /usr/bin/R CMD config BLAS_LIBS: -lblas
   - /usr/bin/R CMD config LAPACK_LIBS: -llapack



[1] Why am I building from source? To match the exact version in BioConductor
devel and for a cluster where I am playing with some compiler options



-- 
Ramon Diaz-Uriarte
Department of Biochemistry, Lab B-31
Facultad de Medicina 
Universidad Aut?noma de Madrid 
Arzobispo Morcillo, 4
28029 Madrid
Spain

Phone: +34-91-497-2412

Email: rdiaz02 at gmail.com
       r.diaz at uam.es
       ramon.diaz at iib.uam.es

https://ligarto.org/rdiaz

If you match the exact build-deps I use and the exact options from
debian/rules you should end up with the exact same output. As that is whay
you desire but do not get we need to drill down into where it deviates. FWIW
last time I build the Debian the (relevant) part (of the large) configure log
was below.  What do you get for BLAS/LAPACK?


R is now configured for x86_64-pc-linux-gnu

  Source directory:            .
  Installation directory:      /usr

  C compiler:                  gcc  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  Fortran fixed-form compiler: gfortran  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -fcf-protection

  Default C++ compiler:        g++ -std=gnu++17  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  C++11 compiler:              g++ -std=gnu++11  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  C++14 compiler:              g++ -std=gnu++14  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  C++17 compiler:              g++ -std=gnu++17  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  C++20 compiler:              g++ -std=gnu++20  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  C++23 compiler:              g++ -std=gnu++23  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -Wformat -Werror=format-security -fcf-protection
-Wdate-time -D_FORTIFY_SOURCE=2
  Fortran free-form compiler:  gfortran  -g -O2
-ffile-prefix-map=/build/r-base-4.3.2=. -fstack-protector-strong
-fstack-clash-protection -fcf-protection
  Obj-C compiler:	        

  Interfaces supported:        X11, tcltk
  External libraries:          pcre2, readline, BLAS(generic), LAPACK(generic),
curl
  Additional capabilities:     PNG, JPEG, TIFF, NLS, cairo, ICU
  Options enabled:             shared R library, R profiling, memory profiling

  Capabilities skipped:        
  Options not enabled:         shared BLAS

  Recommended packages:        no


Despite the display of 'shared BLAS' not enabled, I still have

root at fb498fe53700:/# ldd /usr/lib/R/modules/lapack.so 
        linux-vdso.so.1 (0x00007ffcb09c7000)
        libR.so => /lib/libR.so (0x00007ff7dd7f1000)
        liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3
(0x00007ff7dd0f6000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ff7dd017000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff7dce36000)
        libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3
(0x00007ff7dcdd3000)
        libreadline.so.8 => /lib/x86_64-linux-gnu/libreadline.so.8
(0x00007ff7dcd7b000)
        libpcre2-8.so.0 => /lib/x86_64-linux-gnu/libpcre2-8.so.0
(0x00007ff7dccdf000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5
(0x00007ff7dccb0000)
        libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0
(0x00007ff7dcc9d000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ff7dcc7e000)
        libtirpc.so.3 => /lib/x86_64-linux-gnu/libtirpc.so.3
(0x00007ff7dcc50000)
        libicuuc.so.72 => /lib/x86_64-linux-gnu/libicuuc.so.72
(0x00007ff7dca52000)
        libicui18n.so.72 => /lib/x86_64-linux-gnu/libicui18n.so.72
(0x00007ff7dc726000)
        libgomp.so.1 => /lib/x86_64-linux-gnu/libgomp.so.1
(0x00007ff7dc6de000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff7ddc74000)
        libopenblas.so.0 => /lib/x86_64-linux-gnu/libopenblas.so.0
(0x00007ff7da3f0000)
        libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5
(0x00007ff7da126000)
        libtinfo.so.6 => /lib/x86_64-linux-gnu/libtinfo.so.6
(0x00007ff7da0f3000)
        libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2
(0x00007ff7da0a1000)
        libicudata.so.72 => /lib/x86_64-linux-gnu/libicudata.so.72
(0x00007ff7d82cf000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007ff7d80b5000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
(0x00007ff7d8095000)
        libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0
(0x00007ff7d804e000)
        libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3
(0x00007ff7d7f74000)
        libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3
(0x00007ff7d7f47000)
        libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2
(0x00007ff7d7f41000)
        libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0
(0x00007ff7d7f33000)
        libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1
(0x00007ff7d7f2c000)
        libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2
(0x00007ff7d7f1b000)
root at fb498fe53700:/# ls -l /lib/x86_64-linux-gnu/liblapack.so.3
lrwxrwxrwx 1 root root 49 Nov 27  2022 /lib/x86_64-linux-gnu/liblapack.so.3
-> /etc/alternatives/liblapack.so.3-x86_64-linux-gnu
root at fb498fe53700:/# ls -l /lib/x86_64-linux-gnu/libblas.so.3
lrwxrwxrwx 1 root root 47 Nov 27  2022 /lib/x86_64-linux-gnu/libblas.so.3 ->
/etc/alternatives/libblas.so.3-x86_64-linux-gnu
root at fb498fe53700:/# 

the /etc/alternatives mechanism you are after. (That was from rocker/r-base,
a quick way for me to get to clean and current Debian build.)

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

On Sat, 30 Dec 2023 18:37:06 +0100
Ramon Diaz-Uriarte <rdiaz02 at gmail.com> wrote:
>  configure: searching for an external BLAS
> configure: searching for BLAS in default libraries
> checking for dgemm_... no
> configure: searching for OpenBLAS
The code implementing this logic is in m4/R.m4, starting around line
2600. If --with-blas is given without an argument, the configure script
goes searching for various BLASes. It happens to try -lopenblas before
-lblas, so that's what ends up being linked.

Does it help to use --with-blas=blas or --with-blas=-lblas so that the
configure script doesn't try to detect it automatically?

-- 
Best regards,
Ivan