Sam H
2020-Oct-08 18:37 UTC
[R-sig-Debian] Installing package fails at "testing if installed package can be loaded from temporary location"
Hi, I can not install packages from source which links to RcppArmadillo on Ubuntu 20.04 (after upgrading from 18.04). The following problem occurs: ** testing if installed package can be loaded from temporary location Error: package or namespace load failed for 'myPackage' in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/usr/local/lib/R/site-library/00LOCK-my_package/00new/myPackage/libs/myPackage.so': /usr/local/lib/R/site-library/00LOCK-my_package/00new/myPackge/libs/myPackage.so: undefined symbol: dgesvx_ Error: loading failed Execution halted ERROR: loading failed This dgesvx_ looked to me like a LAPACK problem ( https://www.netlib.org/lapack/explore-html/d7/d3b/group__double_g_esolve_ga9d90ccf6e340cacd08b7bbbb502ceb21.html#ga9d90ccf6e340cacd08b7bbbb502ceb21 ). I was playing with different BLAS/LAPACK versions on my system and researching this for a couple of days now but without success. I was first looking at which LAPACK and BLAS R is "using" user at user:~$ ps aux | grep exec/R user+ 11663 1.9 1.7 1030952 415240 pts/0 Sl+ 06:33 0:13 /usr/lib/R/bin/exec/R --vanilla --quiet user+ 13836 2.5 0.2 441664 52024 pts/1 Sl+ 06:44 0:00 /usr/lib/R/bin/exec/R user+ 13932 0.0 0.0 8900 660 pts/2 S+ 06:45 0:00 grep exec/R user at user:~$ lsof -p 13836 | grep 'blas\|lapack' R 13836 user mem REG 8,1 33562376 3961750 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 R 13836 user mem REG 8,1 27200552 3961749 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 user at user:~$ ls -l /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 -rw-r--r-- 1 root root 27200552 Jun 2 06:55 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 user at user:~$ ls -l /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 -rw-r--r-- 1 root root 33562376 Jun 2 06:55 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 And then tried different versions: user at user:~$ sudo update-alternatives --config libblas.so.3-x86_64-linux-gnu There are 4 choices for the alternative libblas.so.3-x86_64-linux-gnu (providing /usr/lib/x86_64-linux-gnu/libblas.so.3). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 100 auto mode 1 /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3 35 manual mode 2 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 10 manual mode 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3 95 manual mode 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 100 manual mode Press <enter> to keep the current choice[*], or type selection number: 3 update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3 to provide /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in manual mode user at user:~$ sudo update-alternatives --config liblapack.so.3-x86_64-linux-gnu There are 4 choices for the alternative liblapack.so.3-x86_64-linux-gnu (providing /usr/lib/x86_64-linux-gnu/liblapack.so.3). Selection Path Priority Status ------------------------------------------------------------ * 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 100 auto mode 1 /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3 35 manual mode 2 /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 10 manual mode 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3 95 manual mode 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 100 manual mode Press <enter> to keep the current choice[*], or type selection number: 3 update-alternatives: using /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3 to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) in manual mode user at user:~$ sudo update-alternatives --config libblas.so.3-x86_64-linux-gnu There are 4 choices for the alternative libblas.so.3-x86_64-linux-gnu (providing /usr/lib/x86_64-linux-gnu/libblas.so.3). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 100 auto mode 1 /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3 35 manual mode 2 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 10 manual mode * 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3 95 manual mode 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 100 manual mode Press <enter> to keep the current choice[*], or type selection number: 2 update-alternatives: using /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 to provide /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in manual mode user at user:~$ sudo update-alternatives --config liblapack.so.3-x86_64-linux-gnu There are 4 choices for the alternative liblapack.so.3-x86_64-linux-gnu (providing /usr/lib/x86_64-linux-gnu/liblapack.so.3). Selection Path Priority Status ------------------------------------------------------------ 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 100 auto mode 1 /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3 35 manual mode 2 /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 10 manual mode * 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3 95 manual mode 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 100 manual mode Press <enter> to keep the current choice[*], or type selection number: 2 update-alternatives: using /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) in manual mode I have no idea what could be wrong here... Thanks in advance for hints and help on what to do here. Best regards, Sam [[alternative HTML version deleted]]
Dirk Eddelbuettel
2020-Oct-08 19:37 UTC
[R-sig-Debian] Installing package fails at "testing if installed package can be loaded from temporary location"
Hi Sam, Thanks for bringing this here. It is the right place. And for what it is worth I also use 20.04, and update RcppArmadillo frequently (currently working on a pre-release) On 8 October 2020 at 14:37, Sam H wrote: | Hi, | | I can not install packages from source which links to RcppArmadillo on | Ubuntu 20.04 (after upgrading from 18.04). The following problem occurs: | | ** testing if installed package can be loaded from temporary location | Error: package or namespace load failed for 'myPackage' in | dyn.load(file, DLLpath = DLLpath, ...): | unable to load shared object | '/usr/local/lib/R/site-library/00LOCK-my_package/00new/myPackage/libs/myPackage.so': | /usr/local/lib/R/site-library/00LOCK-my_package/00new/myPackge/libs/myPackage.so: | undefined symbol: dgesvx_ Ok, this could be two things. Are you using the R package for Ubuntu from CRAN, or directly from the Rutter PPA ? All this works best with R being compiled using shared libraries and external LAPACK/BLAS. So in R itself, when you type sessionInfo() you should see two lines like this: Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 Second, for any locally compiled package, `ldd` on that .so file should resolve LAPACK and BLAS: edd at rob:~$ ldd /usr/local/lib/R/site-library/RcppZiggurat/libs/RcppZiggurat.so | grep "blas\|lapack" libgslcblas.so.0 => /usr/lib/x86_64-linux-gnu/libgslcblas.so.0 (0x00007effdd3ab000) libblas.so.3 => /usr/lib/x86_64-linux-gnu/libblas.so.3 (0x00007effdb9ac000) edd at rob:~$ | Error: loading failed | Execution halted | ERROR: loading failed | | This dgesvx_ looked to me like a LAPACK problem ( | https://www.netlib.org/lapack/explore-html/d7/d3b/group__double_g_esolve_ga9d90ccf6e340cacd08b7bbbb502ceb21.html#ga9d90ccf6e340cacd08b7bbbb502ceb21 | ). | | I was playing with different BLAS/LAPACK versions on my system and | researching this for a couple of days now but without success. | | I was first looking at which LAPACK and BLAS R is "using" | | user at user:~$ ps aux | grep exec/R | user+ 11663 1.9 1.7 1030952 415240 pts/0 Sl+ 06:33 0:13 | /usr/lib/R/bin/exec/R --vanilla --quiet | user+ 13836 2.5 0.2 441664 52024 pts/1 Sl+ 06:44 0:00 | /usr/lib/R/bin/exec/R | user+ 13932 0.0 0.0 8900 660 pts/2 S+ 06:45 0:00 grep exec/R | user at user:~$ lsof -p 13836 | grep 'blas\|lapack' | R 13836 user mem REG 8,1 33562376 3961750 | /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 Same as mine above, found via a more hardcore way :) | R 13836 user mem REG 8,1 27200552 3961749 | /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | user at user:~$ ls -l /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | -rw-r--r-- 1 root root 27200552 Jun 2 06:55 | /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | user at user:~$ ls -l /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 | -rw-r--r-- 1 root root 33562376 Jun 2 06:55 | /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 | | And then tried different versions: | | user at user:~$ sudo update-alternatives --config libblas.so.3-x86_64-linux-gnu | There are 4 choices for the alternative libblas.so.3-x86_64-linux-gnu | (providing /usr/lib/x86_64-linux-gnu/libblas.so.3). | | Selection Path | Priority Status | ------------------------------------------------------------ | * 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | 100 auto mode | 1 /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3 35 | manual mode | 2 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 10 | manual mode | 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3 95 | manual mode | 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | 100 manual mode Also fine. | Press <enter> to keep the current choice[*], or type selection number: 3 | update-alternatives: using | /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3 to provide | /usr/lib/x86_64-linux-gnu/libblas.so.3 (libblas.so.3-x86_64-linux-gnu) in | manual mode | user at user:~$ sudo update-alternatives --config | liblapack.so.3-x86_64-linux-gnu | There are 4 choices for the alternative liblapack.so.3-x86_64-linux-gnu | (providing /usr/lib/x86_64-linux-gnu/liblapack.so.3). | | Selection Path | Priority Status | ------------------------------------------------------------ | * 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 | 100 auto mode | 1 /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3 | 35 manual mode | 2 /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 | 10 manual mode | 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3 | 95 manual mode | 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 | 100 manual mode | | Press <enter> to keep the current choice[*], or type selection number: 3 | update-alternatives: using | /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3 to provide | /usr/lib/x86_64-linux-gnu/liblapack.so.3 (liblapack.so.3-x86_64-linux-gnu) | in manual mode | user at user:~$ sudo update-alternatives --config libblas.so.3-x86_64-linux-gnu | There are 4 choices for the alternative libblas.so.3-x86_64-linux-gnu | (providing /usr/lib/x86_64-linux-gnu/libblas.so.3). | | Selection Path | Priority Status | ------------------------------------------------------------ | 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | 100 auto mode | 1 /usr/lib/x86_64-linux-gnu/atlas/libblas.so.3 35 | manual mode | 2 /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 10 | manual mode | * 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/libblas.so.3 95 | manual mode | 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 | 100 manual mode | | Press <enter> to keep the current choice[*], or type selection number: 2 | update-alternatives: using /usr/lib/x86_64-linux-gnu/blas/libblas.so.3 to | provide /usr/lib/x86_64-linux-gnu/libblas.so.3 | (libblas.so.3-x86_64-linux-gnu) in manual mode | user at user:~$ sudo update-alternatives --config | liblapack.so.3-x86_64-linux-gnu | There are 4 choices for the alternative liblapack.so.3-x86_64-linux-gnu | (providing /usr/lib/x86_64-linux-gnu/liblapack.so.3). | | Selection Path | Priority Status | ------------------------------------------------------------ | 0 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 | 100 auto mode | 1 /usr/lib/x86_64-linux-gnu/atlas/liblapack.so.3 | 35 manual mode | 2 /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 | 10 manual mode | * 3 /usr/lib/x86_64-linux-gnu/openblas-openmp/liblapack.so.3 | 95 manual mode | 4 /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3 | 100 manual mode | | Press <enter> to keep the current choice[*], or type selection number: 2 | update-alternatives: using /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3 | to provide /usr/lib/x86_64-linux-gnu/liblapack.so.3 | (liblapack.so.3-x86_64-linux-gnu) in manual mode | I have no idea what could be wrong here... Thanks in advance for hints and | help on what to do here. It is possible to 'bust' one's system by playing with update-alternatives. I usually don't touch it, or make triple-sure I am doing it right. I would check whether linear algebra currently works for you. One check: $ Rscript -e 'library(RcppArmadillo); example("fastLm") Dirk | Best regards, | Sam | | [[alternative HTML version deleted]] | | _______________________________________________ | R-SIG-Debian mailing list | R-SIG-Debian at r-project.org | https://stat.ethz.ch/mailman/listinfo/r-sig-debian -- https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
Dirk Eddelbuettel
2020-Oct-08 19:39 UTC
[R-sig-Debian] Installing package fails at "testing if installed package can be loaded from temporary location"
Sam, PS It could of course also be something else: Is you src/Makevars for your test package correct? Did you use `RcppArmadillo.packages.skeleton()` ? You need the lapack/blas link instruction therein. See edd at rob:~$ cat /usr/local/lib/R/site-library/RcppArmadillo/skeleton/Makevars ## With R 3.1.0 or later, you can uncomment the following line to tell R to ## enable compilation with C++11 (where available) ## ## Also, OpenMP support in Armadillo prefers C++11 support. However, for wider ## availability of the package we do not yet enforce this here. It is however ## recommended for client packages to set it. ## ## And with R 3.4.0, and RcppArmadillo 0.7.960.*, we turn C++11 on as OpenMP ## support within Armadillo prefers / requires it CXX_STD = CXX11 PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) edd at rob:~$ Dirk -- https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org
Sam H
2020-Oct-08 20:43 UTC
[R-sig-Debian] Installing package fails at "testing if installed package can be loaded from temporary location"
Dirk, thank you a thousand times. Indeed, src/Makevars was wrong. I modified Makevars so that now looks like the below and the package now compiled and linked properly. CXX_STD = CXX11 PKG_LIBS += $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) $(shell ${R_HOME}/bin/Rscript -e "RcppParallel::RcppParallelLibs()") PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) -I../inst/include Best, Sam On Thu, Oct 8, 2020 at 3:39 PM Dirk Eddelbuettel <edd at debian.org> wrote:> > Sam, > > PS It could of course also be something else: Is you src/Makevars for your > test package correct? Did you use `RcppArmadillo.packages.skeleton()` ? > You > need the lapack/blas link instruction therein. See > > edd at rob:~$ cat > /usr/local/lib/R/site-library/RcppArmadillo/skeleton/Makevars > > ## With R 3.1.0 or later, you can uncomment the following line to tell R > to > ## enable compilation with C++11 (where available) > ## > ## Also, OpenMP support in Armadillo prefers C++11 support. However, for > wider > ## availability of the package we do not yet enforce this here. It is > however > ## recommended for client packages to set it. > ## > ## And with R 3.4.0, and RcppArmadillo 0.7.960.*, we turn C++11 on as > OpenMP > ## support within Armadillo prefers / requires it > CXX_STD = CXX11 > > PKG_CXXFLAGS = $(SHLIB_OPENMP_CXXFLAGS) > PKG_LIBS = $(SHLIB_OPENMP_CXXFLAGS) $(LAPACK_LIBS) $(BLAS_LIBS) $(FLIBS) > edd at rob:~$ > > > Dirk > > -- > https://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org >-- Best regards, Sam [[alternative HTML version deleted]]
Possibly Parallel Threads
- Why is my R package still compiling with the O2 flag?
- Installing package fails at "testing if installed package can be loaded from temporary location"
- Rcpp with OpenMP - Need example Makevars
- Experimental CXX_STD problem in R 3.4
- Experimental CXX_STD problem in R 3.4