R help - Nov 2022 - make install libgfortran.so.5: cannot open shared object file: No such file or directory; conftest.c:1:10: fatal error: jni.h: No such file or directory

On Fri, 18 Nov 2022 16:45:14 -0500
Rob Kudyba <rk3199 at columbia.edu> wrote:
> Here is where the error occurs in make install:
> [...]
> installing packages ...
>   building HTML index ...
> /path/to/R-4.2.2/bin/exec/R: error while loading shared libraries:
> libgfortran.so.5: cannot open shared object file: No such file or
> directory make[2]: *** [install] Error 127
> make[2]: Leaving directory `/path/to/R-4.2.2/src/library'
> make[1]: *** [install] Error 1
> make[1]: Leaving directory `/path/to/rk3199/R-4.2.2/src'
> make: *** [install] Error 1
It should be possible to run R without installing it, as
/path/to/R-4.2.2/bin/R (strictly speaking, as bin/R under the build
directory, if you're building R separately from the source tree). Does
it work?

Does /path/to/R-4.2.2/bin/R -d ldd give you any useful information? Can
you use strace -f -e openat /path/to/R-4.2.2/bin/R to confirm that the
dynamic library loader is looking in /path/to/gcc-11.2/lib64 for the
shared objects?

If /path/to/gcc-11.2/lib64 is missing from both /etc/ld.so.conf and the
environment variable LD_LIBRARY_PATH and you don't want to set it
manually every time, you can either edit /path/to/R-4.2.2/etc/ldpaths
or pass the -Wl,-rpath=/path/to/gcc-11.2/lib64 flag to the linker.
> During make I see this error with Java:
This error is not fatal. rJava won't work, but the rest of R should be
fine.

-- 
Best regards,
Ivan

> It should be possible to run R without installing it, as
> /path/to/R-4.2.2/bin/R (strictly speaking, as bin/R under the build
> directory, if you're building R separately from the source tree). Does
> it work?
>
So far R does seem to be working and I've tested installing some packages.
Is there any sample R program I can run to test against this error?
>
> Does /path/to/R-4.2.2/bin/R -d ldd give you any useful information?

R -d ldd
linux-vdso.so.1 =>  (0x00002aaaaaacd000)
libRblas.so => /path/to/R-4.2/lib64/R/lib/libRblas.so (0x00002aaaaaccf000)
libgfortran.so.5 => /path/to/gcc-11.2/lib64/libgfortran.so.5
(0x00002aaaaaef9000)
libm.so.6 => /lib64/libm.so.6 (0x00002aaaab3a0000)
libquadmath.so.0 => /path/to/gcc-11.2/lib64/libquadmath.so.0
(0x00002aaaab6a2000)
libreadline.so.6 => /lib64/libreadline.so.6 (0x00002aaaab8e9000)
libpcre2-8.so.0 => /path/to/pcre2-10.35/lib/libpcre2-8.so.0
(0x00002aaaabb2f000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00002aaaabd88000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00002aaaabfae000)
libz.so.1 => /lib64/libz.so.1 (0x00002aaaac1be000)
librt.so.1 => /lib64/librt.so.1 (0x00002aaaac3d4000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002aaaac5dc000)
libgomp.so.1 => /path/to/gcc-11.2/lib64/libgomp.so.1 (0x00002aaaac7e0000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002aaaaca21000)
libc.so.6 => /lib64/libc.so.6 (0x00002aaaacc3d000)
/lib64/ld-linux-x86-64.so.2 (0x00002aaaaaaab000)
libgcc_s.so.1 => /path/to/gcc-11.2/lib64/libgcc_s.so.1 (0x00002aaaad00a000)
libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00002aaaad222000)

> Can
> you use strace -f -e openat /path/to/R-4.2.2/bin/R to confirm that the
> dynamic library loader is looking in /path/to/gcc-11.2/lib64 for the
> shared objects?
>
[pid 241836] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=241836,
si_uid=547289, si_status=0, si_utime=0, si_stime=0} ---
openat(AT_FDCWD, "/sys/devices/system/cpu",
O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3

R version 4.2.2 (2022-10-31) -- "Innocent and Trusting"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.

  Natural language support but running in an English locale

R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.

[Previously saved workspace restored]

strace: Process 241837 attached
strace: Process 241838 attached
[pid 241838] +++ exited with 0 +++
[pid 241837] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED,
si_pid=241838, si_uid=547289, si_status=0, si_utime=0, si_stime=0} ---
[pid 241837] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=241837,
si_uid=547289, si_status=0, si_utime=0, si_stime=0} ---
strace: Process 241839 attached
[pid 241839] +++ exited with 0 +++
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=241839,
si_uid=547289, si_status=0, si_utime=0, si_stime=0} ---

If /path/to/gcc-11.2/lib64 is missing from both /etc/ld.so.conf and
the
> environment variable LD_LIBRARY_PATH and you don't want to set it
> manually every time, you can either edit /path/to/R-4.2.2/etc/ldpaths
> or pass the -Wl,-rpath=/path/to/gcc-11.2/lib64 flag to the linker.
>
/path/to/gcc-11.2/lib is definitely in LD_LIBRARY_PATH when loading the GCC
11.2 module.

If using the /path/to/R-4.2.2/etc/ldpaths where would I put the correct
path? I would've hoped the configure/make process would've found this
automatically.

if test -n "/usr/local/lib64"; then
: ${R_LD_LIBRARY_PATH=${R_HOME}/lib:/usr/local/lib64}
else
: ${R_LD_LIBRARY_PATH=${R_HOME}/lib}
fi
if test -n "${R_JAVA_LD_LIBRARY_PATH}"; then
  R_LD_LIBRARY_PATH="${R_LD_LIBRARY_PATH}:${R_JAVA_LD_LIBRARY_PATH}"
fi
## This is DYLD_FALLBACK_LIBRARY_PATH on Darwin (macOS) and
## LD_LIBRARY_PATH elsewhere.
## However, on macOS >=10.11 (if SIP is enabled, the default), the
## environment value will not be passed to a script such as R.sh, so
## would not seen here.
if test -z "${LD_LIBRARY_PATH}"; then
  LD_LIBRARY_PATH="${R_LD_LIBRARY_PATH}"
else
  LD_LIBRARY_PATH="${R_LD_LIBRARY_PATH}:${LD_LIBRARY_PATH}"
fi
export LD_LIBRARY_PATH
> During make I see this error with Java:
>
> This error is not fatal. rJava won't work, but the rest of R should be
> fine.
>
On RHEL from what I see openjdk-devel provides jni.h.

	[[alternative HTML version deleted]]

On Sun, 20 Nov 2022 14:03:34 -0500
Rob Kudyba <rk3199 at columbia.edu> wrote:
> /path/to/gcc-11.2/lib is definitely in LD_LIBRARY_PATH when loading
> the GCC 11.2 module.
> 
> If using the /path/to/R-4.2.2/etc/ldpaths where would I put the
> correct path? I would've hoped the configure/make process would've
> found this automatically.
Interesting that bin/R works but calling bin/R from what amounts to
`make -C src/library install` doesn't. I think that the failing step is
the following command:

@$(ECHO) "  building HTML index ..."
@$(ECHO) "utils:::make.packages.html(.Library, verbose=FALSE,
docdir=\"$(DESTDIR)${rdocdir}\")" | \
 R_DEFAULT_PACKAGES=NULL LC_ALL=C ${R_EXE} >/dev/null

I don't see a reason for Make to lose the LD_LIBRARY_PATH when
launching R like this.

Can you see the failing step in the `make -d install` output? There
will be a lot of text, most of it unrelated, unfortunately.

If something loses the LD_LIBRARY_PATH environment variable on the way
from Make to R, hard-coding it in etc/ldpaths should work around the
problem, but it may be hard to find out what went wrong.

If you run `mkfifo /tmp/1` and insert `read foo </tmp/1` into
etc/ldpaths, it should hang until you write something into /tmp/1.
Using this, it should be possible to inspect /proc/${pid}/environ for
the whole process tree from the parent Make process to the shell that's
about to launch R. Can you find the first process that lacks the proper
LD_LIBRARY_PATH environment variable?

-- 
Best regards,
Ivan