R help - Nov 2022 - make install libgfortran.so.5: cannot open shared object file: No such file or directory; conftest.c:1:10: fatal error: jni.h: No such file or directory

On Sun, 20 Nov 2022 14:03:34 -0500
Rob Kudyba <rk3199 at columbia.edu> wrote:
> /path/to/gcc-11.2/lib is definitely in LD_LIBRARY_PATH when loading
> the GCC 11.2 module.
> 
> If using the /path/to/R-4.2.2/etc/ldpaths where would I put the
> correct path? I would've hoped the configure/make process would've
> found this automatically.
Interesting that bin/R works but calling bin/R from what amounts to
`make -C src/library install` doesn't. I think that the failing step is
the following command:

@$(ECHO) "  building HTML index ..."
@$(ECHO) "utils:::make.packages.html(.Library, verbose=FALSE,
docdir=\"$(DESTDIR)${rdocdir}\")" | \
 R_DEFAULT_PACKAGES=NULL LC_ALL=C ${R_EXE} >/dev/null

I don't see a reason for Make to lose the LD_LIBRARY_PATH when
launching R like this.

Can you see the failing step in the `make -d install` output? There
will be a lot of text, most of it unrelated, unfortunately.

If something loses the LD_LIBRARY_PATH environment variable on the way
from Make to R, hard-coding it in etc/ldpaths should work around the
problem, but it may be hard to find out what went wrong.

If you run `mkfifo /tmp/1` and insert `read foo </tmp/1` into
etc/ldpaths, it should hang until you write something into /tmp/1.
Using this, it should be possible to inspect /proc/${pid}/environ for
the whole process tree from the parent Make process to the shell that's
about to launch R. Can you find the first process that lacks the proper
LD_LIBRARY_PATH environment variable?

-- 
Best regards,
Ivan

On Sun, Nov 20, 2022 at 2:26 PM Ivan Krylov <krylov.r00t at gmail.com>
wrote:
> On Sun, 20 Nov 2022 14:03:34 -0500
> Rob Kudyba <rk3199 at columbia.edu> wrote:
>
> > /path/to/gcc-11.2/lib is definitely in LD_LIBRARY_PATH when loading
> > the GCC 11.2 module.
> >
> > If using the /path/to/R-4.2.2/etc/ldpaths where would I put the
> > correct path? I would've hoped the configure/make process
would've
> > found this automatically.
>
> Interesting that bin/R works but calling bin/R from what amounts to
> `make -C src/library install` doesn't. I think that the failing step is
> the following command:
>
> @$(ECHO) "  building HTML index ..."
> @$(ECHO) "utils:::make.packages.html(.Library, verbose=FALSE,
> docdir=\"$(DESTDIR)${rdocdir}\")" | \
>  R_DEFAULT_PACKAGES=NULL LC_ALL=C ${R_EXE} >/dev/null
>
> I don't see a reason for Make to lose the LD_LIBRARY_PATH when
> launching R like this.
>
> Can you see the failing step in the `make -d install` output? There
> will be a lot of text, most of it unrelated, unfortunately.
>
I've uploaded the output to https://easyupload.io/scuujn
The gcc-related part has this:
  Considering target file `install-main'.
   File `install-main' does not exist.
    Considering target file `install-Rscript'.
     File `install-Rscript' does not exist.
     Finished prerequisites of target file `install-Rscript'.
    Must remake target `install-Rscript'.
Invoking recipe from Makefile:71 to update target `install-Rscript'.
make[2]: Entering directory `/path/to/me/R-4.2.2/src/unix'
gcc -I. -I../../src/include -I../../src/include  -I/usr/local/include
-DHAVE_CONFIG_H    -g -O2  -L/usr/local/lib64
-DR_HOME='"/path/to/R-4.2/lib64/R"' \
  -o Rscript ./Rscript.c
Putting child 0x65bc00 (install-Rscript) PID 240122 on the chain.
Live child 0x65bc00 (install-Rscript) PID 240122

If something loses the LD_LIBRARY_PATH environment variable on the
way
> from Make to R, hard-coding it in etc/ldpaths should work around the
> problem, but it may be hard to find out what went wrong.
>
I edited the last line to be:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/gcc-11.2/lib64 then make
install errored with:
/path/to/R-4.2.2/bin/exec/R: error while loading shared libraries:
libpcre2-8.so.0: cannot open shared object file: No such file or directory

So it appears that the same issue happens for PCRE! I then edited the line
to be:
export
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/gcc-11.2/lib64:/path/to/pcre2-10.35/lib

Then make install finished with:
Successfully remade target file `install'.

>
> If you run `mkfifo /tmp/1` and insert `read foo </tmp/1` into
> etc/ldpaths, it should hang until you write something into /tmp/1.
> Using this, it should be possible to inspect /proc/${pid}/environ for
> the whole process tree from the parent Make process to the shell that's
> about to launch R. Can you find the first process that lacks the proper
> LD_LIBRARY_PATH environment variable?
>
I ran the 1st command and then added the 'read' to the file. I'm not
following what should've happened? I didn't need to include the back
ticks,`, in the file, correct? Perhaps the debug from make install will
provide you more info?

Would this be considered a bug or enhancement?

Thanks for your help on this.

Rob

	[[alternative HTML version deleted]]

? Mon, 21 Nov 2022 10:19:36 -0500
Rob Kudyba <rk3199 at columbia.edu> ?????:
> I edited the last line to be:
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/gcc-11.2/lib64 then
> make install errored with:
> /path/to/R-4.2.2/bin/exec/R: error while loading shared libraries:
> libpcre2-8.so.0: cannot open shared object file: No such file or
> directory
> 
> So it appears that the same issue happens for PCRE! I then edited the
> line to be:
> export
>
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/path/to/gcc-11.2/lib64:/path/to/pcre2-10.35/lib
> 
> Then make install finished with:
> Successfully remade target file `install'.
Does the installed copy of R work?

Are you running make install as root? Programs starting as root are wary
of environment variables such as LD_LIBRARY_PATH and may ignore or
unset them while starting up (otherwise it would be trivial to load user
code into setuid root binaries and take over the system). You may have
to load the module again after you obtain the superuser privileges.
> I ran the 1st command and then added the 'read' to the file.
I'm not
> following what should've happened? I didn't need to include the
back
> ticks,`, in the file, correct?
Yes, I meant to omit the backticks.

What I'm suggesting to perform is Linux process debugging:

1) Make etc/ldpaths hang just before R is (unsuccessfully) launched by
adding an I/O operation that you can control.

2) While make install -> (lots of child processes) -> bin/R is hung,
run `top`, `htop`, or any other process manager to have a look at the
process tree. There should be your shell process, the Make process, and
a number of child shell processes. The grand-(grand-...)child should be
sleeping in I/O.

3) Using either your process manager or the command

 </proc/${pid}/environ tr '\0' '\n' | grep LD_LIBRARY_PATH

(substituting the PID instead of ${pid}), see which processes in the
tree you observed in (2) have the correct value of LD_LIBRARY_PATH and
which don't.

Once you know which process is responsible for losing LD_LIBRARY_PATH,
it should be possible to take a look at its command line, source code,
documentation to find out why it does that and whether it's possible to
prevent it from doing that.

"R Installation and Administration" [1] says that the R build process
remembers the library paths to put into LD_LIBRARY_PATH, but only if
they are specified in the LDFLAGS parameter, so if there's a set of
library paths that's not expected to be normally present in
LD_LIBRARY_PATH, perhaps it's best to mention them explicitly in the
./configure call.

-- 
Best regards,
Ivan

[1] https://cran.r-project.org/doc/manuals/R-admin.html