R devel - Dec 2014 - R build failure under gcc 4.9's link time optimization

R fails to build with visibility on and gcc 4.9's link time optimzation,
because
of its practice of building part of it as archive first. Specifically
it builds some bundled libraries as archive first, the symbols of which
are then entirely invisible in gcc 4.9.

The Matrix package also does this awful practice, with CHOLMOD.a COLAMD.a AMD.a
SuiteSparse_config.a .

One way of fixing R is:

diff --git a/src/main/Makefile.in b/src/main/Makefile.in
index 908b7ab..ce015b4 100644
--- a/src/main/Makefile.in
+++ b/src/main/Makefile.in
@@ -86,7 +86,7 @@ ALL_CPPFLAGS = $(ZLIB_CPPFLAGS) $(BZLIB_CPPFLAGS)
$(PCRE_CPPFLAGS) \
 R_ZLIBS = @BUILD_ZLIB_TRUE@ ../extra/zlib/libz.a
 R_BZLIBS = @BUILD_BZLIB_TRUE@ ../extra/bzip2/libbz2.a
 R_PCRE = @BUILD_PCRE_TRUE@ ../extra/pcre/libpcre.a
-R_TRE = @BUILD_TRE_TRUE@ ../extra/tre/libtre.a
+R_TRE = @BUILD_TRE_TRUE@ `ls ../extra/tre/*.o`
 R_XDR = @BUILD_XDR_TRUE@ ../extra/xdr/libxdr.a
 R_XZ = @BUILD_XZ_TRUE@  ../extra/xz/liblzma.a
 R_LIBINTL = @BUILD_LIBINTL_TRUE@ ../extra/intl/libintl.a

According the gcc changes doc, setting $AR is probably better. (untested).

Here is the relevant excerpt from https://gcc.gnu.org/gcc-4.9/changes.html

Link-time optimization (LTO) improvements: 
When using a linker plugin, compiling with the -flto option now generates slim
object files (.o) which only contain intermediate language representation for
LTO. Use -ffat-lto-objects to create files which contain additionally the object
code. To generate static libraries suitable for LTO processing, use gcc-ar and
gcc-ranlib; to list symbols from a slim object file use gcc-nm. (This requires
that ar, ranlib and nm have been compiled with plugin support.)

Switching on link time optimization in R gives very impressive speed gain:
the 1000 Genome-related code in snpMatrix is more than 10% faster.

There were 3 sets of changes, a bug fix, a command line option, and
a code change to make it happen.

AR=gcc-ar is needed by both R itself, Matrix, (and also R2SWF from my routine
upgrade).

The fix to https://bugs.r-project.org/bugzilla/show_bug.cgi?id=15011
was wrong and ineffective, and needs to be re-opened. I posted my fix there.

And the code change below - an alternative change is to modify
EXTRA_STATIC_LIBS a few lines below in "src/main/Makefile.in" to
start with "-Wl,--whole-archive" and ends with
"-Wl,--no-whole-archive".
(I filed this as https://gcc.gnu.org/bugzilla/show_bug.cgi?id=64390)

All three of these are gcc-specific; but then both R for windows and mac OS X
are
built with gcc. Together with linux, that covers all the major platforms. For
10% speed
gain, I'll let somebody else worry about machines that are not MS windows,
not mac OS X
nor linux.

--------------------------------------------
On Mon, 15/12/14, Hin-Tak Leung <htl10 at users.sourceforge.net> wrote:

 R fails to build with visibility on
 and gcc 4.9's link time optimzation, because
 of its practice of building part of it as archive first.
 Specifically
 it builds some bundled libraries as archive first, the
 symbols of which
 are then entirely invisible in gcc 4.9.

 The Matrix package also does this awful practice, with
 CHOLMOD.a COLAMD.a AMD.a SuiteSparse_config.a .

 One way of fixing R is:

 diff --git a/src/main/Makefile.in b/src/main/Makefile.in
 index 908b7ab..ce015b4 100644
 --- a/src/main/Makefile.in
 +++ b/src/main/Makefile.in
 @@ -86,7 +86,7 @@ ALL_CPPFLAGS = $(ZLIB_CPPFLAGS)
 $(BZLIB_CPPFLAGS) $(PCRE_CPPFLAGS) \
  R_ZLIBS = @BUILD_ZLIB_TRUE@ ../extra/zlib/libz.a
  R_BZLIBS = @BUILD_BZLIB_TRUE@ ../extra/bzip2/libbz2.a
  R_PCRE = @BUILD_PCRE_TRUE@ ../extra/pcre/libpcre.a
 -R_TRE = @BUILD_TRE_TRUE@ ../extra/tre/libtre.a
 +R_TRE = @BUILD_TRE_TRUE@ `ls ../extra/tre/*.o`
  R_XDR = @BUILD_XDR_TRUE@ ../extra/xdr/libxdr.a
  R_XZ = @BUILD_XZ_TRUE@? ../extra/xz/liblzma.a
  R_LIBINTL = @BUILD_LIBINTL_TRUE@ ../extra/intl/libintl.a

 According the gcc changes doc, setting $AR is probably
 better. (untested).

 Here is the relevant excerpt from https://gcc.gnu.org/gcc-4.9/changes.html

 Link-time optimization (LTO) improvements: 
 When using a linker plugin, compiling with the -flto option
 now generates slim object files (.o) which only contain
 intermediate language representation for LTO. Use
 -ffat-lto-objects to create files which contain additionally
 the object code. To generate static libraries suitable for
 LTO processing, use gcc-ar and gcc-ranlib; to list symbols
 from a slim object file use gcc-nm. (This requires that ar,
 ranlib and nm have been compiled with plugin support.)