thr3ads.net - R devel - [Rd] R CMD INSTALL configure.args and CC customization [Sep 2011]

If this information is useful, please help other people find it:
Share via:

Michael Spiegel

2011-Sep-09 18:40 UTC

[Rd] R CMD INSTALL configure.args and CC customization

I am running into the following issue that has been previously
reported on the R-devel mailing list. The short version is that I'm
writing a package for MPI, and I'd like to change CC and SHLIB_LD to
"mpicc". Trying to change them in Makevars.in has no effect, because
the values are clobbered by /etc/R/Makeconf. Will the following
changes to Makeconf.in introduce any problems?

--Michael


Subject:	[Rd] R CMD INSTALL configure.args and CC customization	
From:	Paul Johnson (paul... at gmail.com)
Date:	Jan 20, 2010 11:30:44 am
List:	org.r-project.r-devel
Hi, everybody.

I hope the new year is treating you well.

In a Rocks Cluster Linux system (that's Centos 5.2 based), I have
built R-2.10 and it runs well.

While fiddling with MPI libraries and R packages, I've noticed a few
little wrinkles.

This comes as no surprise to the veteran R programmers, but let me
share to the new guys the following gem:

There is a difference between . and -

Simple, and yet so vital!

install.packages inside R takes a period between "configure" and
"args", as in "configure.args=", while in the shell, the
same option
specified to R CMD INSTALL wants a dash, as in "--configure-args=".
The "period" versus the "hyphen" was  killing me all
morning!

When building Rmpi, I had to specify configure options to let R know
which MPI library to use. I *tried* to avoid that by forcing the
compiler to be "mpicc", not "gcc".   I believe that if one
uses mpicc,
then the compiler really is gcc, but it is initiated with the INC and
LIB dirs set so that the configure  can find what it needs. But gcc
seems to be deeply glued to R installation of packages.

I find various posts in the r-help list in which people claim this should work:

CC=mpicc R CMD INSTALL ...

but it doesn't make a difference for me. The compiler that gets used
is gcc, no matter how many different ways I try to say it should be
mpicc:

$ CC=mpicc R CMD INSTALL
--configure-args=--with-mpi=/usr/lib64/openmpi/1.2.7-gcc
--configure-vars=CC=mpicc Rmpi_0.5-8.tar.gz
Password:
* installing to library '/share/apps/lib/R/site-library'
* installing *source* package 'Rmpi' ...
checking for gcc... gcc -m64 -std=gnu99
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -m64 -std=gnu99 accepts -g... yes
checking for gcc -m64 -std=gnu99 option to accept ISO C89... none needed
Trying to find mpi.h ...
Found in /usr/lib64/openmpi/1.2.7-gcc/include
Trying to find libmpi.so or libmpich.a ...
Found libmpi in /usr/lib64/openmpi/1.2.7-gcc/lib
checking for openpty in -lutil... yes
checking for main in -lpthread... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c RegQuery.c -o
RegQuery.o
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c Rmpi.c -o Rmpi.o
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c conversion.c -o
conversion.o
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c internal.c -o
internal.o
gcc -m64 -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o
Rmpi.o conversion.o internal.o -L/usr/lib64/openmpi/1.2.7-gcc/lib
-lmpi -lutil -lpthread -L/usr/lib64/R/lib -lR
** R
** demo
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
* DONE (Rmpi)
=====================
The same effect (or lack thereof) can be had with configure.vars in
install.packages:

install.packages("Rmpi",
configure.args="--with-mpi=/usr/lib64/openmpi/1.2.7-gcc",
configure.vars="CC=mpicc")
Warning in install.packages("Rmpi", configure.args
"--with-mpi=/usr/lib64/openmpi/1.2.7-gcc",  :
  argument 'lib' is missing: using
'/share/apps/lib/R/site-library'
--- Please select a CRAN mirror for use in this session ---
Loading Tcl/Tk interface ... done
trying URL
'http://cran.mirrors.hoobly.com/src/contrib/Rmpi_0.5-8.tar.gz'
Content type 'application/x-gzip' length 87528 bytes (85 Kb)
opened URL
=================================================downloaded 85 Kb

* installing *source* package 'Rmpi' ...
checking for gcc... gcc -m64 -std=gnu99
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc -m64 -std=gnu99 accepts -g... yes
checking for gcc -m64 -std=gnu99 option to accept ISO C89... none needed
Trying to find mpi.h ...
Found in /usr/lib64/openmpi/1.2.7-gcc/include
Trying to find libmpi.so or libmpich.a ...
Found libmpi in /usr/lib64/openmpi/1.2.7-gcc/lib
checking for openpty in -lutil... yes
checking for main in -lpthread... yes
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c RegQuery.c -o
RegQuery.o
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c Rmpi.c -o Rmpi.o
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c conversion.c -o
conversion.o
gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
-DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
-DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
-DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -c internal.c -o
internal.o
gcc -m64 -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o
Rmpi.o conversion.o internal.o -L/usr/lib64/openmpi/1.2.7-gcc/lib
-lmpi -lutil -lpthread -L/usr/lib64/R/lib -lR
** R
** demo
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices ...
* DONE (Rmpi)

-- 
Paul E. Johnson
Professor, Political Science
1541 Lilac Lane, Room 504
University of Kansas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Makeconf.in.patch
Type: text/x-patch
Size: 599 bytes
Desc: not available
URL:
<https://stat.ethz.ch/pipermail/r-devel/attachments/20110909/61152097/attachment.bin>

Timothy Bates

2011-Sep-12 17:29 UTC

head link

[Rd] R CMD INSTALL configure.args and CC customization

Hi Michael,
That change seems like a good saving for maintainers: cuts 281 lines in the Rmpi
package configure.ac file  down to just  2 lines in Makevars.in: CC=mpicc
SHLIB_LD=mpicc.

Hopefully the patch can get up? ?

entire patch to Makeconf.in (4 new lines):

--- Makeconf.in 2011-03-17 14:18:05.000000000 -0400
+++ Makeconf.in.new     2011-09-09 14:37:15.701103349 -0400
@@ -16,7 +16,9 @@
## next two for future support of cross-compiling, not actually used
BUILD_CC = @BUILD_CC@
BUILD_R = @BUILD_R@
+ifndef CC
CC = @CC@
+endif
CFLAGS = @CFLAGS@
CPICFLAGS = @CPICFLAGS@
CPPFLAGS = @CPPFLAGS@
@@ -88,7 +90,9 @@
SHLIB_EXT = @SHLIB_EXT@
SHLIB_CFLAGS = @SHLIB_CFLAGS@
SHLIB_FFLAGS = @SHLIB_FFLAGS@
+ifndef SHLIB_LD
SHLIB_LD = @SHLIB_LD@
+endif
SHLIB_LDFLAGS = @SHLIB_LDFLAGS@
SHLIB_LINK = $(SHLIB_LD) $(SHLIB_LDFLAGS) $(LDFLAGS)
STRIP_LIBS = @striplib@

On 9 Sep 2011, at 7:40 PM, Michael Spiegel wrote:
> I am running into the following issue that has been previously
> reported on the R-devel mailing list. The short version is that I'm
> writing a package for MPI, and I'd like to change CC and SHLIB_LD to
> "mpicc". Trying to change them in Makevars.in has no effect,
because
> the values are clobbered by /etc/R/Makeconf. Will the following
> changes to Makeconf.in introduce any problems?
> 
> --Michael
> 
> 
> Subject:	[Rd] R CMD INSTALL configure.args and CC customization	
> From:	Paul Johnson (paul... at gmail.com)
> Date:	Jan 20, 2010 11:30:44 am
> List:	org.r-project.r-devel
> Hi, everybody.
> 
> I hope the new year is treating you well.
> 
> In a Rocks Cluster Linux system (that's Centos 5.2 based), I have
> built R-2.10 and it runs well.
> 
> While fiddling with MPI libraries and R packages, I've noticed a few
> little wrinkles.
> 
> This comes as no surprise to the veteran R programmers, but let me
> share to the new guys the following gem:
> 
> There is a difference between . and -
> 
> Simple, and yet so vital!
> 
> install.packages inside R takes a period between "configure" and
> "args", as in "configure.args=", while in the shell,
the same option
> specified to R CMD INSTALL wants a dash, as in
"--configure-args=".
> The "period" versus the "hyphen" was  killing me all
morning!
> 
> When building Rmpi, I had to specify configure options to let R know
> which MPI library to use. I *tried* to avoid that by forcing the
> compiler to be "mpicc", not "gcc".   I believe that if
one uses mpicc,
> then the compiler really is gcc, but it is initiated with the INC and
> LIB dirs set so that the configure  can find what it needs. But gcc
> seems to be deeply glued to R installation of packages.
> 
> I find various posts in the r-help list in which people claim this should
work:
> 
> CC=mpicc R CMD INSTALL ...
> 
> but it doesn't make a difference for me. The compiler that gets used
> is gcc, no matter how many different ways I try to say it should be
> mpicc:
> 
> $ CC=mpicc R CMD INSTALL
> --configure-args=--with-mpi=/usr/lib64/openmpi/1.2.7-gcc
> --configure-vars=CC=mpicc Rmpi_0.5-8.tar.gz
> Password:
> * installing to library '/share/apps/lib/R/site-library'
> * installing *source* package 'Rmpi' ...
> checking for gcc... gcc -m64 -std=gnu99
> checking for C compiler default output file name... a.out
> checking whether the C compiler works... yes
> checking whether we are cross compiling... no
> checking for suffix of executables...
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc -m64 -std=gnu99 accepts -g... yes
> checking for gcc -m64 -std=gnu99 option to accept ISO C89... none needed
> Trying to find mpi.h ...
> Found in /usr/lib64/openmpi/1.2.7-gcc/include
> Trying to find libmpi.so or libmpich.a ...
> Found libmpi in /usr/lib64/openmpi/1.2.7-gcc/lib
> checking for openpty in -lutil... yes
> checking for main in -lpthread... yes
> configure: creating ./config.status
> config.status: creating src/Makevars
> ** libs
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c RegQuery.c -o
> RegQuery.o
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c Rmpi.c -o Rmpi.o
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c conversion.c -o
> conversion.o
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c internal.c -o
> internal.o
> gcc -m64 -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o
> Rmpi.o conversion.o internal.o -L/usr/lib64/openmpi/1.2.7-gcc/lib
> -lmpi -lutil -lpthread -L/usr/lib64/R/lib -lR
> ** R
> ** demo
> ** inst
> ** preparing package for lazy loading
> ** help
> *** installing help indices
> ** building package indices ...
> * DONE (Rmpi)
> =====================> 
> The same effect (or lack thereof) can be had with configure.vars in
> install.packages:
> 
> install.packages("Rmpi",
> configure.args="--with-mpi=/usr/lib64/openmpi/1.2.7-gcc",
> configure.vars="CC=mpicc")
> Warning in install.packages("Rmpi", configure.args >
"--with-mpi=/usr/lib64/openmpi/1.2.7-gcc",  :
>  argument 'lib' is missing: using
'/share/apps/lib/R/site-library'
> --- Please select a CRAN mirror for use in this session ---
> Loading Tcl/Tk interface ... done
> trying URL
'http://cran.mirrors.hoobly.com/src/contrib/Rmpi_0.5-8.tar.gz'
> Content type 'application/x-gzip' length 87528 bytes (85 Kb)
> opened URL
> =================================================> downloaded 85 Kb
> 
> * installing *source* package 'Rmpi' ...
> checking for gcc... gcc -m64 -std=gnu99
> checking for C compiler default output file name... a.out
> checking whether the C compiler works... yes
> checking whether we are cross compiling... no
> checking for suffix of executables...
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc -m64 -std=gnu99 accepts -g... yes
> checking for gcc -m64 -std=gnu99 option to accept ISO C89... none needed
> Trying to find mpi.h ...
> Found in /usr/lib64/openmpi/1.2.7-gcc/include
> Trying to find libmpi.so or libmpich.a ...
> Found libmpi in /usr/lib64/openmpi/1.2.7-gcc/lib
> checking for openpty in -lutil... yes
> checking for main in -lpthread... yes
> configure: creating ./config.status
> config.status: creating src/Makevars
> ** libs
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c RegQuery.c -o
> RegQuery.o
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c Rmpi.c -o Rmpi.o
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c conversion.c -o
> conversion.o
> gcc -m64 -std=gnu99 -I/usr/include/R -DPACKAGE_NAME=\"\"
> -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
-DPACKAGE_STRING=\"\"
> -DPACKAGE_BUGREPORT=\"\" -I/usr/lib64/openmpi/1.2.7-gcc/include
> -DMPI2 -DOPENMPI -I/usr/local/include    -fpic  -O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
> --param=ssp-buffer-size=4 -m64 -mtune=generic -c internal.c -o
> internal.o
> gcc -m64 -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o
> Rmpi.o conversion.o internal.o -L/usr/lib64/openmpi/1.2.7-gcc/lib
> -lmpi -lutil -lpthread -L/usr/lib64/R/lib -lR
> ** R
> ** demo
> ** inst
> ** preparing package for lazy loading
> ** help
> *** installing help indices
> ** building package indices ...
> * DONE (Rmpi)
> 
> -- 
> Paul E. Johnson
> Professor, Political Science
> 1541 Lilac Lane, Room 504
> University of Kansas
> <Makeconf.in.patch>______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

Apparently Analagous Threads

Search for more maybe matching threads

R devel - Sep 2011 - R CMD INSTALL configure.args and CC customization

[Rd] R CMD INSTALL configure.args and CC customization

[Rd] R CMD INSTALL configure.args and CC customization

Apparently Analagous Threads