R devel - Jul 2012 - Are R packages supposed to be "relocatable"? (avoiding BioConductor scripts...)

I've asked a question in the BioConductor list about package
management. My solution depends on your answer to the following
question.

Are installed R packages "relocatable"?

I mean relocatable in the same sense that files in a RedHat RPM file
might be "relocatable" after compiling
(http://www.rpm.org/max-rpm/ch-rpm-reloc.html).  This allows one to
build a package as the ordinary user and then the root user can take
the result and put it wherever it fits well in the path (say,
/usr/local/lib/R/library).

Here is why I asked.  I manage packages in our cluster and some users
have asked me to install some BioConductor packages.  BioConductor
install documents expect me to run a script as root that does a bunch
of changes, and I'm just unwilling to do that.  If I have to do
something as root, it has to be something more focused like running a
particular R command (install.packages(), for example).  It seems
insane to me that they expect (http://www.bioconductor.org/install) a
root user to run

source("http://bioconductor.org/biocLite.R")
biocLite("limma")

If I could do the installs as me with their script, and then copy the
install folder into the system, then it would be OK, if the packages
would work.

Or perhaps post-processing is required to fiddle some paths inside
package files?

pj
-- 
Paul E. Johnson
Professor, Political Science    Assoc. Director
1541 Lilac Lane, Room 504     Center for Research Methods
University of Kansas               University of Kansas
http://pj.freefaculty.org            http://quant.ku.edu

On Jul 19, 2012, at 12:13 PM, Paul Johnson wrote:
> I've asked a question in the BioConductor list about package
> management. My solution depends on your answer to the following
> question.
> 
> Are installed R packages "relocatable"?
> 
It depends on the platform and what you want to relocate (R or just the
packages) and on the package itself. For example on OS X absolute paths are
included in dylibs, so you can move packages that allow it and don't link to
other packages but you can't relocate R or packages that link to other
packages directly or libraries within the package (that should be rare, though).
Whether a package allows relocation depends whether it hard-codes any paths it
uses - typical packages use facilities like system.file to locate files so those
are relocatable, but if a package determines paths at configure time it may not
be relocatable.

Since you didn't even specify the platform you're interested in, this is
only a generic answer ...

> I mean relocatable in the same sense that files in a RedHat RPM file
> might be "relocatable" after compiling
> (http://www.rpm.org/max-rpm/ch-rpm-reloc.html).  This allows one to
> build a package as the ordinary user and then the root user can take
> the result and put it wherever it fits well in the path (say,
> /usr/local/lib/R/library).
> 
> Here is why I asked.  I manage packages in our cluster and some users
> have asked me to install some BioConductor packages.  BioConductor
> install documents expect me to run a script as root that does a bunch
> of changes, and I'm just unwilling to do that.  If I have to do
> something as root, it has to be something more focused like running a
> particular R command (install.packages(), for example).  It seems
> insane to me that they expect (http://www.bioconductor.org/install) a
> root user to run
> 
You can install BioC packages simply with install.packages() - there is no black
magic involved, it's a regular repository just like CRAN ...

Cheers,
Simon

> source("http://bioconductor.org/biocLite.R")
> biocLite("limma")
> 
> If I could do the installs as me with their script, and then copy the
> install folder into the system, then it would be OK, if the packages
> would work.
> 
> Or perhaps post-processing is required to fiddle some paths inside
> package files?
> 
> pj
> -- 
> Paul E. Johnson
> Professor, Political Science    Assoc. Director
> 1541 Lilac Lane, Room 504     Center for Research Methods
> University of Kansas               University of Kansas
> http://pj.freefaculty.org            http://quant.ku.edu
> 
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
> 
>

Short answer:

setRepositories() # chose what you want from BioC
install.packages()

works and always has. 'Smoke and mirrors' are not actually needed.

As for relocatability:

I am guessing you are talking about a Linux cluster ... but the details 
do matter.

The distribution of Windows binaries relies on installed packages being 
relocatable, so most packages are written to be.  There are a very few 
exceptions: the one most frequently encountered is packages linked to 
Rcpp, which on Linux put the path to Rcpp in their DSOs.  So you can 
relocate the package, but not the Rcpp installation it is linked to. 
(Not a problem on other OSes as it statically links elsewhere.)

A question you did not ask is if you can use differently located R: in 
most cases yes (I regularly run different versions of R with the same 
library trees).

We've relocated complete CRAN/BioC installations on Linux several times.

On 19/07/2012 17:13, Paul Johnson wrote:
> I've asked a question in the BioConductor list about package
> management. My solution depends on your answer to the following
> question.
>
> Are installed R packages "relocatable"?
>
> I mean relocatable in the same sense that files in a RedHat RPM file
> might be "relocatable" after compiling
> (http://www.rpm.org/max-rpm/ch-rpm-reloc.html).  This allows one to
> build a package as the ordinary user and then the root user can take
> the result and put it wherever it fits well in the path (say,
> /usr/local/lib/R/library).
>
> Here is why I asked.  I manage packages in our cluster and some users
> have asked me to install some BioConductor packages.  BioConductor
> install documents expect me to run a script as root that does a bunch
> of changes, and I'm just unwilling to do that.  If I have to do
> something as root, it has to be something more focused like running a
> particular R command (install.packages(), for example).  It seems
> insane to me that they expect (http://www.bioconductor.org/install) a
> root user to run
>
> source("http://bioconductor.org/biocLite.R")
> biocLite("limma")
>
> If I could do the installs as me with their script, and then copy the
> install folder into the system, then it would be OK, if the packages
> would work.
>
> Or perhaps post-processing is required to fiddle some paths inside
> package files?
>
> pj
>

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

On 07/19/2012 09:13 AM, Paul Johnson wrote:
> I've asked a question in the BioConductor list about package
> management. My solution depends on your answer to the following
> question.
>
> Are installed R packages "relocatable"?
>
> I mean relocatable in the same sense that files in a RedHat RPM file
> might be "relocatable" after compiling
> (http://www.rpm.org/max-rpm/ch-rpm-reloc.html).  This allows one to
> build a package as the ordinary user and then the root user can take
> the result and put it wherever it fits well in the path (say,
> /usr/local/lib/R/library).
>
> Here is why I asked.  I manage packages in our cluster and some users
> have asked me to install some BioConductor packages.  BioConductor
> install documents expect me to run a script as root that does a bunch
> of changes, and I'm just unwilling to do that.  If I have to do
> something as root, it has to be something more focused like running a
> particular R command (install.packages(), for example).  It seems
> insane to me that they expect (http://www.bioconductor.org/install) a
> root user to run
>
> source("http://bioconductor.org/biocLite.R")
> biocLite("limma")
For the record, this does four things.

(1) The 'source' command installs the BiocInstaller package appropriate 
for your (recent) version of R or, if this package is already installed, 
attaches it.

All remaining steps are executed by scripts in the BiocInstaller 
package, so after an initial invocation the 'source()' statement can be 
replaced by library(BiocInstaller).

(2) biocLite() invokes BiocInstaller::biocLite, which checks whether 
BiocInstaller is current and, if not, updates it. Disable this behaviour 
with the option suppressAutoUpdate=TRUE.

(3) The limma package and needed dependencies are installed, using 
install.packages() and the repositories appropriate for your version of 
R (paying attention to utils::setRepositories() and 
utils::chooseBioCmirror()). The 'lib' argument to biocLite is passed to 
install.packages() so the user has control over the directory in which 
packages will be installed.

(4) biocLite checks whether any packages are out-of-date relative to the 
version in the repository, and offers to update those that are out of 
date. Packages that are in directories that cannot be written are 
reported, via a warning and hence when biocLite finishes, as "installed 
directory not writeable, cannot update packages '...'" (this
message
apparently confused you; we will try for a better wording). Use 
suppressUpdates=TRUE to suppress this step, ask=FALSE to have old 
packages (in writeable directories) updated without being prompted; a 
lib.loc argument can be used to restrict the locations where 
old.packages() searches.

The arguments to biocLite are documented on its help page in the 
BiocInstaller package, ?biocLite.


For step (3), Prof Ripley & Dr Urbanek (and me in my response to your 
Bioconductor post, 
https://stat.ethz.ch/pipermail/bioconductor/2012-July/046999.html) are 
correct in stating that this can be done with a direct call to 
install.packages().

Step (4) arises because Bioconductor packages (a) follow a semi-annual 
release cycle, with a 'release' and 'devel' branch, and (b)
generally
have greater dependencies amongst one another. In the past, two very 
common scenarios leading to problems were that user installations mix 
packages from different releases of Bioconductor, and users report bugs 
that have already been addressed via bug fixes to the current release. 
So step (4) is trying to be pro-active, and based on traffic to the 
mailing list this has generally been successful. This synchronization is 
more important, and difficult for the user to manage directly, now that 
R has moved to an annual release cycle.

Steps (1) and (2) are needed to bootstrap the process. The relatively 
recent introduction of the BiocInstaller package allows us to document 
biocLite in a standard way, as well as provide the code in a more 
familiar fashion. Step (1) is partly a legacy (BiocInstaller is not in 
all releases of Bioconductor), and partly a simplification to avoid 
confusion (avoiding 'first time users should do A, return users should 
do B').


As a 'normal' user it is safe and appropriate to use biocLite to manage 
Bioconductor (and CRAN) packages. One takes a more cautious approach as 
'root', but I personally would be comfortable administering a widely 
used R / Bioconductor installation with biocLite.

Martin Morgan
Bioconductor

> If I could do the installs as me with their script, and then copy the
> install folder into the system, then it would be OK, if the packages
> would work.
>
> Or perhaps post-processing is required to fiddle some paths inside
> package files?
>
> pj
>

-- 
Computational Biology / Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N.
PO Box 19024 Seattle, WA 98109

Location: Arnold Building M1 B861
Phone: (206) 667-2793