Hi, As comments are welcome I will give my two cents and a patch suggestion :-) 2017-07-06 10:42 GMT+02:00 Dirk Eddelbuettel <edd at debian.org>:> > > I have used such settings (such as un-setting R_LIBS_USER or its > predecessors) for over a decade, it just works (if you give write > permissions). It clearly helps us at work because everybody sees by > the default the same packages. I have also spoken with different R > Core members and several find the default installation below $HOME and > in a versioned directory less than ideal as well. But it ensures > writeability. Which I cannot do easily from the package. > > > So maybe the change was too abrupt, and I think I may revert it. I > generally prefer for packagers like myself to not divert from upstream > unless they have good reasom or are unintrusive (and eg the added > tab-completion we have here is both). But leaving newbies without > installable directories is bad, as is possibly hiding existing > installations.There are several entangled issues: 1. What should be the default places where packages are installed? 2. What should be the default places where packages are loaded from? 3. Ensuring that with the default R installation any user can install packages For 1., I believe packages should be installed at: - /usr/lib/R/library, if they are given by a .deb package (core R packages) - /usr/lib/R/site-library/, if they are given by a .deb package (other packages) - /usr/local/lib/R/site-library/, if the user has permissions - ~/R/x86_64-pc-linux-gnu-library/3.4, (or somewhere else under $HOME, otherwise) For 2., I believe packages should be loaded from: - ~/R/x86_64-pc-linux-gnu-library/3.4 - /usr/local/lib/R/site-library/ - /usr/lib/R/site-library/ - /usr/lib/R/library I believe with 3.4.1-1 Dirk tried to move towards that configuration, and for some reasons we ended in a situation where the R library in the HOME directory was not considered to write into and was not considered to load packages from. This is how R works: - When we use `library("package_name")`, the package_name is searched in the directories given by .libPaths(). - When we use `install.packages("package_name")`, by default the package is installed in the first element of .libPaths(). These two things make it very tricky to find a solution to 1, because: - We can't change .libPaths() as we would break with how packages are searched - We may try to change install.packages() so instead of trying to install to .libPaths()[1] by default it tries /usr/local/lib/R/site-library first. We don't know how many things may break because they rely on that assumption, we would need to test CRAN packages like checkpoint and packrat and worse we don't know if we break some custom setups from users. - We can't remove the home directory from .libPaths() because some users won't be able to install packages there anymore nor find their packages, unless they choose to do that in their Renviron file. Considering this, the safest approach is to remove the home directory approach from .libPaths() as Dirk tried to do. The main problem is that there the personal directory approach failed and there was not enough documentation in install.packages() to fix that. Maybe a help message in install.packages would have helped. I attach a DEP-3 compliant patch as a proposal. It works on my computer. I'm no expert so your review is very welcome. The patch does a small change in how install.packages works: If the package can't be installed into .libPaths()[1] (/usr/local/lib/R/site-library/), and if the user directory is NA (because it is not set anymore in /etc/Renviron) then: * the default user directory is used instead: ~/R/%p-library/%v * and "Would you like to use \\n~/R/%p-library/%v \nto in the future to install packages into?" is asked. if the answer is "yes", then it is appended to the ~/.Renviron file.> > I am a little pressed for time (at useR!) and system (main server is > ill, as is backup machine) but I should get a 3.4.1-2 out. >I hope this helps, I am missing the useR conference, I hope I can assist next year :-) Best, Sergio
On Thu, Jul 06, 2017 at 02:02:52PM +0200, Sergio Oller wrote:> Hi, > > As comments are welcome I will give my two cents and a patch suggestion :-) > > 2017-07-06 10:42 GMT+02:00 Dirk Eddelbuettel <edd at debian.org>: > > > > > > I have used such settings (such as un-setting R_LIBS_USER or its > > predecessors) for over a decade, it just works (if you give write > > permissions). It clearly helps us at work because everybody sees by > > the default the same packages. I have also spoken with different R > > Core members and several find the default installation below $HOME and > > in a versioned directory less than ideal as well. But it ensures > > writeability. Which I cannot do easily from the package. > > > > > > So maybe the change was too abrupt, and I think I may revert it. I > > generally prefer for packagers like myself to not divert from upstream > > unless they have good reasom or are unintrusive (and eg the added > > tab-completion we have here is both). But leaving newbies without > > installable directories is bad, as is possibly hiding existing > > installations. > > There are several entangled issues: > 1. What should be the default places where packages are installed? > 2. What should be the default places where packages are loaded from?That's both the same -- what .libPaths() shows, and which can be set in several ways.> 3. Ensuring that with the default R installation any user can install > packagesYes, and that went wrong this time.> For 1., I believe packages should be installed at: > - /usr/lib/R/library, if they are given by a .deb package (core R packages) > - /usr/lib/R/site-library/, if they are given by a .deb package (other > packages) > - /usr/local/lib/R/site-library/, if the user has permissions > - ~/R/x86_64-pc-linux-gnu-library/3.4, (or somewhere else under $HOME, > otherwise)The first three (in reverse order) have been our default for 15 or years. The fourth is what R adds (even if a few people have reservations) and it provides a fallback.> For 2., I believe packages should be loaded from: > - ~/R/x86_64-pc-linux-gnu-library/3.4 > - /usr/local/lib/R/site-library/ > - /usr/lib/R/site-library/ > - /usr/lib/R/librarySee above. We have that.> I believe with 3.4.1-1 Dirk tried to move towards that configuration, and > for some reasons we ended in a situation where the R library in the HOME > directory was not considered to write into and was not considered to load > packages from. > > This is how R works: > > - When we use `library("package_name")`, the package_name is searched in > the directories given by .libPaths(). > > - When we use `install.packages("package_name")`, by default the package is > installed in the first element of .libPaths(). > > These two things make it very tricky to find a solution to 1, because: > > - We can't change .libPaths() as we would break with how packages are > searchedWe can. See help(Startup) and other places. But setting good default are hard.> - We may try to change install.packages() so instead of trying to install > to .libPaths()[1] by default it tries /usr/local/lib/R/site-library first. > We don't know how many things may break because they rely on that > assumption, we would need to test CRAN packages like checkpoint and packrat > and worse we don't know if we break some custom setups from users. > - We can't remove the home directory from .libPaths() because some users > won't be able to install packages there anymore nor find their packages, > unless they choose to do that in their Renviron file. > > Considering this, the safest approach is to remove the home directory > approach from .libPaths() as Dirk tried to do. The main problem is that > there the personal directory approach failed and there was not enough > documentation in install.packages() to fix that. > > Maybe a help message in install.packages would have helped. I attach a > DEP-3 compliant patch as a proposal. It works on my computer. I'm no expert > so your review is very welcome. The patch does a small change in how > install.packages works: > > If the package can't be installed into .libPaths()[1] > (/usr/local/lib/R/site-library/), and if the user directory is NA (because > it is not set anymore in /etc/Renviron) then: > * the default user directory is used instead: ~/R/%p-library/%v > * and "Would you like to use \\n~/R/%p-library/%v \nto in the future to > install packages into?" is asked. if the answer is "yes", then it is > appended to the ~/.Renviron file.Some of this way work, but we need some testing first. In the near term, reverting the patch may be easiest. Dirk> > > > > I am a little pressed for time (at useR!) and system (main server is > > ill, as is backup machine) but I should get a 3.4.1-2 out. > > > > I hope this helps, I am missing the useR conference, I hope I can assist > next year :-) > > Best, > > Sergio > _______________________________________________ > R-SIG-Debian mailing list > R-SIG-Debian at r-project.org > https://stat.ethz.ch/mailman/listinfo/r-sig-debian-- Three out of two people have difficulties with fractions.
On 7 July 2017 at 05:06, Dirk Eddelbuettel <edd at debian.org> wrote:> On Thu, Jul 06, 2017 at 02:02:52PM +0200, Sergio Oller wrote: > > Hi, > > > > As comments are welcome I will give my two cents and a patch suggestion > :-) > > > > 2017-07-06 10:42 GMT+02:00 Dirk Eddelbuettel <edd at debian.org>: > > > > > > > > > I have used such settings (such as un-setting R_LIBS_USER or its > > > predecessors) for over a decade, it just works (if you give write > > > permissions). It clearly helps us at work because everybody sees by > > > the default the same packages. I have also spoken with different R > > > Core members and several find the default installation below $HOME and > > > in a versioned directory less than ideal as well. But it ensures > > > writeability. Which I cannot do easily from the package. > > > > > > > > > So maybe the change was too abrupt, and I think I may revert it. I > > > generally prefer for packagers like myself to not divert from upstream > > > unless they have good reasom or are unintrusive (and eg the added > > > tab-completion we have here is both). But leaving newbies without > > > installable directories is bad, as is possibly hiding existing > > > installations. > > > > There are several entangled issues: > > 1. What should be the default places where packages are installed? > > 2. What should be the default places where packages are loaded from? > > That's both the same -- what .libPaths() shows, and which can be set > in several ways. > > > 3. Ensuring that with the default R installation any user can install > > packages > > Yes, and that went wrong this time. > > > For 1., I believe packages should be installed at: > > - /usr/lib/R/library, if they are given by a .deb package (core R > packages) > > - /usr/lib/R/site-library/, if they are given by a .deb package (other > > packages) > > - /usr/local/lib/R/site-library/, if the user has permissions > > - ~/R/x86_64-pc-linux-gnu-library/3.4, (or somewhere else under $HOME, > > otherwise) > > The first three (in reverse order) have been our default for 15 or years. > > The fourth is what R adds (even if a few people have reservations) and > it provides a fallback. > > > For 2., I believe packages should be loaded from: > > - ~/R/x86_64-pc-linux-gnu-library/3.4 > > - /usr/local/lib/R/site-library/ > > - /usr/lib/R/site-library/ > > - /usr/lib/R/library > > See above. We have that. > > > I believe with 3.4.1-1 Dirk tried to move towards that configuration, and > > for some reasons we ended in a situation where the R library in the HOME > > directory was not considered to write into and was not considered to load > > packages from. > > > > This is how R works: > > > > - When we use `library("package_name")`, the package_name is searched in > > the directories given by .libPaths(). > > > > - When we use `install.packages("package_name")`, by default the > package is > > installed in the first element of .libPaths(). >If the first element isn't writable, does `install.packages(.)` move to the next?> > > > These two things make it very tricky to find a solution to 1, because: > > > > - We can't change .libPaths() as we would break with how packages are > > searched > > We can. See help(Startup) and other places. But setting good default are > hard. >The problem of "finding a good default" affects any package that supports optional add-ons (TeX Live, Octave, Netbeans Java apps that use modules, etc). In some cases, add-ons require versions of dll's that conflict with standard packages. For TeX Live, we have distro packages that follow the distro policies and also the CTAN version that puts everything in one place. Red Hat has "software collections" which go in `/opt/rh` and `scl enable <collection> ...` that runs `...` with an environment tailored to support the chosen collection. I suspect there are use cases for R that would encounter fewer problems by creating a new directory tree to contain the complete R system and libraries. Maybe those are already being handled with virtualization.> > - We may try to change install.packages() so instead of trying to install > > to .libPaths()[1] by default it tries /usr/local/lib/R/site-library > first. > > We don't know how many things may break because they rely on that > > assumption, we would need to test CRAN packages like checkpoint and > packrat > > and worse we don't know if we break some custom setups from users. > > - We can't remove the home directory from .libPaths() because some users > > won't be able to install packages there anymore nor find their packages, > > unless they choose to do that in their Renviron file. > > > > Considering this, the safest approach is to remove the home directory > > approach from .libPaths() as Dirk tried to do. The main problem is that > > there the personal directory approach failed and there was not enough > > documentation in install.packages() to fix that. > > > > Maybe a help message in install.packages would have helped. I attach a > > DEP-3 compliant patch as a proposal. It works on my computer. I'm no > expert > > so your review is very welcome. The patch does a small change in how > > install.packages works: > > > > If the package can't be installed into .libPaths()[1] > > (/usr/local/lib/R/site-library/), and if the user directory is NA > (because > > it is not set anymore in /etc/Renviron) then: > > * the default user directory is used instead: ~/R/%p-library/%v > > * and "Would you like to use \\n~/R/%p-library/%v \nto in the future to > > install packages into?" is asked. if the answer is "yes", then it is > > appended to the ~/.Renviron file. > > Some of this way work, but we need some testing first. >Without clear policies in place, the future is different and testing may not solve the future problems. 1. make things as simple as possible for new or casual users. A big part of the value of R is in the number of users, so never do anything that might discourage new users. 2. diligence is needed to avoid barriers to installing complete R systems in a private location. This gets around things the R developers and packagers don't control, such as site policies that prevent users from creating new groups or installing libraries to `/usr/local/lib/R`. If lots of people to this an "R Live" package system might be useful.> In the near term, reverting the patch may be easiest. > > DirkTake the easy route -- I'm sure you have other ways to use your time and there doesn't seem to be a quick fix that works for everyone. -- George N. White III <aa056 at chebucto.ns.ca> Head of St. Margarets Bay, Nova Scotia [[alternative HTML version deleted]]