Hello,
I just subscribed to the list to join the discussion after being
blindsided by the change and reading Dirk Eddelbuettel's reply to my
bug report at https://bugs.debian.org/866768 .
As far as I can tell the advantages of site library are:
1. Saves disk space and a little bit of user time spent installing and
upgrading.
2. Other Debian package manages, like pip, default?to trying to install
to a site library.
However, it seems to me that the case for status quo ante is stronger,
even if the jarring behaviour (asking user whether to create a personal
directory, then failing to do so) is fixed:
1. Correct me if I am wrong, but wouldn't the change make the default
behaviour of R on Debian different from its default behaviour on
other distributions?
2. Site library has no benefit over user library on a single-user
system.
3. Making the site library writeable by default is a severe security
vulnerability, since it enables users to rewrite library code that
will be blindly executed by other users on the system.
4. Making the site library not-writeable by default means that users
relying on site library need to contact an administrator to get a
package installed or upgraded. This seems to me to largely defeat
the benefit of a site library for most systems, since most users
aren't so patient and would just figure out how to use a user
library.
5. The use case where defaulting to installing to site library is most
beneficial---a system shared by few trusted users---is rare compared
to the single-user and the many-untrusted-user systems.
6. There are workarounds, and, indeed, users can always use user
libraries, but the new setup puts the burden of tweaking the system
on the *least skilled* users, whereas the old setup Just Worked for
them (since R 2.5.0).
7. Even in Python and others, the user library takes precedence over
the site library (which, in turn, takes precedence over the dpkg-
installed library). This means that .libPaths() should have the user
library (if it exists) in first position, not as a fallback.
So, overall, I think the change does more harm than good. Am I missing anything?
Best Regards,
Pavel
--
Pavel Krivitsky
Lecturer in Statistics
National Institute of Applied Statistics Research Australia (NIASRA)
School of Mathematics and Applied Statistics | Building 39C Room 154
University of Wollongong NSW 2522 Australia
T +61 2 4221 3713
Web (NIASRA): http://niasra.uow.edu.au/index.html
Web (Personal): http://www.krivitsky.net/research
ORCID: 0000-0002-9101-3362
NOTICE: This email is intended for the addressee named and may contain
confidential information. If you are not the intended recipient, please
delete it and notify the sender. Please consider the environment before
printing this email.
Pavel,
Your tone does not exactly help in this discussion.
Briefly, I (like many other people) consider Linux and Unix to be
multi-user systems. You can argue as passionately for a default
installation in /usr/loca/lib/R/site-library as you did against. And
some of your arguments are just silly ("dangerous group": dude, it is
one 'sudo addgroup r-adm' [or anotther name...]) away).
I have used such settings (such as un-setting R_LIBS_USER or its
predecessors) for over a decade, it just works (if you give write
permissions). It clearly helps us at work because everybody sees by
the default the same packages. I have also spoken with different R
Core members and several find the default installation below $HOME and
in a versioned directory less than ideal as well. But it ensures
writeability. Which I cannot do easily from the package.
So maybe the change was too abrupt, and I think I may revert it. I
generally prefer for packagers like myself to not divert from upstream
unless they have good reasom or are unintrusive (and eg the added
tab-completion we have here is both). But leaving newbies without
installable directories is bad, as is possibly hiding existing
installations.
I am a little pressed for time (at useR!) and system (main server is
ill, as is backup machine) but I should get a 3.4.1-2 out.
I'd welcome other comments, for or against.
Dirk
--
Three out of two people have difficulties with fractions.
Hi,> I have used such settings (such as un-setting R_LIBS_USER or its > predecessors) for over a decade, it just works (if you give write > permissions). It clearly helps us at work because everybody sees by > the default the same packages. I have also spoken with different R > Core members and several find the default installation below $HOME and > in a versioned directory less than ideal as well. But it ensures > writeability. Which I cannot do easily from the package.Could the package make /usr/local/lib/R/site-library owned by a dedicated group, e.g. rlibs, so people could just adduser $USER rlibs? I do not see why R users should read logs, which is what the adm group was apparently made for. And then, this tip could be given when install.packages() fails, in addition to the option to create a user (but not version) specific library. Usually R libraries are not version specific, even though in the case of R 3.0.0 and 3.4.0 compatibility was interrupted.> So maybe the change was too abrupt, and I think I may revert it. I > generally prefer for packagers like myself to not divert from upstream > unless they have good reasom or are unintrusive (and eg the added > tab-completion we have here is both). But leaving newbies without > installable directories is bad, as is possibly hiding existing > installations.I also like the idea of multiuser libraries, and in general the idea is that the most current package versions play well together, and are better then earlier versions. If this should not be the case, this should be an exceptional situation, which may warrant a user specific library. So an alternative to reverting the change would be to keep your change, but to give better support for people that a) have user (and version) specific libraries that were created from within R in the past ("do you want to create a personal library instead") b) want to use personal libraries for some good reason via this list and maybe also via improved messages from R for the case that install.package() fails. Johannes
Hi, As comments are welcome I will give my two cents and a patch suggestion :-) 2017-07-06 10:42 GMT+02:00 Dirk Eddelbuettel <edd at debian.org>:> > > I have used such settings (such as un-setting R_LIBS_USER or its > predecessors) for over a decade, it just works (if you give write > permissions). It clearly helps us at work because everybody sees by > the default the same packages. I have also spoken with different R > Core members and several find the default installation below $HOME and > in a versioned directory less than ideal as well. But it ensures > writeability. Which I cannot do easily from the package. > > > So maybe the change was too abrupt, and I think I may revert it. I > generally prefer for packagers like myself to not divert from upstream > unless they have good reasom or are unintrusive (and eg the added > tab-completion we have here is both). But leaving newbies without > installable directories is bad, as is possibly hiding existing > installations.There are several entangled issues: 1. What should be the default places where packages are installed? 2. What should be the default places where packages are loaded from? 3. Ensuring that with the default R installation any user can install packages For 1., I believe packages should be installed at: - /usr/lib/R/library, if they are given by a .deb package (core R packages) - /usr/lib/R/site-library/, if they are given by a .deb package (other packages) - /usr/local/lib/R/site-library/, if the user has permissions - ~/R/x86_64-pc-linux-gnu-library/3.4, (or somewhere else under $HOME, otherwise) For 2., I believe packages should be loaded from: - ~/R/x86_64-pc-linux-gnu-library/3.4 - /usr/local/lib/R/site-library/ - /usr/lib/R/site-library/ - /usr/lib/R/library I believe with 3.4.1-1 Dirk tried to move towards that configuration, and for some reasons we ended in a situation where the R library in the HOME directory was not considered to write into and was not considered to load packages from. This is how R works: - When we use `library("package_name")`, the package_name is searched in the directories given by .libPaths(). - When we use `install.packages("package_name")`, by default the package is installed in the first element of .libPaths(). These two things make it very tricky to find a solution to 1, because: - We can't change .libPaths() as we would break with how packages are searched - We may try to change install.packages() so instead of trying to install to .libPaths()[1] by default it tries /usr/local/lib/R/site-library first. We don't know how many things may break because they rely on that assumption, we would need to test CRAN packages like checkpoint and packrat and worse we don't know if we break some custom setups from users. - We can't remove the home directory from .libPaths() because some users won't be able to install packages there anymore nor find their packages, unless they choose to do that in their Renviron file. Considering this, the safest approach is to remove the home directory approach from .libPaths() as Dirk tried to do. The main problem is that there the personal directory approach failed and there was not enough documentation in install.packages() to fix that. Maybe a help message in install.packages would have helped. I attach a DEP-3 compliant patch as a proposal. It works on my computer. I'm no expert so your review is very welcome. The patch does a small change in how install.packages works: If the package can't be installed into .libPaths()[1] (/usr/local/lib/R/site-library/), and if the user directory is NA (because it is not set anymore in /etc/Renviron) then: * the default user directory is used instead: ~/R/%p-library/%v * and "Would you like to use \\n~/R/%p-library/%v \nto in the future to install packages into?" is asked. if the answer is "yes", then it is appended to the ~/.Renviron file.> > I am a little pressed for time (at useR!) and system (main server is > ill, as is backup machine) but I should get a 3.4.1-2 out. >I hope this helps, I am missing the useR conference, I hope I can assist next year :-) Best, Sergio
Dirk,> Your tone does not exactly help in this discussion.My apologies. In my original bug report, I honestly thought that the change was unintentional.> Briefly, I (like many other people) consider Linux and Unix to be > multi-user systems.?I agree, but I think that an important part of the multi-user paradigm is to limit the ability of users to affect each other in non-explicit ways.> ?You can argue as passionately for a default > installation in /usr/loca/lib/R/site-library as you did against. And > some of your arguments are just silly ("dangerous group": dude, it is > one 'sudo addgroup r-adm' [or anotther name...]) away).The security issue would arise if R's default configuration allowed any user (or any user with a shell) to write to /usr/loca/lib/R/site- library. My understanding is that that's what would need to happen in order to make R package installation work out of the box with R_LIBS_USER unset by default. I guess it could be an installation-time configuration option in Debian.> I have used such settings (such as un-setting R_LIBS_USER or its > predecessors) for over a decade, it just works (if you give write > permissions). It clearly helps us at work because everybody sees by > the default the same packages.I understand, and I've used a variety of settings as well. In particular, even a decade later, I remember 2.5.0 coming out with R_LIBS_USER and a sensible default, and things Just Working on all of a sudden. (This was on a cluster, and during the 32- to 64-bit transition as well, so automatically loading libraries with the right architecture was a godsend, replacing a cumbersome workaround of my own.) Perhaps that's why I'm so passionate.> I have also spoken with different R > Core members and several find the default installation below $HOME > and > in a versioned directory less than ideal as well.??But it ensures > writeability. Which I cannot do easily from the package.I appreciate that it's less than ideal, but it also seems to me that status quo ante already fulfils the goal of encouraging the use of the site-library: it checks if the site-library is writeable, and only if it's not, offers to create a personal library as a fallback. As far as I can tell, knocking out that last step doesn't help those users who want to use the site library (since they already already know to make it readable, so they won't even see the offer to create a personal library), but it does hurt those users who want to use the personal library, by requiring them to take extra steps. Is the goal here to "nudge" users towards the site-library by making it harder to use the personal library?> So maybe the change was too abrupt, and I think I may revert it. I > generally prefer for packagers like myself to not divert from > upstream > unless they have good reasom or are unintrusive (and eg the added > tab-completion we have here is both).??But leaving newbies without > installable directories is bad, as is possibly hiding existing > installations.Part of my concern is that this change might propagate to Debian- derivatives like Ubuntu, hitting more naive users. Best Regards, Pavel P.S. Something else just occurred to me, but I don't have time to test it at the moment: R --vanilla ignores a lot of environment settings; which of the workarounds described still work? -- Pavel Krivitsky Lecturer in Statistics National Institute of Applied Statistics Research Australia (NIASRA) School of Mathematics and Applied Statistics | Building 39C Room 154 University of Wollongong NSW 2522 Australia T +61 2 4221 3713 Web (NIASRA): http://niasra.uow.edu.au/index.html Web (Personal): http://www.krivitsky.net/research ORCID: 0000-0002-9101-3362 NOTICE: This email is intended for the addressee named and may contain confidential information. If you are not the intended recipient, please delete it and notify the sender. Please consider the environment before printing this email.