Martin Maechler
2018-Jul-24 11:50 UTC
[Rd] Library lib.loc Option Ignored for Dependencies
>>>>> Benjamin Tyner >>>>> on Sat, 21 Jul 2018 13:42:43 -0400 writes:> Not sure whether it is the same issue as was raised here: > https://stat.ethz.ch/pipermail/r-devel/2010-October/058729.html > but in any case perhaps the problem could partially be remedied on line > 245 of src/library/base/R/library.R by passing the lib.loc to > .getRequiredPackages2() ...here is a patch (untested) > Index: src/library/base/R/library.R > ================================================================== > --- src/library/base/R/library.R??? (revision 74997) > +++ src/library/base/R/library.R??? (working copy) > @@ -242,7 +242,7 @@ > ???????????????????? pos <- 2 > ???????????????? } else pos <- npos > ???????????? } > -??????????? .getRequiredPackages2(pkgInfo, quietly = quietly) > +??????????? .getRequiredPackages2(pkgInfo, lib.loc = lib.loc, quietly = quietly) > ???????????? deps <- unique(names(pkgInfo$Depends)) > ???????????? ## If the namespace mechanism is available and the package This - directly - fails even more miserably e.g. in my own setup when I have dependency to my package. But it seems a good idea to use 'lib.loc', and safer and probably better than the current code maybe to use .getRequiredPackages2(pkgInfo, lib.loc = c(lib.loc, .libPaths()), quietly = quietly) instead of the current code which uses lib.loc = NULL equivalently to lib.loc = .libPaths() Others / ideas? Reproducible examples with small fake packages? Martin > On 07/21/2018 12:34 PM, Martin Maechler wrote: >>>>>>> Benjamin Tyner >>>>>>> on Fri, 20 Jul 2018 19:42:09 -0400 writes: >> > Here's a trick/workaround; if lib.loc is the path to your >> > library, then prior to calling library(), >> >> >> environment(.libPaths)$.lib.loc <- lib.loc >> >> Well, that is quite a "trick" -- and potentially a pretty >> dangerous one, not intended when making .libPaths a closure .... >> >> >> I do think that there is a problem with R's dealing of R_LIBS >> and other libPaths settings, notably when checking packages and >> within that recompiling vignettes etc, where the R process >> starts new versions of R via system() / system2() and then gets >> to wrong .libPaths() settings, >> and I personally would be very happy if we got reprex'es with >> small fake packages -- possibly only easily reproducible on >> unix-alikes ... so we could address this as a bug (or more than >> one) to be fixed. >> >> Notably with the 3.4.x --> 3.5.0 transition and my/our tendency >> of having quite a few paths in R_LIBS / lib.loc / ... I've been >> bitten by problems when the wrong version of package was taken >> from the wrong library path .... >> >> Martin >> >> >> >> ------------------------------------------------------------------------ >> >> Good day, >> >> >> >> If there's a library folder of the latest R packages and >> >> a particular package from it is loaded using the lib.loc >> >> option, the dependencies of that package are still >> >> attempted to be loaded from another folder of older >> >> packages specified by R_LIBS, which may cause errors >> >> about version requirements not being met. The >> >> documentation of the library function doesn't explain >> >> what the intended result is in such a case, but it could >> >> reasonably be expected that R would also load the >> >> dependencies from the user-specified lib.loc folder. >> >> >> >> -------------------------------------- >> >> Dario Strbenac University of Sydney Camperdown NSW 2050 >> >> Australia
On 07/24/2018 07:50 AM, Martin Maechler wrote:>>>>>> Benjamin Tyner >>>>>> on Sat, 21 Jul 2018 13:42:43 -0400 writes: > > Not sure whether it is the same issue as was raised here: > > https://stat.ethz.ch/pipermail/r-devel/2010-October/058729.html > > > but in any case perhaps the problem could partially be remedied on line > > 245 of src/library/base/R/library.R by passing the lib.loc to > > .getRequiredPackages2() ...here is a patch (untested) > > > Index: src/library/base/R/library.R > > ==================================================================> > --- src/library/base/R/library.R??? (revision 74997) > > +++ src/library/base/R/library.R??? (working copy) > > @@ -242,7 +242,7 @@ > > ???????????????????? pos <- 2 > > ???????????????? } else pos <- npos > > ???????????? } > > -??????????? .getRequiredPackages2(pkgInfo, quietly = quietly) > > +??????????? .getRequiredPackages2(pkgInfo, lib.loc = lib.loc, quietly = quietly) > > ???????????? deps <- unique(names(pkgInfo$Depends)) > > > ???????????? ## If the namespace mechanism is available and the package > > This - directly - fails even more miserably e.g. in my own setup > when I have dependency to my package. > > But it seems a good idea to use 'lib.loc', and safer and > probably better than the current code maybe to use > > .getRequiredPackages2(pkgInfo, > lib.loc = c(lib.loc, .libPaths()), > quietly = quietly) > > instead of the current code which uses lib.loc = NULL > equivalently to lib.loc = .libPaths()Seems reasonable to me.> > Others / ideas? > Reproducible examples with small fake packages?Or how about an example with a "real" CRAN package with just one dependency: ??? > dir.create("~/lib") ??? > list.files("~/lib") ??? character(0) ??? > install.packages("spam", lib = "~/lib", dependencies = TRUE) ??? > list.files("~/lib") ??? [1] "dotCall64" "spam" ??? > library(spam, lib.loc = "~/lib") ??? Error: package ?dotCall64? required by ?spam? could not be found> > Martin > > > On 07/21/2018 12:34 PM, Martin Maechler wrote: > >>>>>>> Benjamin Tyner > >>>>>>> on Fri, 20 Jul 2018 19:42:09 -0400 writes: > >> > Here's a trick/workaround; if lib.loc is the path to your > >> > library, then prior to calling library(), > >> > >> >> environment(.libPaths)$.lib.loc <- lib.loc > >> > >> Well, that is quite a "trick" -- and potentially a pretty > >> dangerous one, not intended when making .libPaths a closure .... > >> > >> > >> I do think that there is a problem with R's dealing of R_LIBS > >> and other libPaths settings, notably when checking packages and > >> within that recompiling vignettes etc, where the R process > >> starts new versions of R via system() / system2() and then gets > >> to wrong .libPaths() settings, > >> and I personally would be very happy if we got reprex'es with > >> small fake packages -- possibly only easily reproducible on > >> unix-alikes ... so we could address this as a bug (or more than > >> one) to be fixed. > >> > >> Notably with the 3.4.x --> 3.5.0 transition and my/our tendency > >> of having quite a few paths in R_LIBS / lib.loc / ... I've been > >> bitten by problems when the wrong version of package was taken > >> from the wrong library path .... > >> > >> Martin > >> > >> > >> >> ------------------------------------------------------------------------ > >> >> Good day, > >> >> > >> >> If there's a library folder of the latest R packages and > >> >> a particular package from it is loaded using the lib.loc > >> >> option, the dependencies of that package are still > >> >> attempted to be loaded from another folder of older > >> >> packages specified by R_LIBS, which may cause errors > >> >> about version requirements not being met. The > >> >> documentation of the library function doesn't explain > >> >> what the intended result is in such a case, but it could > >> >> reasonably be expected that R would also load the > >> >> dependencies from the user-specified lib.loc folder. > >> >> > >> >> -------------------------------------- > >> >> Dario Strbenac University of Sydney Camperdown NSW 2050 > >> >> Australia
Good day, A self-contained example which reproduces the problem is packagesFolder <- tempdir() latestFolder <- file.path(packagesFolder, "latest") dir.create(latestFolder) devtools::create(file.path(latestFolder, "statistics"), description = list(Version = "1.1.0", Depends = "extras (>= 1.0.5)")) devtools::create(file.path(packagesFolder, "extras"), description = list(Version = "1.0.0")) devtools::create(file.path(latestFolder, "extras"), description = list(Version = "1.1.0")) latestInstall <- file.path(.libPaths()[1], "latest") dir.create(latestInstall) install.packages(file.path(packagesFolder, "extras"), type = "source", repos = NULL) install.packages(file.path(latestFolder, "extras"), type = "source", repos = NULL, lib = latestInstall) install.packages(file.path(latestFolder, "statistics"), type = "source", repos = NULL, lib = latestInstall)> library(statistics, lib.loc = latestInstall)Error: package ?extras? 1.0.0 was found, but >= 1.0.5 is required by ?statistics? The latest version of the statistics package is loaded, but the directory of latest packages is ignored by R when it considers the package dependency. -------------------------------------- Dario Strbenac University of Sydney Camperdown NSW 2050 Australia
On 24.07.2018 23:37, Benjamin Tyner wrote:> > > On 07/24/2018 07:50 AM, Martin Maechler wrote: >>>>>>> Benjamin Tyner >>>>>>> ???? on Sat, 21 Jul 2018 13:42:43 -0400 writes: >> ???? > Not sure whether it is the same issue as was raised here: >> ???? > https://stat.ethz.ch/pipermail/r-devel/2010-October/058729.html >> >> ???? > but in any case perhaps the problem could partially be remedied >> on line >> ???? > 245 of src/library/base/R/library.R by passing the lib.loc to >> ???? > .getRequiredPackages2() ...here is a patch (untested) >> >> ???? > Index: src/library/base/R/library.R >> ???? > >> ==================================================================>> ???? > --- src/library/base/R/library.R??? (revision 74997) >> ???? > +++ src/library/base/R/library.R??? (working copy) >> ???? > @@ -242,7 +242,7 @@ >> ???? > ???????????????????? pos <- 2 >> ???? > ???????????????? } else pos <- npos >> ???? > ???????????? } >> ???? > -??????????? .getRequiredPackages2(pkgInfo, quietly = quietly) >> ???? > +??????????? .getRequiredPackages2(pkgInfo, lib.loc = lib.loc, >> quietly = quietly) >> ???? > ???????????? deps <- unique(names(pkgInfo$Depends)) >> >> ???? > ???????????? ## If the namespace mechanism is available and the >> package >> >> This - directly - fails even more miserably e.g. in my own setup >> when I have dependency to my package. >> >> But it seems a good idea to use 'lib.loc', and safer and >> probably better than the current code maybe to use >> >> ?? .getRequiredPackages2(pkgInfo, >> ??????????? lib.loc = c(lib.loc, .libPaths()), >> ??????????? quietly = quietly) >> >> instead of the current code which uses lib.loc = NULL >> equivalently to?? lib.loc = .libPaths() > Seems reasonable to me. >> >> Others / ideas? >> Reproducible examples with small fake packages? > Or how about an example with a "real" CRAN package with just one > dependency: > > ??? > dir.create("~/lib") > ??? > list.files("~/lib") > ??? character(0) > ??? > install.packages("spam", lib = "~/lib", dependencies = TRUE) > ??? > list.files("~/lib") > ??? [1] "dotCall64" "spam" > ??? > library(spam, lib.loc = "~/lib") > ??? Error: package ?dotCall64? required by ?spam? could not be foundNot sure if this is a bug: where is it documented that this works not as above (which is what I would expect)? For development puposes, you may want to try some devekopment version of one package agaiunst release versions of the others. This is the purpose of the above: take spam from ~lib and the other from .libPath(). Best, Uwe Ligges>> >> Martin >> >> ???? > On 07/21/2018 12:34 PM, Martin Maechler wrote: >> ???? >>>>>>> Benjamin Tyner >> ???? >>>>>>> on Fri, 20 Jul 2018 19:42:09 -0400 writes: >> ???? >> > Here's a trick/workaround; if lib.loc is the path to your >> ???? >> > library, then prior to calling library(), >> ???? >> >> ???? >> >> environment(.libPaths)$.lib.loc <- lib.loc >> ???? >> >> ???? >> Well, that is quite a "trick"? -- and potentially a pretty >> ???? >> dangerous one, not intended when making .libPaths a closure .... >> ???? >> >> ???? >> >> ???? >> I do think that there is a problem with R's dealing of R_LIBS >> ???? >> and other libPaths settings, notably when checking packages and >> ???? >> within that recompiling vignettes etc, where the R process >> ???? >> starts new versions of R via system() / system2() and then gets >> ???? >> to wrong .libPaths() settings, >> ???? >> and I personally would be very happy if we got reprex'es with >> ???? >> small fake packages -- possibly only easily reproducible on >> ???? >> unix-alikes ... so we could address this as a bug (or more than >> ???? >> one) to be fixed. >> ???? >> >> ???? >> Notably with the 3.4.x --> 3.5.0 transition and my/our tendency >> ???? >> of having quite a few paths in R_LIBS / lib.loc / ... I've been >> ???? >> bitten by problems when the wrong version of package was taken >> ???? >> from the wrong library path .... >> ???? >> >> ???? >> Martin >> ???? >> >> ???? >> >> ???? >> >> >> ------------------------------------------------------------------------ >> ???? >> >> Good day, >> ???? >> >> >> ???? >> >> If there's a library folder of the latest R packages and >> ???? >> >> a particular package from it is loaded using the lib.loc >> ???? >> >> option, the dependencies of that package are still >> ???? >> >> attempted to be loaded from another folder of older >> ???? >> >> packages specified by R_LIBS, which may cause errors >> ???? >> >> about version requirements not being met. The >> ???? >> >> documentation of the library function doesn't explain >> ???? >> >> what the intended result is in such a case, but it could >> ???? >> >> reasonably be expected that R would also load the >> ???? >> >> dependencies from the user-specified lib.loc folder. >> ???? >> >> >> ???? >> >> -------------------------------------- >> ???? >> >> Dario Strbenac University of Sydney Camperdown NSW 2050 >> ???? >> >> Australia > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
On 25.07.2018 04:00, Dario Strbenac wrote:> Good day, > > A self-contained example which reproduces the problem is > > packagesFolder <- tempdir() > latestFolder <- file.path(packagesFolder, "latest") > dir.create(latestFolder) > > devtools::create(file.path(latestFolder, "statistics"), description = list(Version = "1.1.0", Depends = "extras (>= 1.0.5)")) > devtools::create(file.path(packagesFolder, "extras"), description = list(Version = "1.0.0")) > devtools::create(file.path(latestFolder, "extras"), description = list(Version = "1.1.0")) > > latestInstall <- file.path(.libPaths()[1], "latest") > dir.create(latestInstall) > install.packages(file.path(packagesFolder, "extras"), type = "source", repos = NULL) > install.packages(file.path(latestFolder, "extras"), type = "source", repos = NULL, lib = latestInstall) > install.packages(file.path(latestFolder, "statistics"), type = "source", repos = NULL, lib = latestInstall) > >> library(statistics, lib.loc = latestInstall) > Error: package ?extras? 1.0.0 was found, but >= 1.0.5 is required by ?statistics? > > The latest version of the statistics package is loaded, but the directory of latest packages is ignored by R when it considers the package dependency.See my other mail, I think this is epctected, why don't youi set the .libPaths() correctly? Nor is this self contained as we need some extra package to try your example... Best, Uwe Ligges> -------------------------------------- > Dario Strbenac > University of Sydney > Camperdown NSW 2050 > Australia > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >