R devel - Aug 2023 - Vectorize library() to improve handling of versioned deps [sprint project proposal]

Hi All,

This is a proposal for a project which could be worked on during the R
development Sprint at the end of this month; it was requested that we start
a discussion here to see what R-core's thoughts on it were before we
officially add it to the docket.


AFAIK, R officially supports both versioned dependencies (almost
exclusively of the >=version variety) and library paths with more than one
directory. Further, I believe if at least de facto supports the same
package being installed in different directories along the lib path. The
most common of these, I'd bet, would be different versions of the same
library being installed in a site library and in a user's personal library,
though that's not the only way this can happen.

The combination of these two features, however can give rise to
packages which are all correctly installed and all loadable individually,
but which must be loaded in a particular order when used together or the
loading of some of them will fail.

Consider the following dependency structure between packages

pkgA: pkgB (>= 0.5.0)

pkgC: pkgB (>= 0.6.0)

Consider the following multi-libpath setup:

~/pth1/: pkgA, pkg B [0.5.1]
~/pth2/: pkg C, pkg B [0.6.5]

And consider that we have the libpath c("~/pth1/",
"~/pth2").

If we do

library(pkgA)

Things will work great.

Same if we do

library(pkgC)

BUT, if we do

library(pkgA)
library(pkgC)

pkgC will not be able to be loaded, because an insufficient version of
pkgB will
already be loaded.

I propose that library be modified to be able to take a character vector of
package names, when it does, it performs the dependency calculations to
determine how all packages in the vector can be loaded (in the order they
appear). In the example above, this would mean that if we did

library(c("pkgA", "pkgC"))

It would determine that pkgB version 0.6.5 was needed (or alternatively,
that version 0.5.1 was insufficient) and use that *when loading the
dependencies of pkgA*.

The proposal issue for the sprint itself is here:
https://github.com/r-devel/r-project-sprint-2023/discussions/15

Thoughts?

~G

	[[alternative HTML version deleted]]

Hi Gabriel,

Nice idea! I have encountered this problem several times, while probably a
better management of libraries could avoided the issue this is an elegant
solution.

How would this proposal handle arguments mask.ok, include.only and exclude?
For example, in the (edge) case two packages export a function with the
same name but we want to mask one of them (pkgA::fun, pkgC::fun).
How would that work for instance in library(c("pkgA",
"pkgC"), mask.ok = c(
pkgC = "fun"))?
Would the name of the character be useful to decide which one is ok to
mask? (in the example I would expect the pkgC::fun be masked by pkgA::fun in
the inverse order or loading).

If this doesn't go ahead, perhaps a function could be implemented to check
these situations in the .libPaths (in a package if not in base) ?

Best,

Llu?s

On Mon, 7 Aug 2023 at 22:34, Gabriel Becker <gabembecker at gmail.com>
wrote:
> Hi All,
>
> This is a proposal for a project which could be worked on during the R
> development Sprint at the end of this month; it was requested that we start
> a discussion here to see what R-core's thoughts on it were before we
> officially add it to the docket.
>
>
> AFAIK, R officially supports both versioned dependencies (almost
> exclusively of the >=version variety) and library paths with more than
one
> directory. Further, I believe if at least de facto supports the same
> package being installed in different directories along the lib path. The
> most common of these, I'd bet, would be different versions of the same
> library being installed in a site library and in a user's personal
library,
> though that's not the only way this can happen.
>
> The combination of these two features, however can give rise to
> packages which are all correctly installed and all loadable individually,
> but which must be loaded in a particular order when used together or the
> loading of some of them will fail.
>
> Consider the following dependency structure between packages
>
> pkgA: pkgB (>= 0.5.0)
>
> pkgC: pkgB (>= 0.6.0)
>
> Consider the following multi-libpath setup:
>
> ~/pth1/: pkgA, pkg B [0.5.1]
> ~/pth2/: pkg C, pkg B [0.6.5]
>
> And consider that we have the libpath c("~/pth1/",
"~/pth2").
>
> If we do
>
> library(pkgA)
>
> Things will work great.
>
> Same if we do
>
> library(pkgC)
>
> BUT, if we do
>
> library(pkgA)
> library(pkgC)
>
> pkgC will not be able to be loaded, because an insufficient version of
> pkgB will
> already be loaded.
>
> I propose that library be modified to be able to take a character vector of
> package names, when it does, it performs the dependency calculations to
> determine how all packages in the vector can be loaded (in the order they
> appear). In the example above, this would mean that if we did
>
> library(c("pkgA", "pkgC"))
>
> It would determine that pkgB version 0.6.5 was needed (or alternatively,
> that version 0.5.1 was insufficient) and use that *when loading the
> dependencies of pkgA*.
>
> The proposal issue for the sprint itself is here:
> https://github.com/r-devel/r-project-sprint-2023/discussions/15
>
> Thoughts?
>
> ~G
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
	[[alternative HTML version deleted]]