R devel - Dec 2016 - Upgrading a package to which other packages are LinkingTo

Hi


I'd like to suggest to make R more informative when a user updates a 
package A where there's at least one package B that has "LinkingTo:
A"
in its description.

To illustrate the problem, assume package A is updated so that its C/C++ 
header interface (in inst/include) is changed. For package B to pick up 
these changes, we need to reinstall package A. In extreme cases, if B 
also imports A and uses functions from A's shared library, failure to 
reinstall B may lead to all sorts of undefined behavior.

I've stumbled over this recently for A = Rcpp 0.12.8 and B = dplyr 0.5.0 
[1], with a bug fix available in Rcpp 0.12.8.2. Simply upgrading Rcpp to 
0.12.8.2 wasn't enough to propagate the bug fix to dplyr; we need to 
reinstall dplyr 0.5.0 too.

I've prepared an example with R-devel r71799. The initial configuration 
[2] is Rcpp 0.12.8 and dplyr 0.5.0. There is no warning from R after 
upgrading Rcpp to 0.12.8.2 [3], and no warning when loading the (now 
"broken") dplyr 0.5.0 linked against Rcpp 0.12.8 but importing Rcpp 
0.12.8.2 [4].

As a remedy, I'd like to suggest that upgrading Rcpp gives a warning 
about installed packages that are LinkingTo it [3], and that loading 
dplyr gives a warning that it has been built against a different version 
of Rcpp [4], just like the warning when packages are built against a 
different version of R.

Thanks.


Best regards

Kirill


[1] https://github.com/hadley/dplyr/issues/2308#issuecomment-267495075
[2] https://travis-ci.org/krlmlr/pkg.upgrade.test#L589-L593
[3] https://travis-ci.org/krlmlr/pkg.upgrade.test#L619-L645
[4] https://travis-ci.org/krlmlr/pkg.upgrade.test#L671-L703

I think there's one typo in your post which may confuse some readers; 
I've edited it inline below.  My comments on the suggestion are at the 
bottom of the message.


On 16/12/2016 5:35 AM, Kirill M?ller wrote:
> Hi
>
>
> I'd like to suggest to make R more informative when a user updates a
> package A where there's at least one package B that has
"LinkingTo: A"
> in its description.
>
> To illustrate the problem, assume package A is updated so that its C/C++
> header interface (in inst/include) is changed. For package B to pick up
> these changes, we need to reinstall package A.
This should be "reinstall package B", I think.

 > In extreme cases, if B
> also imports A and uses functions from A's shared library, failure to
> reinstall B may lead to all sorts of undefined behavior.
>
> I've stumbled over this recently for A = Rcpp 0.12.8 and B = dplyr
0.5.0
> [1], with a bug fix available in Rcpp 0.12.8.2. Simply upgrading Rcpp to
> 0.12.8.2 wasn't enough to propagate the bug fix to dplyr; we need to
> reinstall dplyr 0.5.0 too.
>
> I've prepared an example with R-devel r71799. The initial configuration
> [2] is Rcpp 0.12.8 and dplyr 0.5.0. There is no warning from R after
> upgrading Rcpp to 0.12.8.2 [3], and no warning when loading the (now
> "broken") dplyr 0.5.0 linked against Rcpp 0.12.8 but importing
Rcpp
> 0.12.8.2 [4].
>
> As a remedy, I'd like to suggest that upgrading Rcpp gives a warning
> about installed packages that are LinkingTo it [3], and that loading
> dplyr gives a warning that it has been built against a different version
> of Rcpp [4], just like the warning when packages are built against a
> different version of R.
I'd call it a bug that we allow the situation to exist without some sort 
of warning or error.

Your suggestion is one remedy, but might lead to too many warnings (or 
too many unnecessary recompiles).

An argument could be made that it's a bug in package A that it has 
updated its interface in a way that breaks packages that use it.

Perhaps the solution is to recommend that packages which export their 
C-level entry points either guarantee them not to change or offer 
(require?) version checks by user code.  So dplyr should start out by 
saying "I'm using Rcpp interface 0.12.8".  If Rcpp has a new
version
with a compatible interface, it replies "that's fine".  If Rcpp
has
changed its interface, it says "Sorry, I don't support that any
more."

Duncan Murdoch
>
> Thanks.
>
>
> Best regards
>
> Kirill
>
>
> [1] https://github.com/hadley/dplyr/issues/2308#issuecomment-267495075
> [2] https://travis-ci.org/krlmlr/pkg.upgrade.test#L589-L593
> [3] https://travis-ci.org/krlmlr/pkg.upgrade.test#L619-L645
> [4] https://travis-ci.org/krlmlr/pkg.upgrade.test#L671-L703
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

On 16 December 2016 at 08:20, Duncan Murdoch wrote:
| Perhaps the solution is to recommend that packages which export their 
| C-level entry points either guarantee them not to change or offer 
| (require?) version checks by user code.  So dplyr should start out by 
| saying "I'm using Rcpp interface 0.12.8".  If Rcpp has a new
version
| with a compatible interface, it replies "that's fine".  If Rcpp
has
| changed its interface, it says "Sorry, I don't support that any
more."

We try. But it's hard, and I'd argue, likely impossible.

For example I even added a "frozen" package [1] in the sources / unit
tests
to test for just this. In practice you just cannot hit every possible access
point of the (rich, in our case) API so the tests pass too often.

Which is why we relentlessly test against reverse-depends to _at least ensure
buildability_ from our releases.

As for seamless binary upgrade, I don't think in can work in practice.  Ask
Uwe one day we he rebuilds everything every time on Windows. And for what it
is worth, we essentially do the same in Debian.

Sometimes you just need to rebuild.  That may be the price of admission for
using the convenience of rich C++ interfaces.

Dirk

[1] https://github.com/RcppCore/Rcpp/tree/master/inst/unitTests/testRcppPackage


| Duncan Murdoch
| 
| >
| > Thanks.
| >
| >
| > Best regards
| >
| > Kirill
| >
| >
| > [1] https://github.com/hadley/dplyr/issues/2308#issuecomment-267495075
| > [2] https://travis-ci.org/krlmlr/pkg.upgrade.test#L589-L593
| > [3] https://travis-ci.org/krlmlr/pkg.upgrade.test#L619-L645
| > [4] https://travis-ci.org/krlmlr/pkg.upgrade.test#L671-L703
| >
| > ______________________________________________
| > R-devel at r-project.org mailing list
| > https://stat.ethz.ch/mailman/listinfo/r-devel
| >
| 
| ______________________________________________
| R-devel at r-project.org mailing list
| https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org