R devel - Dec 2016 - Upgrading a package to which other packages are LinkingTo

On 16 December 2016 at 08:20, Duncan Murdoch wrote:
| Perhaps the solution is to recommend that packages which export their 
| C-level entry points either guarantee them not to change or offer 
| (require?) version checks by user code.  So dplyr should start out by 
| saying "I'm using Rcpp interface 0.12.8".  If Rcpp has a new
version
| with a compatible interface, it replies "that's fine".  If Rcpp
has
| changed its interface, it says "Sorry, I don't support that any
more."

We try. But it's hard, and I'd argue, likely impossible.

For example I even added a "frozen" package [1] in the sources / unit
tests
to test for just this. In practice you just cannot hit every possible access
point of the (rich, in our case) API so the tests pass too often.

Which is why we relentlessly test against reverse-depends to _at least ensure
buildability_ from our releases.

As for seamless binary upgrade, I don't think in can work in practice.  Ask
Uwe one day we he rebuilds everything every time on Windows. And for what it
is worth, we essentially do the same in Debian.

Sometimes you just need to rebuild.  That may be the price of admission for
using the convenience of rich C++ interfaces.

Dirk

[1] https://github.com/RcppCore/Rcpp/tree/master/inst/unitTests/testRcppPackage


| Duncan Murdoch
| 
| >
| > Thanks.
| >
| >
| > Best regards
| >
| > Kirill
| >
| >
| > [1] https://github.com/hadley/dplyr/issues/2308#issuecomment-267495075
| > [2] https://travis-ci.org/krlmlr/pkg.upgrade.test#L589-L593
| > [3] https://travis-ci.org/krlmlr/pkg.upgrade.test#L619-L645
| > [4] https://travis-ci.org/krlmlr/pkg.upgrade.test#L671-L703
| >
| > ______________________________________________
| > R-devel at r-project.org mailing list
| > https://stat.ethz.ch/mailman/listinfo/r-devel
| >
| 
| ______________________________________________
| R-devel at r-project.org mailing list
| https://stat.ethz.ch/mailman/listinfo/r-devel

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org

On 16/12/2016 8:37 AM, Dirk Eddelbuettel wrote:
>
> On 16 December 2016 at 08:20, Duncan Murdoch wrote:
> | Perhaps the solution is to recommend that packages which export their
> | C-level entry points either guarantee them not to change or offer
> | (require?) version checks by user code.  So dplyr should start out by
> | saying "I'm using Rcpp interface 0.12.8".  If Rcpp has a
new version
> | with a compatible interface, it replies "that's fine".  If
Rcpp has
> | changed its interface, it says "Sorry, I don't support that any
more."
>
> We try. But it's hard, and I'd argue, likely impossible.
>
> For example I even added a "frozen" package [1] in the sources /
unit tests
> to test for just this. In practice you just cannot hit every possible
access
> point of the (rich, in our case) API so the tests pass too often.
>
> Which is why we relentlessly test against reverse-depends to _at least
ensure
> buildability_ from our releases.
>
> As for seamless binary upgrade, I don't think in can work in practice. 
Ask
> Uwe one day we he rebuilds everything every time on Windows. And for what
it
> is worth, we essentially do the same in Debian.
>
> Sometimes you just need to rebuild.  That may be the price of admission for
> using the convenience of rich C++ interfaces.
>
Okay, so would you say that Kirill's suggestion is not overkill?  Every 
time package B uses LinkingTo: A, R should assume it needs to rebuild B 
when A is updated?

Duncan Murdoch

On 16 December 2016 at 10:14, Duncan Murdoch wrote:
| On 16/12/2016 8:37 AM, Dirk Eddelbuettel wrote:
| >
| > On 16 December 2016 at 08:20, Duncan Murdoch wrote:
| > | Perhaps the solution is to recommend that packages which export their
| > | C-level entry points either guarantee them not to change or offer
| > | (require?) version checks by user code.  So dplyr should start out by
| > | saying "I'm using Rcpp interface 0.12.8".  If Rcpp has a
new version
| > | with a compatible interface, it replies "that's fine". 
If Rcpp has
| > | changed its interface, it says "Sorry, I don't support that
any more."
| >
| > We try. But it's hard, and I'd argue, likely impossible.
| >
| > For example I even added a "frozen" package [1] in the sources
/ unit tests
| > to test for just this. In practice you just cannot hit every possible
access
| > point of the (rich, in our case) API so the tests pass too often.
| >
| > Which is why we relentlessly test against reverse-depends to _at least
ensure
| > buildability_ from our releases.

I meant to also add:  "... against a large corpus of other packages."
The intent is to empirically answer this.

| > As for seamless binary upgrade, I don't think in can work in
practice.  Ask
| > Uwe one day we he rebuilds everything every time on Windows. And for what
it
| > is worth, we essentially do the same in Debian.
| >
| > Sometimes you just need to rebuild.  That may be the price of admission
for
| > using the convenience of rich C++ interfaces.
| >
| 
| Okay, so would you say that Kirill's suggestion is not overkill?  Every 
| time package B uses LinkingTo: A, R should assume it needs to rebuild B 
| when A is updated?

Based on my experience is a "halting problem" -- i.e. cannot know ex
ante.

So "every time" would be overkill to me.  Sometimes you know you must
recompile (but try to be very prudent with public-facing API).  Many times
you do not. It is hard to pin down.

At work we have a bunch of servers with Rcpp and many packages against them
(installed system-wide for all users). We _very really_ needs rebuild.  

Dirk

-- 
http://dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org