R SIG Debian - Aug 2023 - pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04

Hi,

I am running a couple of shiny servers with several apps that are based 
around own CRAN packages. It worked stable for years, but due to the 
growing number of packages, the compile time for regular manual package 
installation and updates became inconvenient.

Therefore, I have been very happy to use pre-compiled packages from the 
c2d4u repository  -- and from now on testing r2u following a hint from 
Dirk Edelbuettel. @Dirk: installation worked like a charm.

The downside is, that the r-cran-* binaries are being installed 
automatically, together with the system update. So experienced repeated 
cases that crashed some of the shiny apps. The main reason with c2d4u 
were conflicts between the binary packages and other packages installed 
from sources. This may be less relevant with r2u, but in the interest of 
reproducibility and stability, I would prefer conservative manual updates.

My question: what is best practise, to disallow automatic updates for 
all (or part of) r-cran-* packages? Uncommenting the complete package 
source in the apt/sources.list.d/cd4u...list file? Fiddling around with 
/etc/apt/preferences ?

The ideal approach would be to put a plain textfile of all installed 
r-cran packages somewhere to the system, where packages that are to be 
upgraded (or oppositely: pinned) are just commented or outcommented.

Thanks in advance,

Thomas


Links:

https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu/

https://eddelbuettel.github.io/r2u

Hi Thomas,

Thanks a lot for bringing this over from r-package-devel!

As for your question, "there is a lot of meat on these bones" and I
really
think we should discuss this here.

Quick comments inline, more recap at the bottom.

On 9 August 2023 at 17:29, Thomas Petzoldt wrote:
| Hi,
| 
| I am running a couple of shiny servers with several apps that are based 
| around own CRAN packages. It worked stable for years, but due to the 
| growing number of packages, the compile time for regular manual package 
| installation and updates became inconvenient.
| 
| Therefore, I have been very happy to use pre-compiled packages from the 
| c2d4u repository  -- and from now on testing r2u following a hint from 
| Dirk Edelbuettel. @Dirk: installation worked like a charm.

Glad to hear!

It also generally works flawlessly (and repeatedly) for myself and others in
many context involving many packages.  Per the logs, we have shipped over 6.5
million packages in the ~ 15 months it has been up and generally do more than
10k each day now. 

| The downside is, that the r-cran-* binaries are being installed 
| automatically, together with the system update. So experienced repeated 
| cases that crashed some of the shiny apps. The main reason with c2d4u 
| were conflicts between the binary packages and other packages installed 
| from sources. This may be less relevant with r2u, but in the interest of 
| reproducibility and stability, I would prefer conservative manual updates.

Yes. If and when that happens, it is a bug!

The correct (but laborious way) is to file and report them.

I think the overall issue here is the mix and match from 'distro'
packages,
and added packages. Given the complete coverage, I prefer r2u for
consistency.  Which is why it is pinned higher at 700.

But one can have issue if because of ("semi-random") package renmaing
/
versioning / use of epoch [a leading version digit overriding] the distro
wins.

Eg we had a recent issue for r2u when following the Debian and the R 4.3.*
catchup, several packages (all around the graphics API that needed a rebuilt)
got force-rebuilt and ended up with, say, 1.2.3-2 beating 1.2.3-1.c2u2204.1.
So I quickly rebuilt that dozen. It was more of an issue for jammy (22.04)
than focal as the (distro) focal packages are generally way behind what CRAN
and hence r2u has.

The only really reliable way to fix this is by reporting the bugs the old
school way so that r2u can take care of it.  I would need help from users and
that is why I am asking for it here.
 
| My question: what is best practise, to disallow automatic updates for 
| all (or part of) r-cran-* packages? Uncommenting the complete package 
| source in the apt/sources.list.d/cd4u...list file? Fiddling around with 
| /etc/apt/preferences ?
| 
| The ideal approach would be to put a plain textfile of all installed 
| r-cran packages somewhere to the system, where packages that are to be 
| upgraded (or oppositely: pinned) are just commented or outcommented.

I mentioned in the initial short reply on the other list I can see several
options (now up to four)

1) use dpkg to put packages on 'hold'. They will not be touched or
updated. Reliable, manual, tedious for many. See 'man dpkg'.

2) look into 'apt pinning' via preferences and its wiki page at
https://wiki.debian.org/AptConfiguration etc   This may work, I have not had
the patience to work the details out.  But there are other value beyond the
'500 to 900' default range. This may work, and be quite elegant.

3) as you hint, just 'hide' the sources.list for r2u or c2d4u but it
still
risks updates from the distro versions :-/  Maybe 'easiest yet worst'.

4) maybe go the other way use eg the Rocker Project container for shiny and
run you sacred production in a shielded setting? This sounds like more work,
but it may be better. If it works for you depends.

Let's keep at this and see if we can make something 'already pretty
good'
even better!

Cheers,  Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | edd at debian.org