Two quite related and recent threads on R-devel:
[1] proposal for WRE: clarify that use of S4 classes implies use of superclasses
https://stat.ethz.ch/pipermail/r-devel/2023-July/082739.html
[2] Should package version requirements assume installation from sources?
https://stat.ethz.ch/pipermail/r-devel/2022-September/081971.html
Notably, if SeuratObject 4.1.3, the current version, had
importClassesFrom(Matrix, dgCMatrix,
## and the exported superclasses:
CsparseMatrix, Matrix, compMatrix, dMatrix,
dsparseMatrix, generalMatrix, sparseMatrix)
and not just
importClassesFrom(Matrix, dgCMatrix)
then the silent breakage due to missing symbols would not have occurred.
That is because _imported_ class definitions are retrieved at load time,
not at install time. SeuratObject 4.1.3 retrieves the CsparseMatrix
definition at install time, so indeed it is liable to become stale with
Matrix updates ...
Not to derail Dirk's proposal for "Breaks", which could certainly
have
uses beyond this particular example where a "correct" NAMESPACE
directive
would solve the problems ...
Mikael
On 2023-08-07 6:00 am, r-devel-request at r-project.org
wrote:> I would support this suggestion. There is a similar binary dependency chain
from
> Matrix ? TMB ? glmmTMB; we have implemented various checks to make users
aware
> that they need to reinstall from source, and to some extent we've tried
to push
> out synchronous updates (i.e., push an update of TMB to CRAN every time
Matrix
> changes, and an update of glmmTMB after that), but centralized machinery
for
> this would certainly be nice. FWIW some of the machinery is here:
>
https://github.com/glmmTMB/glmmTMB/blob/d9ee7b043281341429381faa19b5e53cb5a378c3/glmmTMB/R/utils.R#L209-L295
-- it relies on a Makefile rule that caches the current installed version of
TMB:
https://github.com/glmmTMB/glmmTMB/blob/d9ee7b043281341429381faa19b5e53cb5a378c3/glmmTMB/R/utils.R#L209-L295
cheers Ben Bolker On 2023-08-06 5:05 p.m., Dirk Eddelbuettel wrote:
>> CRAN, by relying on the powerful package management system that is part
of R,
>> provides an unparalleled framework for extending R with nearly 20k
packages.
>>
>> We recently encountered an issue that highlights a missing element in
the
>> otherwise outstanding package management system. So we would like to
start a
>> discussion about enhancing its feature set. As shown below, a mechanism
to
>> force reinstallation of packages may be needed.
>>
>> A demo is included below, it is reproducible in a container. We find
the
>> easiest/fastest reproduction is by saving the code snippet below in the
>> current directory as eg 'matrixIssue.R' and have it run in a
container as
>>
>> docker run --rm -ti -v `pwd`:/mnt rocker/r2u Rscript
/mnt/matrixIssue.R
>>
>> This runs in under two minutes, first installing the older Matrix, next
>> installs SeuratObject, and then by removing the older Matrix making the
>> (already installed) current Matrix version the default. This simulates
a
>> package update for Matrix. Which, as the final snippet demonstrates,
silently
>> breaks SeuratObject as the cached S4 method Csparse_validate is now
missing.
>> So when SeuratObject was installed under Matrix 1.5.1, it becomes
unuseable
>> under Matrix 1.6.0.
>>
>> What this shows is that a call to update.packages() will silently
corrupt an
>> existing installation. We understand that this was known and addressed
at
>> CRAN by rebuilding all binary packages (for macOS and Windows).
>>
>> But it leaves both users relying on source installation as well as
>> distributors of source packages in a dire situation. It hurt me three
times:
>> my default R installation was affected with unit tests (involving
>> SeuratObject) silently failing. It similarly broke our CI setup at
work. And
>> it created a fairly bad headache for the Debian packaging I am involved
with
>> (and I surmise it affects other distro similarly).
>>
>> It would be good to have a mechanism where a package, when being
upgraded,
>> could flag that 'more actions are required' by the system
(administrator).
>> We think this example demonstrates that we need such a mechanism to
avoid
>> (silently !!) breaking existing installations, possibly by forcing
>> reinstallation of other packages. R knows the package dependency graph
and
>> could trigger this, possibly after an 'opt-in' variable the
user / admin
>> sets.
>>
>> One possibility may be to add a new (versioned) field
'Breaks:'. Matrix could
>> then have added 'Breaks: SeuratObject (<= 4.1.3)' preventing
an installation
>> of Matrix 1.6.0 when SeuratObject 4.1.3 (or earlier) is present, but
>> permitting an update to Matrix 1.6.0 alongside a new version, say,
4.1.4 of
>> SeuratObject which could itself have a versioned Depends: Matrix (>=
1.6.0).
>>
>> Regards, Dirk
>>
>>
>> ## Code example follows. Recommended to run the rocker/r2u container.
>> ## Could also run 'apt update -qq; apt upgrade -y' but not
required
>> ## Thanks to my colleague Paul Hoffman for the core of this example
>>
>> ## now have Matrix 1.6.0 because r2u and CRAN remain current but we can
install an older Matrix
>> remotes::install_version('Matrix', '1.5.1')
>>
>> ## we can confirm that we have Matrix 1.5.1
>> packageVersion("Matrix")
>>
>> ## we now install SeuratObject from source and to speed things up we
first install the binary
>> install.packages("SeuratObject") # in this container via
bspm/r2u as binary
>> ## and then force a source installation (turning bspm off) _while
Matrix is at 1.5.1_
>> if (requireNamespace("bspm", quietly=TRUE) bspm::disable()
>> Sys.setenv(PKG_CXXFLAGS='-Wno-ignored-attributes') # Eigen
compilation noise silencer
>> install.packages('SeuratObject')
>>
>> ## we now remove the Matrix package version 1.5.1 we installed into
/usr/local leaving 1.6.0
>> remove.packages("Matrix")
>> packageVersion("Matrix")
>>
>> ## and we now run a bit of SeuratObject code that is now broken as
Csparse_validate is gone
>> suppressMessages(library(SeuratObject))
>> data('pbmc_small')
>> graph <- pbmc_small[['RNA_snn']]
>> class(graph)
>> getClass('Graph')
>> show(graph) # this fails
>>
>>