R help - Feb 2025 - Unexpected result after auto-attaching of the Matrix package

Dear all,

I just stumbled on this behavior - not sure if this is expected:

R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
Copyright (C) 2024 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu

 > library(Matrix)

 > sparseMatrix(i=1:10, j=1:10, x=rnorm(10)) |>
saveRDS("~/Downloads/sparsemat_test.rds")
 > q()
Save workspace image? [y/n/c]: n
# start R again
 > d <- readRDS("~/Downloads/sparsemat_test.rds")
 > class(d)
[1] "dgCMatrix"
attr(,"package")
[1] "Matrix"
 > dim(d)
Loading required package: Matrix
NULL
 > dim(d)
[1] 10 10

So the first call to dim(d) loads/attaches the Matrix package but
returns NULL for the call. Only the second call to dim returns the
correct results. I would expect the first call to succeed also,
otherwise what is the purpose of loading the package?

Best regards

Hilmar


 > sessionInfo()
R version 4.4.2 (2024-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.1 LTS

Matrix products: default
BLAS:?? /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0

locale:
 ?[1] LC_CTYPE=en_US.UTF-8?????? LC_NUMERIC=C
 ?[3] LC_TIME=de_DE.UTF-8??????? LC_COLLATE=en_US.UTF-8
 ?[5] LC_MONETARY=de_DE.UTF-8??? LC_MESSAGES=en_US.UTF-8
 ?[7] LC_PAPER=de_DE.UTF-8?????? LC_NAME=C
 ?[9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C

time zone: Europe/Berlin
tzcode source: system (glibc)

attached base packages:> sessionInfo()
R version 4.4.2 (2024-10-31)
Platform: x86_64-pc-linux-gnu
Running under: Ubuntu 24.04.1 LTS

Matrix products: default
BLAS:?? /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0

locale:
 ?[1] LC_CTYPE=en_US.UTF-8?????? LC_NUMERIC=C
 ?[3] LC_TIME=de_DE.UTF-8??????? LC_COLLATE=en_US.UTF-8
 ?[5] LC_MONETARY=de_DE.UTF-8??? LC_MESSAGES=en_US.UTF-8
 ?[7] LC_PAPER=de_DE.UTF-8?????? LC_NAME=C
 ?[9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C
[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C

time zone: Europe/Berlin
tzcode source: system (glibc)

attached base packages:
[1] stats???? graphics? grDevices utils???? datasets? methods base

other attached packages:
[1] Matrix_1.7-2

loaded via a namespace (and not attached):
[1] compiler_4.4.2 grid_4.4.2???? lattice_0.22-5

[1] stats???? graphics? grDevices utils???? datasets? methods base

other attached packages:
[1] Matrix_1.7-2

loaded via a namespace (and not attached):
[1] compiler_4.4.2 grid_4.4.2???? lattice_0.22-5

Why aren't you loading the package before you load the object that assumes
the relevant methods for that object are available?

On February 11, 2025 4:51:48 AM PST, H B via R-help <r-help at
r-project.org> wrote:
>Dear all,
>
>I just stumbled on this behavior - not sure if this is expected:
>
>R version 4.4.2 (2024-10-31) -- "Pile of Leaves"
>Copyright (C) 2024 The R Foundation for Statistical Computing
>Platform: x86_64-pc-linux-gnu
>
>> library(Matrix)
>
>> sparseMatrix(i=1:10, j=1:10, x=rnorm(10)) |>
>saveRDS("~/Downloads/sparsemat_test.rds")
>> q()
>Save workspace image? [y/n/c]: n
># start R again
>> d <- readRDS("~/Downloads/sparsemat_test.rds")
>> class(d)
>[1] "dgCMatrix"
>attr(,"package")
>[1] "Matrix"
>> dim(d)
>Loading required package: Matrix
>NULL
>> dim(d)
>[1] 10 10
>
>So the first call to dim(d) loads/attaches the Matrix package but
>returns NULL for the call. Only the second call to dim returns the
>correct results. I would expect the first call to succeed also,
>otherwise what is the purpose of loading the package?
>
>Best regards
>
>Hilmar
>
>
>> sessionInfo()
>R version 4.4.2 (2024-10-31)
>Platform: x86_64-pc-linux-gnu
>Running under: Ubuntu 24.04.1 LTS
>
>Matrix products: default
>BLAS:?? /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0
>LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0
>
>locale:
>?[1] LC_CTYPE=en_US.UTF-8?????? LC_NUMERIC=C
>?[3] LC_TIME=de_DE.UTF-8??????? LC_COLLATE=en_US.UTF-8
>?[5] LC_MONETARY=de_DE.UTF-8??? LC_MESSAGES=en_US.UTF-8
>?[7] LC_PAPER=de_DE.UTF-8?????? LC_NAME=C
>?[9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C
>[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
>
>time zone: Europe/Berlin
>tzcode source: system (glibc)
>
>attached base packages:> sessionInfo()
>R version 4.4.2 (2024-10-31)
>Platform: x86_64-pc-linux-gnu
>Running under: Ubuntu 24.04.1 LTS
>
>Matrix products: default
>BLAS:?? /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.12.0
>LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.12.0
>
>locale:
>?[1] LC_CTYPE=en_US.UTF-8?????? LC_NUMERIC=C
>?[3] LC_TIME=de_DE.UTF-8??????? LC_COLLATE=en_US.UTF-8
>?[5] LC_MONETARY=de_DE.UTF-8??? LC_MESSAGES=en_US.UTF-8
>?[7] LC_PAPER=de_DE.UTF-8?????? LC_NAME=C
>?[9] LC_ADDRESS=C?????????????? LC_TELEPHONE=C
>[11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
>
>time zone: Europe/Berlin
>tzcode source: system (glibc)
>
>attached base packages:
>[1] stats???? graphics? grDevices utils???? datasets? methods base
>
>other attached packages:
>[1] Matrix_1.7-2
>
>loaded via a namespace (and not attached):
>[1] compiler_4.4.2 grid_4.4.2???? lattice_0.22-5
>
>[1] stats???? graphics? grDevices utils???? datasets? methods base
>
>other attached packages:
>[1] Matrix_1.7-2
>
>loaded via a namespace (and not attached):
>[1] compiler_4.4.2 grid_4.4.2???? lattice_0.22-5
>
>______________________________________________
>R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
https://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.
-- 
Sent from my phone. Please excuse my brevity.

? Tue, 11 Feb 2025 13:51:48 +0100
H B via R-help <r-help at r-project.org> ?????:
>  > class(d)  
> [1] "dgCMatrix"
> attr(,"package")
> [1] "Matrix"
>  > dim(d)  
> Loading required package: Matrix
> NULL
Here's what happens here:

1. .Primitive("dim") considers dispatching a dim() method:
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/attrib.c#L1181
2. DispatchOrEval() first considers S4 dispatch:
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/eval.c#L4152
3. But then R_has_methods(dim) finds out that no S4 methods are
currently registered for dim() - Matrix not being loaded yet - and S4
dispatch is abandoned:
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/objects.c#L1578
4. Instead, R tries usemethod(...) for S3 dispatch:
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/eval.c#L4215
5. UseMethod() wants to find out the S3 class vector for 'd' and calls
R_data_class2():
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/objects.c#L493
6. R_data_class2() sees it's an S4 object and calls
methods:::.extendsForS3(class(d)), which finally loads Matrix in order
to find out classes extended by "dgCMatrix":
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/attrib.c#L840
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/attrib.c#L736-L741
7. S3 dispatch fails for all those S4 classes or the "default" method,
so dim(d) returns attr(d, "dim") (which is not set):
https://github.com/r-devel/r-svn/blob/fa73948a1977da05788499807469e69d5a21bd98/src/main/attrib.c#L1184

Then why does print(d) work without loading Matrix first?
print.default() internally performs S4 dispatch after Matrix was loaded
by R_data_class2() on step (6) above.

So while Matrix is not being loaded by accident in your example, things
are not set up to load the packages containing S4 classes
automatically. It could become a feature request for DispatchOrEval()
to ensure that the the package owning the S4 class is loaded before
trying to dispatch an S4 method, although it could be challenging to
implement while avoiding additional slowdowns.

-- 
Best regards,
Ivan