R devel - Aug 2021 - svd For Large Matrix

Good day,

I have a real scenario involving 45 million biological cells (samples) and 60
proteins (variables) which leads to a segmentation fault for svd. I thought this
might be a good example of why it might benefit from a long vector upgrade.

test <- matrix(rnorm(45000000*60), ncol = 60)
testSVD <- svd(test)

 *** caught segfault ***
address 0x7fe93514d618, cause 'memory not mapped'

Traceback:
 1: La.svd(x, nu, nv)
 2: svd(test)

--------------------------------------
Dario Strbenac
University of Sydney
Camperdown NSW 2050
Australia

[copying the list]

svd() does support matrices with long vector data. Your example works
fine for me on a machine with enough memory with either the reference
BLAS/LAPACK or the BLAS/LAPACK used on Fedora 33 (flexiblas backed, I
believe, by a version of openBLAS). Take a look at sessionInfo() to
see what you are using and consider switching to another BLAS/LAPACK
if necessary. Running under gdb may help tracking down where the issue
is and reporting it for the BLAS/LAPACK you are using.

Best,

luke

On Fri, 13 Aug 2021, Dario Strbenac via R-devel wrote:
> Good day,
>
> I have a real scenario involving 45 million biological cells (samples) and
60 proteins (variables) which leads to a segmentation fault for svd. I thought
this might be a good example of why it might benefit from a long vector upgrade.
>
> test <- matrix(rnorm(45000000*60), ncol = 60)
> testSVD <- svd(test)
>
> *** caught segfault ***
> address 0x7fe93514d618, cause 'memory not mapped'
>
> Traceback:
> 1: La.svd(x, nu, nv)
> 2: svd(test)
>
> --------------------------------------
> Dario Strbenac
> University of Sydney
> Camperdown NSW 2050
> Australia
>
> ______________________________________________
> R-devel at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>
-- 
Luke Tierney
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa                  Phone:             319-335-3386
Department of Statistics and        Fax:               319-335-3017
    Actuarial Science
241 Schaeffer Hall                  email:   luke-tierney at uiowa.edu
Iowa City, IA 52242                 WWW:  http://www.stat.uiowa.edu