Jose Quesada
2009-Oct-22 22:26 UTC
[R] dgTMatrix --- [, , drop=F] strange behavior, Matrix 0.999375-20
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, I have something strange here... I want to subset a large sparse matrix, with the subset being still in sparse form. Easily achievable with mm[i,,drop=F] , right? Well, it doesn't work on the matrix I'm working on. This is a very large wikipedia Matrix (now I can play with it as I just got 16Gb of memory):> mm at Dim[1] 793251 1027355> nnzero(mm)[1] 205746204> class(mm)[1] "dgTMatrix" attr(,"package") [1] "Matrix" Imagine I want just the first row:> a = mm[1,,drop=F] > length(a)[1] 1027355> nnzero(a)[1] 1291>class(a)"numeric" However, _a_ is just a numeric (dense) vector (!). This takes a long time and a big bite off my memory. This is NOT what I get when I play with a toy example. In this case it works as expected:> (m <- Matrix(c(0,0,2:0), 3,5))3 x 5 sparse Matrix of class "dgCMatrix" [1,] . 1 . . 2 [2,] . . 2 . 1 [3,] 2 . 1 . .> a = m[1,,drop=F] > a1 x 5 sparse Matrix of class "dgCMatrix" [1,] . 1 . . 2 I wonder if it's a matter of size:> M <- kronecker(m, diag(10000)) > a = m[1,,drop=F] > class(a)[1] "dgCMatrix" attr(,"package") [1] "Matrix" Looks like it's working as expected on a sizeable example. What could be going on? What's the difference between mm and M, (other than size?). I'm not sure this is enough info to reproduce the problem (i.e., the example matrices I posted work, mine doesn't). The problematic matrix is .5gb compressed as .Rdata, so it's not easy to send it around. This is 0.999375-20, not the latest version as of today: 0.999375-29; however it's the only one available in the revolution 64 repositories (win64). Any idea what to try next? I cannot afford to operate with dense representations. Best, - -Jose - -- === I'm learning a new keyboard layout. I type slowly. Please excuse typos and slower-that-usual responses. Thanks for your patience==Jose Quesada, PhD. Max Planck Institute, Center for Adaptive Behavior and Cognition -ABC-, Lentzeallee 94, office 224, 14195 Berlin http://www.josequesada.name/ http://twitter.com/Quesada -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) iQIcBAEBAgAGBQJK4NwwAAoJEGMobUVGH+HKqccQAIiT+7642HI56OWvfN9OqriK 5QsimU8tyaHwYn6Xlb6wx1Da2uBKpMBz9AfSHOO12xIr5Irm3DtHKPTjkGS3QV9P kYb1zUA3b5iZrFOj6hgChs4m6xx1MmsAwpG7Ft1oLXKY9/SF+Yqt4OoUHqRsNaI1 m+SpjZLEu8QKfq0F7FGVCJ8sGPMJ8E7nOq7qcRHQ3eIOcUoKZ0bFZrixDxD9oH02 enX4cqcHGJFiS27e70IIBoh4Q8mG96Z9LLm7/EbFdJMyWyzJQDVcBL+o/UUYjmTi AxMUQjfFf2/2nDz6mFJ3raU0AS6vLuliq1/B2K52znJxqa+GT9pNc7l79OTVa1oc F6BUBuAPj+1UQX1xO4lG6yjNoN1KZKmyoE/5tWnvWnp9tSxmszlty7f86/fpx3Mz 44dvLOcuucTUTLOdvkT9rSqWXS9M3GdS+kH5qD5RVd4ONw54iv4K3kxGfmmyb+6K b19DbP7sVMe8qgybYLAyuPmeOskWbcRh/glkRrXV642NoiQVtu1UcIiGoNPxpAF4 Qu5bHIRCvkayUVbxFuqNo3xHEaTdvTX5IqzlseVpDYOKkRv3SJeiLGfZgnUvH0wG zL5BIYBPXhtQjysL69VSyWs4mASUwZ0N5WhhWWTFVSnG+p1QS+CNURWwXYVTpvos 9kEvQ45lcQEKJ1bNuPBm =XUeB -----END PGP SIGNATURE-----