R help - Apr 2016 - Matrix: How create a _row-oriented_ sparse Matrix (=dgRMatrix)?

Using the Matrix package, how can I create a row-oriented sparse
Matrix from scratch populated with some data?  By default a
column-oriented one is created and I'm aware of the note that the
package is optimized for column-oriented ones, but I'm only interested
in using it for holding my sparse row-oriented data and doing basic
subsetting by rows (even using drop=FALSE).

Here is what I get when I set up a column-oriented sparse Matrix:
> Cc <- Matrix(0, nrow=5, ncol=5, sparse=TRUE)
> Cc[1:3,1] <- 1
> Cc
5 x 5 sparse Matrix of class "dgCMatrix"

[1,] 1 . . . .
[2,] 1 . . . .
[3,] 1 . . . .
[4,] . . . . .
[5,] . . . . .
> str(Cc)
Formal class 'dgCMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:3] 0 1 2
  ..@ p       : int [1:6] 0 3 3 3 3 3
  ..@ Dim     : int [1:2] 5 5
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x       : num [1:3] 1 1 1
  ..@ factors : list()

When I try to do the analogue for a row-oriented matrix, I get a
"dgTMatrix", whereas I would expect a "dgRMatrix":
> Cr <- Matrix(0, nrow=5, ncol=5, sparse=TRUE)
> Cr <- as(Cr, "dsRMatrix")
> Cr[1,1:3] <- 1
> Cr
5 x 5 sparse Matrix of class "dgTMatrix"

[1,] 1 1 1 . .
[2,] . . . . .
[3,] . . . . .
[4,] . . . . .
[5,] . . . . .
> str(Cr)
Formal class 'dgTMatrix' [package "Matrix"] with 6 slots
  ..@ i       : int [1:3] 0 0 0
  ..@ j       : int [1:3] 0 1 2
  ..@ Dim     : int [1:2] 5 5
  ..@ Dimnames:List of 2
  .. ..$ : NULL
  .. ..$ : NULL
  ..@ x       : num [1:3] 1 1 1
  ..@ factors : list()


Trying with explicit coercion does not work:
> as(Cc, "dgRMatrix")
Error in as(Cc, "dgRMatrix") :
  no method or default for coercing "dgCMatrix" to
"dgRMatrix"
> as(Cr, "dgRMatrix")
Error in as(Cr, "dgRMatrix") :
  no method or default for coercing "dgTMatrix" to
"dgRMatrix"


Am I doing some wrong here?  Or is this what means that the package is
optimized for the column-oriented representation and I shouldn't
really work with row-oriented ones?  I'm really only interested in
access to efficient Cr[row,,drop=FALSE] subsetting (and a small memory
footprint).

Thanks,

Henrik

>>>>> Henrik Bengtsson <henrik.bengtsson at gmail.com>
>>>>>     on Tue, 19 Apr 2016 14:04:11 -0700 writes:
    > Using the Matrix package, how can I create a row-oriented sparse
    > Matrix from scratch populated with some data?  By default a
    > column-oriented one is created and I'm aware of the note that the
    > package is optimized for column-oriented ones, but I'm only
interested
    > in using it for holding my sparse row-oriented data and doing basic
    > subsetting by rows (even using drop=FALSE).

    > Here is what I get when I set up a column-oriented sparse Matrix:

    >> Cc <- Matrix(0, nrow=5, ncol=5, sparse=TRUE)
    >> Cc[1:3,1] <- 1

A general ("teaching") remark :
The above use of Matrix() is seen in many places, and is fine
for small matrices and the case where you only use the `[<-`
method very few times (as above).
Also using  Matrix()  is nice when being introduced to using the
Matrix package.

However, for efficience in non-small cases, do use

   sparseMatrix()

directly to construct sparse matrices.


    >> Cc
    > 5 x 5 sparse Matrix of class "dgCMatrix"

    > [1,] 1 . . . .
    > [2,] 1 . . . .
    > [3,] 1 . . . .
    > [4,] . . . . .
    > [5,] . . . . .
    >> str(Cc)
    > Formal class 'dgCMatrix' [package "Matrix"] with 6
slots
    > ..@ i       : int [1:3] 0 1 2
    > ..@ p       : int [1:6] 0 3 3 3 3 3
    > ..@ Dim     : int [1:2] 5 5
    > ..@ Dimnames:List of 2
    > .. ..$ : NULL
    > .. ..$ : NULL
    > ..@ x       : num [1:3] 1 1 1
    > ..@ factors : list()

    > When I try to do the analogue for a row-oriented matrix, I get a
    > "dgTMatrix", whereas I would expect a "dgRMatrix":

    >> Cr <- Matrix(0, nrow=5, ncol=5, sparse=TRUE)
    >> Cr <- as(Cr, "dsRMatrix")
    >> Cr[1,1:3] <- 1
    >> Cr
    > 5 x 5 sparse Matrix of class "dgTMatrix"

    > [1,] 1 1 1 . .
    > [2,] . . . . .
    > [3,] . . . . .
    > [4,] . . . . .
    > [5,] . . . . .

The reason for the above behavior has been

a) efficiency.  All the subassignment ( `[<-` ) methods for
   "RsparseMatrix" objects (of which "dsRMatrix" is a
special case)
   are implemented via  TsparseMatrix.
b) because of the general attitude that Csparse (and Tsparse to
   some extent) are well supported in Matrix,
   and e.g. further operations on Rsparse matrices would *again*
   go via T* or C* sparse ones, I had decided to keep things Tsparse.

[...]

    > Trying with explicit coercion does not work:

    >> as(Cc, "dgRMatrix")
    > Error in as(Cc, "dgRMatrix") :
    > no method or default for coercing "dgCMatrix" to
"dgRMatrix"

    >> as(Cr, "dgRMatrix")
    > Error in as(Cr, "dgRMatrix") :
    > no method or default for coercing "dgTMatrix" to
"dgRMatrix"

The general philosophy in 'Matrix' with all the class
hierarchies and the many specific classes has been to allow and
foster coercing to abstract super classes,
i.e, to  "dMatrix"  or "generalMatrix",
"triangularMatrix", or
then "denseMatrix", "sparseMatrix",
"CsparseMatrix" or
"RsparseMatrix", etc

So in the above  as(*, "RsparseMatrix")   should work always.


As a summary, in other words,  for what you want,

   as(sparseMatrix(.....), "RsparseMatrix")

should give you what you want reliably and efficiently.


    > Am I doing some wrong here?  Or is this what means that the package is
    > optimized for the column-oriented representation and I shouldn't
    > really work with row-oriented ones?  I'm really only interested in
    > access to efficient Cr[row,,drop=FALSE] subsetting (and a small memory
    > footprint).

{ though you could equivalently use   Cc[,row, drop=FALSE]
  with a CsparseMatrix Cc := t(Cr),
  couldn't you ?
}


Martin Maechler  (maintainer of 'Matrix')
ETH Zurich

On Wed, Apr 20, 2016 at 1:25 AM, Martin Maechler
<maechler at stat.math.ethz.ch> wrote:
>>>>>> Henrik Bengtsson <henrik.bengtsson at gmail.com>
>>>>>>     on Tue, 19 Apr 2016 14:04:11 -0700 writes:
>
>     > Using the Matrix package, how can I create a row-oriented sparse
>     > Matrix from scratch populated with some data?  By default a
>     > column-oriented one is created and I'm aware of the note that
the
>     > package is optimized for column-oriented ones, but I'm only
interested
>     > in using it for holding my sparse row-oriented data and doing
basic
>     > subsetting by rows (even using drop=FALSE).
>
>     > Here is what I get when I set up a column-oriented sparse Matrix:
>
>     >> Cc <- Matrix(0, nrow=5, ncol=5, sparse=TRUE)
>     >> Cc[1:3,1] <- 1
>
> A general ("teaching") remark :
> The above use of Matrix() is seen in many places, and is fine
> for small matrices and the case where you only use the `[<-`
> method very few times (as above).
> Also using  Matrix()  is nice when being introduced to using the
> Matrix package.
>
> However, for efficience in non-small cases, do use
>
>    sparseMatrix()
>
> directly to construct sparse matrices.
>
>
>     >> Cc
>     > 5 x 5 sparse Matrix of class "dgCMatrix"
>
>     > [1,] 1 . . . .
>     > [2,] 1 . . . .
>     > [3,] 1 . . . .
>     > [4,] . . . . .
>     > [5,] . . . . .
>     >> str(Cc)
>     > Formal class 'dgCMatrix' [package "Matrix"] with
6 slots
>     > ..@ i       : int [1:3] 0 1 2
>     > ..@ p       : int [1:6] 0 3 3 3 3 3
>     > ..@ Dim     : int [1:2] 5 5
>     > ..@ Dimnames:List of 2
>     > .. ..$ : NULL
>     > .. ..$ : NULL
>     > ..@ x       : num [1:3] 1 1 1
>     > ..@ factors : list()
>
>     > When I try to do the analogue for a row-oriented matrix, I get a
>     > "dgTMatrix", whereas I would expect a
"dgRMatrix":
>
>     >> Cr <- Matrix(0, nrow=5, ncol=5, sparse=TRUE)
>     >> Cr <- as(Cr, "dsRMatrix")
>     >> Cr[1,1:3] <- 1
>     >> Cr
>     > 5 x 5 sparse Matrix of class "dgTMatrix"
>
>     > [1,] 1 1 1 . .
>     > [2,] . . . . .
>     > [3,] . . . . .
>     > [4,] . . . . .
>     > [5,] . . . . .
>
> The reason for the above behavior has been
>
> a) efficiency.  All the subassignment ( `[<-` ) methods for
>    "RsparseMatrix" objects (of which "dsRMatrix" is a
special case)
>    are implemented via  TsparseMatrix.
> b) because of the general attitude that Csparse (and Tsparse to
>    some extent) are well supported in Matrix,
>    and e.g. further operations on Rsparse matrices would *again*
>    go via T* or C* sparse ones, I had decided to keep things Tsparse.
Thanks, understanding these design decisions is helpful.
Particularly, since I consider myself a rookie when it comes to the
Matrix package.
>
> [...]
>
>     > Trying with explicit coercion does not work:
>
>     >> as(Cc, "dgRMatrix")
>     > Error in as(Cc, "dgRMatrix") :
>     > no method or default for coercing "dgCMatrix" to
"dgRMatrix"
>
>     >> as(Cr, "dgRMatrix")
>     > Error in as(Cr, "dgRMatrix") :
>     > no method or default for coercing "dgTMatrix" to
"dgRMatrix"
>
> The general philosophy in 'Matrix' with all the class
> hierarchies and the many specific classes has been to allow and
> foster coercing to abstract super classes,
> i.e, to  "dMatrix"  or "generalMatrix",
"triangularMatrix", or
> then "denseMatrix", "sparseMatrix",
"CsparseMatrix" or
> "RsparseMatrix", etc
>
> So in the above  as(*, "RsparseMatrix")   should work always.
Thanks for pointing this out (and confirming as I since discovered the
virtual RsparseMatrix class in the help).
>
>
> As a summary, in other words,  for what you want,
>
>    as(sparseMatrix(.....), "RsparseMatrix")
>
> should give you what you want reliably and efficiently.
Perfect.
>
>
>     > Am I doing some wrong here?  Or is this what means that the
package is
>     > optimized for the column-oriented representation and I
shouldn't
>     > really work with row-oriented ones?  I'm really only
interested in
>     > access to efficient Cr[row,,drop=FALSE] subsetting (and a small
memory
>     > footprint).
>
> { though you could equivalently use   Cc[,row, drop=FALSE]
>   with a CsparseMatrix Cc := t(Cr),
>   couldn't you ?
> }
Yes, I actually went ahead did that, but since the code I'm writing
supports both plain matrix:es and sparse Matrix:es, and the underlying
model operates row-by-row, I figured the code would be more consistent
if I could use row-orientation everywhere.  Not a big deal.

Thanks Martin

Henrik
>
>
> Martin Maechler  (maintainer of 'Matrix')
> ETH Zurich
>