R help - Nov 2009 - how to pass matrices from C to R effectively

Hi,

I have C code to produce a lot of matrices to be analyzed. As these
matrices are large (> 1000*10000) and are a lot (> 1000), I am thinking
about how to pass them from C to R effectively.

Would you think about the following solution? In R, I create a wrapper
function

passDataFromCToR = function(row, col) {
         mat = matrix(0, row, col)
	.C("passDataFromCToR",mat)[[1]]
}

It wraps the following C function
void passDataFromCToR (double *m, int *row, int* col){
   mymat = f() // my c function to produce a matrix
   // then I copy mymat to m element by element via a loop

}

Then to use these functions, I would write in R
mat = passDataFromCToR(1000, 10000)

Two issues with this approach are that 1) I have to copy the data once and
2)to the worse, in R I have to know the dimension of the matrices to be
passed from C. This information is not always available.

I would appreciate if you share with me your thoughts on how to solve this
problem better.

thanks

Jeff

On 12/11/2009 6:45 PM, Hao Cen wrote:
> Hi,
> 
> I have C code to produce a lot of matrices to be analyzed. As these
> matrices are large (> 1000*10000) and are a lot (> 1000), I am
thinking
> about how to pass them from C to R effectively.
> 
> Would you think about the following solution? In R, I create a wrapper
> function
> 
> passDataFromCToR = function(row, col) {
>          mat = matrix(0, row, col)
> 	.C("passDataFromCToR",mat)[[1]]
That .C call doesn't match the header of the function below:  you didn't
pass row and col.
> }
> 
> It wraps the following C function
> void passDataFromCToR (double *m, int *row, int* col){
>    mymat = f() // my c function to produce a matrix
>    // then I copy mymat to m element by element via a loop
> 
> }
> 
> Then to use these functions, I would write in R
> mat = passDataFromCToR(1000, 10000)
> 
> Two issues with this approach are that 1) I have to copy the data once and
> 2)to the worse, in R I have to know the dimension of the matrices to be
> passed from C. This information is not always available.
Why would you have to copy the data?  Just tell your C function to use 
the memory that R sent to it, instead of copying entries into it.  (R 
may be doing a copy when you extract the results, though.)

If your C code can't use memory provided to it, then there are no simple 
ways to avoid copying.  The complicated way is to use the .Call 
interface instead of .C.  There are examples in Writing R Extensions. 
They *will* require modifications to your code if it can't take a 
pointer to the memory to use.

For the question of the dimensions being unknown in advance:  Just write 
two functions.  The first does the calculations up to the point where it 
needs the memory, the second works on an appropriately sized array, that 
your R code allocated based on the first result.  Or use the .Call() 
interface.

Duncan Murdoch
> 
> I would appreciate if you share with me your thoughts on how to solve this
> problem better.
> 
> thanks
> 
> Jeff
> 
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.