To whom it may concern, I am trying to maximize a log-likelihood function using optim. This is a simple problem with only 18 parameters. To conserve memory, I am using sparse matrices (SLAM) for some of the data matrices used in the computation of the likelihood. However, optim appears to convert the sparse matrix back to regular data format. This causes me to run out of memory as R tries to create an 8GB matrix. In short, it does not look as though "optim" is compatible with sparse matrices. Does anyone have a suggestion for how I can maximize a function in R using sparse matrices for some of the data inputs? Thanks, JP ____________________________________________ Jean-Pierre H. Dubé Sigmund E. Edelstone Professor of Marketing The University of Chicago | Booth School of Business 5807 S. Woodlawn Avenue Chicago, IL 60637 Tel: (773)-834-5377 e-mail: jdube@ChicagoBooth.edu<mailto:jdube@ChicagoBooth.edu> WWW: ChicagoBooth.edu/fac/jean-pierre.dube SSRN: http://ssrn.com/author= 105881<http://ssrn.com/author=%20105881> [[alternative HTML version deleted]]
Dube, Jean-Pierre wrote:> > To whom it may concern, > > I am trying to maximize a log-likelihood function using optim. This is a > simple problem with only 18 parameters. To conserve memory, I am using > sparse matrices (SLAM) for some of the data matrices used in the > computation of the likelihood. However, optim appears to convert the > sparse matrix back to regular data format. This causes me to run out of > memory as R tries to create an 8GB matrix. In short, it does not look as > though "optim" is compatible with sparse matrices. Does anyone have a > suggestion for how I can maximize a function in R using sparse matrices > for some of the data inputs? >Have a look at BB. Berend -- View this message in context: http://r.789695.n4.nabble.com/optimization-with-Sparse-matrices-tp3604373p3604453.html Sent from the R help mailing list archive at Nabble.com.
Dube, Jean-Pierre <Jean-Pierre.Dube <at> chicagobooth.edu> writes:> > To whom it may concern, > > I am trying to maximize a log-likelihood function using optim. > This is a simple problem with only 18 > parameters. To conserve memory, I am using sparse matrices > (SLAM) for some of the data matrices used in the > computation of the likelihood. However, optim appears to > convert the sparse matrix back to regular data > format. This causes me to run out of memory as R tries > to create an 8GB matrix. In short, it does not look as > though "optim" is compatible with sparse matrices. > Does anyone have a suggestion for how I can maximize a > function in R using sparse matrices for some of the data inputs? >I wouldn't guess that optim() itself would do any conversion of the data values (are these passed as additional parameters or used as global variables?) The only thing optim() does with parameters in '...' is pass them along to the objective function as follows: fn1 <- function(par) fn(par, ...) and I can't see a reason that would convert them in any way. It seems more likely to me (?) that there is some step in your objective function where you do an operation that accidentally converts the sparse back to non-sparse matrices ... Can you construct a minimal example that shows this happening?