Dear all, I am trying to use rlm in MASS to do an analysis on a 21632*12 matrix. There are 4 factors A,D,V,G; A of level 6, D of level 2, V of level 2, G of level 21632, and 21632*12 observations Y. The data format is A D V G Y 1 1 1 1 0.1 1 1 1 2 0.2 ... The command I use is rlm(Y~A+D+V*G,data=data.dat) I got the same error message: Error in rep(data, t1) : cannot allocate vector of length 467748756 I am using a unix machine of 4GM memory. I would like to know if R is using all the available memory and/or how much memory R would use to handle is data set. Jing -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Your numbers are a bit confusing. You said the "matrix" (shouldn't it be a data frame?) is 21632*12. Does that mean you have 21632 rows and 12 columns? Then how can Y also be 21632*12? If G is really a factor with 21632 levels, and there are 21632 observations, then having G alone in the model will give you perfect prediction, but quite useless. Andy> -----Original Message----- > From: Jing Wu [mailto:jingw at cse.ucsc.edu] > Sent: Thursday, September 05, 2002 10:58 PM > To: r-help at stat.math.ethz.ch > Subject: [R] R memory question (fwd) > > > Dear all, > > I am trying to use rlm in MASS to do an analysis on a 21632*12 matrix. > There are 4 factors A,D,V,G; A of level 6, D of level 2, V > of level 2, G > of level 21632, and 21632*12 observations Y. The data format is > A D V G Y > 1 1 1 1 0.1 > 1 1 1 2 0.2 > ... > > The command I use is rlm(Y~A+D+V*G,data=data.dat) > I got the same error message: > Error in rep(data, t1) : cannot allocate vector of length 467748756 > > I am using a unix machine of 4GM memory. I would like to know if R is > using all the available memory and/or how much memory R would use to > handle is data set. > > > Jing > > > > > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-. > -.-.-.-.-.-.-.-.- > r-help mailing list -- Read > http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html > Send "info", "help", or "[un]subscribe" > (in the "body", not the subject !) To: > r-help-request at stat.math.ethz.ch > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._. > _._._._._._._._._ >------------------------------------------------------------------------------ Notice: This e-mail message, together with any attachments, contains information of Merck & Co., Inc. (Whitehouse Station, New Jersey, USA) that may be confidential, proprietary copyrighted and/or legally privileged, and is intended solely for the use of the individual or entity named on this message. If you are not the intended recipient, and have received this message in error, please immediately return this by e-mail and then delete it. ============================================================================= -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Sounds like the design matrix is 21632*12 by over 2*21632. That's 10 billion numbers. You can't do that in 4Gb of memory, and doubt you actually have 4GM bytes. If this really makes statistical sense you will have to exploit the special structure that makes it sensible. On Thu, 5 Sep 2002, Jing Wu wrote:> Dear all, > > I am trying to use rlm in MASS to do an analysis on a 21632*12 matrix. > There are 4 factors A,D,V,G; A of level 6, D of level 2, V of level 2, G > of level 21632, and 21632*12 observations Y. The data format is > A D V G Y > 1 1 1 1 0.1 > 1 1 1 2 0.2 > ... > > The command I use is rlm(Y~A+D+V*G,data=data.dat) > I got the same error message: > Error in rep(data, t1) : cannot allocate vector of length 467748756 > > I am using a unix machine of 4GM memory. I would like to know if R is > using all the available memory and/or how much memory R would use to > handle is data set. > > > Jing > > > > > -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- > r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html > Send "info", "help", or "[un]subscribe" > (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch > _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._ >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._