Jeannette Wicks-Lim
2003-Mar-20 18:56 UTC
[R] Question about Error due to memory allocation issue
Hi, I'm new to using R and I'm having problems in trying to do some quantile regressions with the package quantreg. When I try to use rq [specifically, I type in: rq(lnwagehr~lnmin1 + lncpi + ue03 + ue35 + ue57 + ue79, tau=.05, data=mydata, weights=pworwgt, method="fn", na.omit) ] , the computer churns for about 15 minutes and then spits out the following error message: Error: cannot allocate vector of size 109035 Kb I'm not sure what it going on but here is some (possibly) relevant info: -My computer has 512Mb of memory -My data set was originally in Stata and I read it into R using the package foreign -My data set is VERY LARGE: about 1.4 million obs. The data set was 90Mb in Stata. The model I've tried to evaluate only has 5 variables, but I'd like to expand this to about 43 (I have lots of state and year dummies). I have tried using the command: memory.size(max=TRUE):> memory.size(max=TRUE)[1] 2076487680 but I then get (after submitting another rq command): Error: cannot allocate vector of size 76324 Kb Thanks very much for any help you can provide! Jeannette Wicks-Lim Department of Economics University of Massachusetts, Amherst Amherst, MA 01003
Thomas Lumley
2003-Mar-21 07:37 UTC
[R] Question about Error due to memory allocation issue
On Thu, 20 Mar 2003, Jeannette Wicks-Lim wrote:> Hi, > > I'm new to using R and I'm having problems in trying to do some quantile > regressions with the package quantreg. When I try to use rq [specifically, I > type in: rq(lnwagehr~lnmin1 + lncpi + ue03 + ue35 + ue57 + ue79, tau=.05, > data=mydata, weights=pworwgt, method="fn", na.omit) ] , the computer churns for > about 15 minutes and then spits out the following error message: > > Error: cannot allocate vector of size 109035 Kb >This means that it tried to allocate 109035k on top of what was already allocated, and couldn't. You probably can't fit this model in the memory. Fortunately you probably don't want to. You can almost certainly either aggregate to a smaller number of observations or analyse subgroups of the data and get a more useful result as well as one that R can compute. -thomas