On Aug 2, 2011, at 11:45 , Guillaume wrote:
> Dear all,
> I am trying to aggregate a table (divided in two lists here), but get a
> memory error.
> Here is the code I'm running :
>
> sessionInfo()
>
> print(paste("memory.limit() ", memory.limit()))
> print(paste("memory.size() ", memory.size()))
> print(paste("memory.size(TRUE) ", memory.size(TRUE)))
>
> print(paste("size listX ", object.size(listX)))
> print(paste("size listBy ", object.size(listBy)))
> print(paste("length ", object.size(nrow(listX))))
>
> tableAgg <- aggregate(x = listX
> , by = listBy
> , FUN = "max")
>
>
> It returns :
>
> R version 2.9.0 Patched (2009-05-09 r48513)
> i386-pc-mingw32
> locale:
>
LC_COLLATE=French_France.1252;LC_CTYPE=French_France.1252;LC_MONETARY=French_France.1252;LC_NUMERIC=C;LC_TIME=French_France.1252
> attached base packages:
> [1];stats;graphics;grDevices;utils;datasets;methods;base
> other attached packages:
> [1];RODBC_1.3-2;HarpTools_1.4;HarpReport_1.9
> loaded via a namespace (and not attached):
> [1];tools_2.9.0
> [1];"memory.limit() 4095"
> [1];"memory.size() 31.92"
> [1];"memory.size(TRUE) 166.94"
> [1];"size listX 218312"
> [1];"size listBy 408552"
> [1];"length 9083"
> Erreur in vector("list", prod(extent)) :
> cannot allocate vector of length 1224643220
>
> (the last line is translated from the french error message "impossible
> d'allouer un vecteur de longueur 1224643220" )
>
> Why would R create such a long vector (my original lists , and is there a
> way to avoid this error ?
>
It would be easier if you described your data rather than just tell us their
size, but as far as I can see, listX has about 50K columns and listBy has 100K.
So you are trying to form a table of the max of 50000 variables over the
cartesian product of 100000 classifiers? That's basically an infinite number
of cells.
> Thank you for your help,
>
> Guillaume
>
> --
> View this message in context:
http://r.789695.n4.nabble.com/Memory-limit-in-Aggregate-tp3711819p3711819.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: pd.mes at cbs.dk Priv: PDalgd at gmail.com
"D?den skal tape!" --- Nordahl Grieg