I am dealing with very large data frames, artificially created with the following code, that are combined using rbind. a <- rnorm(5000000) b <- rnorm(5000000) c <- rnorm(5000000) d <- rnorm(5000000) first <- data.frame(one=a, two=b, three=c, four=d) second <- data.frame(one=d, two=c, three=b, four=a) rbind(first, second) which results in the following error for each of the statements:> a <- rnorm(5000000)Error: cannot allocate vector of size 38.1 Mb> b <- rnorm(5000000)Error: cannot allocate vector of size 38.1 Mb> c <- rnorm(5000000)Error: cannot allocate vector of size 38.1 Mb> d <- rnorm(5000000)Error: cannot allocate vector of size 38.1 Mb> first <- data.frame(one=a, two=b, three=c, four=d)Error: cannot allocate vector of size 38.1 Mb> second <- data.frame(one=d, two=c, three=b, four=a)Error: cannot allocate vector of size 38.1 Mb> rbind(first, second)When running memory.limit() I am getting this: memory.limit() [1] 2047 Which shows me that I have 2 GB of memory available. What is wrong? Shouldn't 38 MB be very feasible? Best, Ralf
On Thu, Aug 05, 2010 at 03:53:21AM -0400, Ralf B wrote:> > a <- rnorm(5000000) > Error: cannot allocate vector of size 38.1 Mb> > When running memory.limit() I am getting this: > > memory.limit() > [1] 2047 > > Which shows me that I have 2 GB of memory available. What is wrong? > Shouldn't 38 MB be very feasible?>From what I gather fomr ?memory.limit it does not tell you how muchmemory it currently available. So my guess is that you have som rather large objects in your workspace already and thus there is not enough space left for you vectors. cu Philipp -- Dr. Philipp Pagel Lehrstuhl f?r Genomorientierte Bioinformatik Technische Universit?t M?nchen Wissenschaftszentrum Weihenstephan Maximus-von-Imhof-Forum 3 85354 Freising, Germany http://webclu.bio.wzw.tum.de/~pagel/
Hi r-help-bounces at r-project.org napsal dne 05.08.2010 09:53:21:> I am dealing with very large data frames, artificially created with > the following code, that are combined using rbind. > > > a <- rnorm(5000000) > b <- rnorm(5000000) > c <- rnorm(5000000) > d <- rnorm(5000000) > first <- data.frame(one=a, two=b, three=c, four=d) > second <- data.frame(one=d, two=c, three=b, four=a)Up to this point there is no error on my system> version_ platform i386-pc-mingw32 arch i386 os mingw32 system i386, mingw32 status Under development (unstable) major 2 minor 12.0 year 2010 month 05 day 31 svn rev 52164 language R version.string R version 2.12.0 Under development (unstable) (2010-05-31 r52164)> sessionInfo()R version 2.12.0 Under development (unstable) (2010-05-31 r52164) Platform: i386-pc-mingw32/i386 (32-bit) attached base packages: [1] stats grDevices datasets utils graphics methods base other attached packages: [1] lattice_0.18-8 fun_1.0 loaded via a namespace (and not attached): [1] grid_2.12.0 tools_2.12.0> rbind(first, second)Although size of first and second is only roughly 160 MB their concatenation probably consumes all remaining memory space as you already have a-d first and second in memory. Regards Petr> > which results in the following error for each of the statements: > > > a <- rnorm(5000000) > Error: cannot allocate vector of size 38.1 Mb > > b <- rnorm(5000000) > Error: cannot allocate vector of size 38.1 Mb > > c <- rnorm(5000000) > Error: cannot allocate vector of size 38.1 Mb > > d <- rnorm(5000000) > Error: cannot allocate vector of size 38.1 Mb > > first <- data.frame(one=a, two=b, three=c, four=d) > Error: cannot allocate vector of size 38.1 Mb > > second <- data.frame(one=d, two=c, three=b, four=a) > Error: cannot allocate vector of size 38.1 Mb > > rbind(first, second) > > When running memory.limit() I am getting this: > > memory.limit() > [1] 2047 > > Which shows me that I have 2 GB of memory available. What is wrong? > Shouldn't 38 MB be very feasible? > > Best, > Ralf > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code.
On Aug 5, 2010, at 3:53 AM, Ralf B wrote:> I am dealing with very large data frames, artificially created with > the following code, that are combined using rbind. ><snipped>> When running memory.limit() I am getting this: > > memory.limit() > [1] 2047 > > Which shows me that I have 2 GB of memory available. What is wrong? > Shouldn't 38 MB be very feasible?No. This suggests very strongly that you have not taken the time to read the RW-FAQ. R has become sufficiently popular that a Google search bring it up as the first hit to a search on: "RW FAQ" http://cran.r-project.org/bin/windows/base/rw-FAQ.html> > Best, > Ralf > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius, MD West Hartford, CT