Hi all, I am trying to run some plots on data, but when loading he CSV data file R is stopping and I am getting an out of memory error. Anyway to tweak this somehow to get it to run? Using WinXP with 4 GB RAM Tnx Bruce
"PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code." Otherwise...the only other possible suggestion is to use a smaller file. Julian Neotropical bat risk assessments wrote:> Hi all, > > I am trying to run some plots on data, but when loading he CSV data > file R is stopping and I am getting an out of memory error. > > Anyway to tweak this somehow to get it to run? > > Using WinXP with 4 GB RAM > > Tnx > > Bruce > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
Is your CSV file really that big? Have you passed the correct parameters to read.table()? For example, specified the correct sep="," parameter? If you've done everything correctly and are still running out of memory, maybe you need to go to 64-bit. R works great under 64-bit Ubuntu (8GB on my machine). - Ken Neotropical bat risk assessments wrote:> > Hi all, > > I am trying to run some plots on data, but when loading he CSV data > file R is stopping and I am getting an out of memory error. > > Anyway to tweak this somehow to get it to run? > > Using WinXP with 4 GB RAM > > Tnx > > Bruce > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > >-- View this message in context: http://www.nabble.com/Out-of-memory-issue-tp23226922p23227138.html Sent from the R help mailing list archive at Nabble.com.
Hi all,
Julian just reminded me I need to include the lines of code so here it is
below.
I am trying to run some plots on data, but when loading the CSV data file it
doss that but in ggplot R is stopping and I am getting an out of memory
error.
>
> d <- read.csv batcalls.rda
Error: unexpected symbol in "d <- read.csv batcalls.rda"
> attach(d)
Error in attach(d) : object "d" not found
> library(ggplot2)
> print(qplot(Sc, Fc))
Error in eval(expr, envir, enclos) : object "Sc" not found
> data(batcalls)
> attach(batcalls)
> print(qplot(Fc,geom="density"))
> b<-kmeans(Fc,c(10,15,18,30))
> print(qplot(Sc,Fc,colour=as.factor(b$cluster)))
>
> print(qplot(Dur,geom="density"))
> b<-kmeans(Dur,c(10,15,18,30))
> print(qplot(Dur,Fc,colour=as.factor(b$cluster)))
> d <- read.csv ("C:/Rainey/RainyAllbats.csv")
> attach(d)
The following object(s) are masked from batcalls :
Dc Dur Fc Fk Fmax Fmean Fmin Qk Qual S1 Sc st TBC Tc Tk
> library(ggplot2)
> print(qplot(Sc, Fc))
Warning messages:
1: In data.frame(..., check.names = FALSE) :
Reached total allocation of 1535Mb: see help(memory.size)
2: In data.frame(..., check.names = FALSE) :
Reached total allocation of 1535Mb: see help(memory.size)
3: Removed 1200249 rows containing missing values (geom_point).
> data(d)
Warning message:
In data(d) : data set 'd' not found
> attach(d)
The following object(s) are masked from d ( position 3 ) :
Dc Dur Fc Filename Fk Fmax Fmean Fmin Qk Qual S1 Sc st TBC Tc Tk
The following object(s) are masked from batcalls :
Dc Dur Fc Fk Fmax Fmean Fmin Qk Qual S1 Sc st TBC Tc Tk
> print(qplot(Fc,geom="density"))
There were 15 warnings (use warnings() to see them)
> b<-kmeans(Fc,c(10,15,18,30))
Error in switch(nmeth, { : NA/NaN/Inf in foreign function call (arg 1)
> f=jpeg(file="Sc x Fc.jpg")
> print(qplot(Sc,Fc,colour=as.factor(b$cluster)))
Error in data.frame(colour = c(3L, 4L, 4L, 4L, 4L, 1L, 4L, 2L, 1L, 1L, :
arguments imply differing number of rows: 9404, 2400489
> dev.off()
Anyway to tweak this somehow to get it to run?
Using WinXP with 4 GB RAM
Tnx
Bruce
Hi again all,
I can read all of the data is in CSV format with a header row and with
1,200,240 rows.
I can do all the plots and ggplot2 has no problems.
Seems to be a problem with the MASS package???
It seems that the contour plots are the problem with the memory error.
Error: cannot allocate vector of size 228.9 Mb
I can run smaller data sets and do the contour plots w/o problems.
The data can not really be summarized as I need to plot the actual values to
get the insight into the large data sets.
I am using latest R version 2.9.0 (2009-04-17)
My alternative is to bump up the memory of my one Dell workstation to 16 GB
as it runs with Ubuntu and can access that amount if I figure out how to
install R on a Linux box.
It may not truly be a memory issue but a limit to how MASS does the contour
plots?
This works 100% with the large data sets.
Bodega <- read.csv ("C:/R-Stats/Bat calls/Bodega.csv")
attach(Bodega)
library(ggplot2)
f=jpeg(file="Bodega Sc_Fc plot.jpg")
print(qplot(Sc, Fc))
dev.off()
f=jpeg(file="Bodega Fc Dens plot.jpg")
print(qplot(Fc,geom="density"))
dev.off()
This does not.and I get the vector size error.
library(MASS)
library(batcalls)
BRC<-kde2d(Sc,Fc)
f=jpeg(file="Rainey contour plot.jpg")
filled.contour(BRC)
dev.off()
Thanks again for in sight as I trundle up the learn curve for R.
Bruce
Neotropical bat risk assessments wrote:> Hi again all, > I can read all of the data is in CSV format with a header row and with > 1,200,240 rows. > I can do all the plots and ggplot2 has no problems. > Seems to be a problem with the MASS package??? > It seems that the contour plots are the problem with the memory error. > Error: cannot allocate vector of size 228.9 Mb > I can run smaller data sets and do the contour plots w/o problems. > The data can not really be summarized as I need to plot the actual values to > get the insight into the large data sets. > I am using latest R version 2.9.0 (2009-04-17) > My alternative is to bump up the memory of my one Dell workstation to 16 GB > as it runs with Ubuntu and can access that amount if I figure out how to > install R on a Linux box. > It may not truly be a memory issue but a limit to how MASS does the contour > plots? > This works 100% with the large data sets. > Bodega <- read.csv ("C:/R-Stats/Bat calls/Bodega.csv") > attach(Bodega) > library(ggplot2) > f=jpeg(file="Bodega Sc_Fc plot.jpg") > print(qplot(Sc, Fc)) > dev.off() > f=jpeg(file="Bodega Fc Dens plot.jpg") > print(qplot(Fc,geom="density")) > dev.off() > This does not.and I get the vector size error. > library(MASS) > library(batcalls) > BRC<-kde2d(Sc,Fc) > f=jpeg(file="Rainey contour plot.jpg") > filled.contour(BRC) > dev.off() > Thanks again for in sight as I trundle up the learn curve for R.You need to think about how many points you are plotting into a filled contour plot (i.e. estimate what the memory is comsumed for just holding the image in memory) and how much memory has been wasted already. You may want to reduce the size of the stuff you are going to plot. Uwev Ligges> Bruce > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.