Good afternoon, I have been experiencing a lot of crashes working with large vectors in R. Specifically, I am using XTS of length of minimum 120k elements. My problem is that I cannot display the vector (otherwise R crashes), I cannot plot it either (otherwise R crashes). That could be solved by reducing the amount of points. However, I have been performing some statistical opreations on is and even sd(myXTS) crashes R. By "crashes", I mean shuts down without any warning whatsoever. I use R 2.11.1 (64). Has anyone had the same kind of problem? Can we solve this? Best, -- Jeremie [[alternative HTML version deleted]]
Hi Jeremie, Maybe you can take a look at the bigmemory package. If you have multi core or have access to clusters, you may want to use any parallel computing strategy. For plotting of large data, if you are using basic R graphics, first try to use "line" instead of using 'point', if this still doesn't working, you may want to try an alternative way by using qtinterface (R-forge project), install qtbase and qtpaint, they are still under development, but painting in QT interface is a lot faster for large data set. Best Tengfei On Fri, Jul 9, 2010 at 2:42 AM, Jeremie Smaga <jeremie@4ecap.com> wrote:> Good afternoon, > > I have been experiencing a lot of crashes working with large vectors in R. > > Specifically, I am using XTS of length of minimum 120k elements. > > My problem is that I cannot display the vector (otherwise R crashes), I > cannot plot it either (otherwise R crashes). That could be solved by > reducing the amount of points. > > However, I have been performing some statistical opreations on is and even > sd(myXTS) crashes R. > > By "crashes", I mean shuts down without any warning whatsoever. > > I use R 2.11.1 (64). > > Has anyone had the same kind of problem? > > Can we solve this? > > Best, > > > -- > Jeremie > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Tengfei Yin MCDB PhD student 1620 Howe Hall, 2274, Iowa State University Ames, IA,50011-2274 Homepage: www.tengfei.name [[alternative HTML version deleted]]
Jeremie Smaga <jeremie <at> 4ecap.com> writes:> > Good afternoon, > > I have been experiencing a lot of crashes working with large vectors in R. > > Specifically, I am using XTS of length of minimum 120k elements. > > My problem is that I cannot display the vector (otherwise R crashes), I > cannot plot it either (otherwise R crashes). That could be solved by > reducing the amount of points. > > However, I have been performing some statistical opreations on is and even > sd(myXTS) crashes R. > > By "crashes", I mean shuts down without any warning whatsoever. > > I use R 2.11.1 (64). > > Has anyone had the same kind of problem? > > Can we solve this?You need to provide a reproducible example, or as nearly reproducible as possible. You need to provide more information about your system (Windows?) -- the results of sessionInfo() would help. For example, I have no problem taking the standard deviation of a million-element vector:> set.seed(1001) > sd(runif(1e6))[1] 0.2884210> sessionInfo()R version 2.11.1 (2010-05-31) i486-pc-linux-gnu locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_US.UTF-8 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base what happens on your system? sd(
Hi r-help-bounces at r-project.org napsal dne 09.07.2010 11:42:47:> Good afternoon, > > I have been experiencing a lot of crashes working with large vectors inR.> > Specifically, I am using XTS of length of minimum 120k elements. > > My problem is that I cannot display the vector (otherwise R crashes), I > cannot plot it either (otherwise R crashes). That could be solved by > reducing the amount of points.I do not know what you mean by XTS but when you say it is a vector it seems that you have something rong with your R installation. x<-rnorm(1e7) # vector of 10 mil values. mean(x) sd(x) is almost instant. For plotting it is better to use a sample of such data plot(sample(x, 100000)) Regards Petr> > However, I have been performing some statistical opreations on is andeven> sd(myXTS) crashes R. > > By "crashes", I mean shuts down without any warning whatsoever. > > I use R 2.11.1 (64). > > Has anyone had the same kind of problem? > > Can we solve this? > > Best, > > > -- > Jeremie > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guidehttp://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code.