Hello R users, I am trying to run a cox model for the prediction of relapse of 80 cancer tumors, taking into account the expression of 17000 genes. The data are large and I retrieve an error: "Cannot allocate vector of 2.4 Mb". I increase the memory.limit to 4000 (which is the largest supported by my computer) but I still retrieve the error because of other big variables that I have in the workspace. Does anyone know how to overcome this problem? Many thanks in advance, Eleni [[alternative HTML version deleted]]
I have a similar problem, saying "cannot allocate vector size of 300MB". I would also appreciate if someone can offer some suggestion on this. Best, Shige On Jan 31, 2008 2:48 PM, Eleni Christodoulou <elenichri at gmail.com> wrote:> Hello R users, > > I am trying to run a cox model for the prediction of relapse of 80 cancer > tumors, taking into account the expression of 17000 genes. The data are > large and I retrieve an error: > "Cannot allocate vector of 2.4 Mb". I increase the memory.limit to 4000 > (which is the largest supported by my computer) but I still retrieve the > error because of other big variables that I have in the workspace. Does > anyone know how to overcome this problem? > > Many thanks in advance, > Eleni > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
On Thu, 31 Jan 2008, Eleni Christodoulou wrote:> Hello R users, > > I am trying to run a cox model for the prediction of relapse of 80 cancer > tumors, taking into account the expression of 17000 genes. The data are > large and I retrieve an error: > "Cannot allocate vector of 2.4 Mb". I increase the memory.limit to 4000 > (which is the largest supported by my computer) but I still retrieve the > error because of other big variables that I have in the workspace. Does > anyone know how to overcome this problem?Use a 64-bit version of R. (The 'minimal information' asked for in the posting guide would have helped us give a more information answer, but likely the problem is too big for a 32-bit OS.)> > Many thanks in advance, > Eleni > > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Elena, Page 23 of the R Installation Guide provides some memory guidelines that you might find helpful. There are a few things you could try using R, at least to get up and running: - Look at fewer tumors at a time using standard R as you have been. - Look at the ff package, which leaves the data in flat files with memory mapped pages. - It may be that package filehash does something similar using a database (I'm less familiar with this). - Wait for the upcoming package bigmemoRy package, which is designed to place large objects like this in RAM (using C++) but gives you a close-to-seamless interaction with it from R. Caveat below. With any of these options, you are still very much restricted by the type of analysis you are attempting. Almost any existing procedure (e.g. a cox model) would need a regular R object (probably impossible) and you are back to square one. An exception to this is Thomas Lumley's biglm package, which processes the data in chunks. We need more tools like these. Ultimately, you'll need to find some method of analysis that is pretty smart memory-wise, and this may not be easy. Best of luck, Jay ------------------------- Original message: I am trying to run a cox model for the prediction of relapse of 80 cancer tumors, taking into account the expression of 17000 genes. The data are large and I retrieve an error: "Cannot allocate vector of 2.4 Mb". I increase the memory.limit to 4000 (which is the largest supported by my computer) but I still retrieve the error because of other big variables that I have in the workspace. Does anyone know how to overcome this problem? Many thanks in advance, Eleni -- John W. Emerson (Jay) Assistant Professor of Statistics Director of Graduate Studies Department of Statistics Yale University http://www.stat.yale.edu/~jay Statistical Consultant, REvolution Computing