Hi, I am working with a file (900MB in size) that has around 10 million records (in particular FASTQ records). I am able to read in the file as an object of BStringSet. When I start to manipulate the data, after almost 4 hours, I get the error message as Error: cannot allocate vector of size X.0 Mb (where X was once 160MB and then 180MB). The R version used is 2.11.1. I am not sure how to check if this is 64-bit or not so that I can use >4GB of RAM. I typed 'version' at the R prompt and see this: platform x86_64-unknown-linux-gnu arch x86_64 os linux-gnu system x86_64, linux-gnu. Does this mean it is already a 64-bit version of R I use? Please advice. -- View this message in context: http://r.789695.n4.nabble.com/Error-cannot-allocate-vector-of-size-X-0-Mb-tp2539031p2539031.html Sent from the R help mailing list archive at Nabble.com.
On Sep 14, 2010, at 9:09 AM, John1983 wrote:> > Hi, > > I am working with a file (900MB in size) that has around 10 million records > (in particular FASTQ records). > I am able to read in the file as an object of BStringSet. When I start to > manipulate the data, after almost 4 hours, I get the error message as Error: > cannot allocate vector of size X.0 Mb (where X was once 160MB and then > 180MB). > > The R version used is 2.11.1. I am not sure how to check if this is 64-bit > or not so that I can use >4GB of RAM. I typed 'version' at the R prompt and > see this: > platform x86_64-unknown-linux-gnu > arch x86_64 > os linux-gnu > system x86_64, linux-gnu. > Does this mean it is already a 64-bit version of R I use? > > Please advice.In an R session, see what the result of: .Machine$sizeof.pointer gets you. If it returns 4, you are using 32 bit R and you will need to install 64 bit R. If it returns 8, you are using 64 bit R. The information above only tells us/you that you are running a 64 bit OS. HTH, Marc Schwartz
Yes I see. So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). Is there anything else I need to check to remove this error? -- View this message in context: http://r.789695.n4.nabble.com/Error-cannot-allocate-vector-of-size-X-0-Mb-tp2539031p2539078.html Sent from the R help mailing list archive at Nabble.com.
On 14.09.2010 16:47, John1983 wrote:> > Yes I see. > So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). > > Is there anything else I need to check to remove this error?Yes: If the amount of RAM in your machine is sufficient.... Best, Uwe
On Sep 14, 2010, at 9:47 AM, John1983 wrote:> > Yes I see. > So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). > > Is there anything else I need to check to remove this error?1. Add more RAM. 2. Depending upon what you are doing relative to data management/analysis, you might want to see if there are more efficient approaches. 3. Free up RAM that may be in use by other processes, etc. 4. Look at R packages such as biglm, ff/bit, biganalytics, etc. 5. Consider using a backend data base application (eg. MySQL, Postgres, etc.) to manage the data sets and use R only for analyses, as may be appropriate relative to splitting the tasks. You did not indicate just how much RAM you have on the machine in question, but if it is within your budget/capability, adding more RAM would be the most transparent approach. HTH, Marc Schwartz