Dear all, I want to read a 1.2 G file with 136,736,218 numbers with scan function, and then want make a histogram with those numbers, but it showed Error: cannot allocate vector of size 2.0 Gb. How can I import the file then? The version I used:> sessionInfo()R version 2.8.1 (2008-12-22) i686-pc-linux-gnu> mem.limits()nsize vsize NA NA and the computer I use has 4G memory in total. Thank you very much! Dongyan Song -- View this message in context: http://www.nabble.com/How-to-read-large-file-tp22540291p22540291.html Sent from the R help mailing list archive at Nabble.com.
Example 6e of http://code.google.com/p/sqldf/#Example_6._File_Input shows how to use the sqldf package to read a sample of rows from a file without reading the file itself into R. On Mon, Mar 16, 2009 at 11:11 AM, Dongyan Song <yzhskdls at hotmail.com> wrote:> > Dear all, > > I want to read a 1.2 G file with 136,736,218 numbers with scan function, and > then want make a histogram with those numbers, but it showed Error: cannot > allocate vector of size 2.0 Gb. How can I import the file then? > > The version I used: >> sessionInfo() > R version 2.8.1 (2008-12-22) > i686-pc-linux-gnu >> mem.limits() > nsize vsize > ? NA ? ?NA > and the computer I use has 4G memory in total. > > Thank you very much! > > Dongyan Song > > > -- > View this message in context: http://www.nabble.com/How-to-read-large-file-tp22540291p22540291.html > Sent from the R help mailing list archive at Nabble.com. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Try putting "memory limit linux" or "mem.limits" in the nabble help. Or type ?mem.limits in the R gui. Also have a look at: http://www.nabble.com/RAM%2C-swap%2C-Error%3A-cannot-allocate-vector-of-size%2C-Linux%3A-tt11680213.html#a11680213 Think you also need to check / understand how R deals with what you want to do so you know how many copies of the data it will need to hold in memory. Hope that helps, Darren Dongyan Song wrote:> > Dear all, > > I want to read a 1.2 G file with 136,736,218 numbers with scan function, > and then want make a histogram with those numbers, but it showed Error: > cannot allocate vector of size 2.0 Gb. How can I import the file then? > > The version I used: >> sessionInfo() > R version 2.8.1 (2008-12-22) > i686-pc-linux-gnu >> mem.limits() > nsize vsize > NA NA > and the computer I use has 4G memory in total. > > Thank you very much! > > Dongyan Song > > >-- View this message in context: http://www.nabble.com/How-to-read-large-file-tp22540291p22547077.html Sent from the R help mailing list archive at Nabble.com.