Good evening to everybody, I have problems to import in R a really big dataset (more than 1000000 values). Which is the best package to install? Is there someone who works with this kind of dataset and can help me, please? Thank you very much, Regards Dr.ssa Erika Frigo Department of Veterinary Sciences and Technology for Food Safety University of Milan Via Grasselli, 7 20137 Milano Tel. +39 0250318515 Fax +39 0250318501 [[alternative HTML version deleted]]
Is it just a file with a million values or is it some type of a structure with a million rows of indeterinent columns? If it is just a million numbers, you can easily read with is 'scan' or 'read.table' with no problem. I work with data structures that have several million rows and 4-5 columns without any problems. What is the format of the input? On 3/4/08, Erika Frigo <erika.frigo at unimi.it> wrote:> Good evening to everybody, > I have problems to import in R a really big dataset (more than 1000000 values). Which is the best package to install? > Is there someone who works with this kind of dataset and can help me, please? > > Thank you very much, > Regards > > > > > > Dr.ssa Erika Frigo > Department of Veterinary Sciences and Technology for Food Safety > University of Milan > > Via Grasselli, 7 > 20137 Milano > Tel. +39 0250318515 > Fax +39 0250318501 > [[alternative HTML version deleted]] > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Jim Holtman Cincinnati, OH +1 513 646 9390 What is the problem you are trying to solve?
Hi, Erika Frigo wrote:> Good evening to everybody, > I have problems to import in R a really big dataset (more than 1000000 values). Which is the best package to install? > Is there someone who works with this kind of dataset and can help me, please? >Maybe the package SQLiteDF could be useful for you. http://cran.r-project.org/web/packages/SQLiteDF/index.html But since you mention that the data has 1 mio values, I think it should be no problem to read the data set "conventionally". > (object.size(rnorm(1e06)))/(1024^2) [1] 7.629417 Assuming that all data are numeric, the data-set should consume less than 8MB. I hope this helps, Roland
On Tue, Mar 4, 2008 at 10:35 AM, Erika Frigo <erika.frigo at unimi.it> wrote:> Good evening to everybody, > I have problems to import in R a really big dataset (more than 1000000 values). Which is the best package to install? > Is there someone who works with this kind of dataset and can help me, please?A good place to start is the manual "R Data Import/Export" that comes with every installed version of R.