Hello, I exprerienced a problem with large data sets in R: I cannot import these data via procedure "read.table" (insuficient memory) and some other functions end with the same expception. Could you tell me how to handle large data sets in R? Thank you. Pavel Vanecek ____________________________________________________________ Licitovat nejvyhodnejsi nab?dku je postavene na hlavu! Skoda Fabia nyni se zvyhodnenim az 50.000 Kc!http://ad2.seznam.cz/redir.cgi?instance=68739%26url=http://www.skoda-auto.cz/action/fast
Read the manual distributed with R about import/export data. However with large data set is preferable the direct use of the function scan (read.table is built with it). best regards! ...and good year A.S. ---------------------------- Alessandro Semeria Models and Simulations Laboratory Montecatini Environmental Research Center (Edison Group), Via Ciro Menotti 48, 48023 Marina di Ravenna (RA), Italy Tel. +39 544 536811 Fax. +39 544 538663 E-mail: alessandro.semeria at cramont.it
On Fri, 9 Jan 2004, PaTa PaTaS wrote:> I exprerienced a problem with large data sets in R: I cannot import > these data via procedure "read.table" (insuficient memory) and some > other functions end with the same expception. Could you tell me how to > handle large data sets in R?We need more details. Have you followed all the hints in ?read.table and the Data Import/Export manual? If you have, then probably your data set is too large for the memory of your version of R, and the simplest solution is to get more memory. To be more helpful we would need full details of the dataset and of the commands you used and the environment you are using (OS, how much RAM and how much virtual memory at least). -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Thank you all for your help. The problem is not only with reading the data (5000 cases times 2000 integer variables, imported either from SPSS or TXT file) into my R 1.8.0 but also with the procedure I would like to use = "randomForest" from library "randomForest". It is not possible to run it with such a data set (because of the insuficient memory exception). Moreover, my data has factors with more than 32 classes, which causes another error. Could you suggest any solution for my problem? Thank you a lot. ____________________________________________________________ Licitovat nejvyhodnejsi nab?dku je postavene na hlavu! Skoda Octavia nyni se zvyhodnenim az 90.000 Kc! http://ad2.seznam.cz/redir.cgi?instance=68740%26url=http://www.skoda-auto.cz/action/fast