Dr. Stephan Martin Rudolfer
2001-Aug-23 13:13 UTC
[R] Reading of a large SAS Export dataset
I am trying to read in a large SAS dataset (1706 observations, 72 variables) using the foreign package. R crashes, with the error message "This program has performed an illegal operation and will be shut down." The details given include "RGUI caused an invalid page fault in module FOREIGN.DLL at 015f:14c327e7." Bearing in mind the statement on p.2 of the R Data Import/Export document, "In general, statistical systems like R are not particularly well suited to manipulations of large-scale data," I must reluctantly conclude that my dataset is too large to be handled by R. Confirmation of this would save me a lot of time spent in futile further attempts at achieving the impossible. Many thanks again in advance. ****************************************************************************************** * Dr. Stephan M. Rudolfer Office tel: +44 161 275 5912 * * Chairman, Manchester Group, Royal Statistical Society * * Mathematics Department Dept. tel: +44 161 275 5800 * * The University Fax : +44 161 275 5819 * * MANCHESTER M13 9PL Email : s.m.rudolfer at man.ac.uk * * UK WWW : http://www.maths.man.ac.uk/DeptWeb/Homepages/smr/ * ****************************************************************************************** -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
On Thu, 23 Aug 2001, Dr. Stephan Martin Rudolfer wrote:> I am trying to read in a large SAS dataset (1706 observations, 72 > variables) using the foreign package. > > R crashes, with the error message "This program has performed an > illegal operation and will be shut down." The details given include > "RGUI caused an invalid page fault in module FOREIGN.DLL at > 015f:14c327e7." > > Bearing in mind the statement on p.2 of the R Data Import/Export > document, "In general, statistical systems like R are not > particularly well suited to manipulations of large-scale data," I > must reluctantly conclude that my dataset is too large to be > handled by R. > > Confirmation of this would save me a lot of time spent in futile > further attempts at achieving the impossible.That's not a large data set: it would be with a million rows (nad we have datasets that size). Package foreign is somewhat fragile, but do you have the latest version (which is not that in 1.3.0)? Try update.packages(), as the previous tow versions had serious memory-allocation bugs. If the problem persists, try the advice about dumping to a text file .... -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
Dear Everybody, Scatterplot matrices and lattice plots in S and R label respective axes on alternate sides. I kind of like this, but one of my collaborators doesn't like this feature. He is sort of a stick-in-the-mud when it comes to something he has never seen before, and in his defense, I don't think I have seen it before in any of our usual journals (e.g., Ecology, Journal of Ecology, Ecology Letters, Science, Nature). The primary merit to the alternate sides feature seems to be that it allows more space around each axis label. The two problems seem to be that (1) reviewers don't like unusual formatting, and (2) when examining the data you often have to hunt for the axis label (these two problems are, obviously, related). I know I can specify label placement in S+, and I assume I can do so in R. I am inclined, however, to go with the default (alternate side) approach, because I trust in the wisdom of the R developers, and I like unusual, novel things. I will appreciate any opinions on this subject. Henry ***************************** Martin Henry H. Stevens HStevens at muohio.edu tel: (513) 529 - 4206 FAX: (513) 529 - 4243 338 Pearson Hall Botany Department Miami University Oxford, OH 45056 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._