[R] data.frame building

Jean Eid
Mon, 02 Jun 2003 15:27:35 -0000

Hi all,
I have two seperate questions that both deal with the way R handels data
sets:

First, I am trying to read a data set of 80M into R. I am using
read.table(). The file is a tab file and I have tested the function for a
small amount of lines. It seems to work fine (i.e. correct amount of columns
and rows). However, when I try to read the full data set, R outputs a memory
limit error and stops responding to any command even the gc()  and the
quit() commands. It says it reached maximum memory of 489MB (that is the
amount of RAM I have). I tried the to increase vsize and nsize but no luck.

My second question concerns building data frames in a loop. I know a while
back Prof. Ripley has suggested to construct a data frame of the right size
outside the loop and fill it within the loop. unfortunately, it is taken so
long (this is on P4 2.4 G 489M RAM). I am wondering if there was any other
way that is faster to do this job. I also do not jnow why it is faster to
construct columns in a data.frame than it is to do rows. i.e. I had to
construct 719 columns and it does not take any time to do so (each) however
if I say data[i,]<-data2[j,] it takes so long versus
data$temp<-data$a==data$b for example.


any help is greatly appreciated

Jean Eid