hy all,
I'm under linux,
I have now a big file (45 m??ga bytes), this file is a line of header defining
the names of the columns, the rows are only integers and there is 75 columns
(and 239096 rows included the header).
Before working with this big file i was working with another one (1.7 m??ga
bytes) , loading it with read.table("data.dat",h=T) (i've tested
too with scan("data.dat",what="integer",skip=1)), there
where no problems, it where taken approx 1 sec to load it into R.
but now with the big file R it seems that R stay stuck (i've wait more than
5 minutes and R don't come back from the read.table function, i've
tested with scan("data.dat",what="integer",skip=1)).
I say it stay stuck because i've spy R with the linux "top"
command line, and R begin to take all my memory (no prob with that) and 100% of
my cpu (no prob too), then after 3 or 4 minutes it take only 3% of cpu and the
memory allocated to R is average 90%, but R never come back from these functions
(or i haven't wait enough, and i need to give a result as speed as i can
unfortunatly).
I've divided the size of the file by 2 , then it works (i've only tested
with scan) but it is too slow...
Are there some solutions to load it in one piece?
Or have i to truncate the file and access it piece by piece?
thks all
guillaume.
