Hi All,
I want to print a square matrix of 7000 x 7000 into a text file. But I
got a error after few hours of computation...
--------
> write.table(MyDistMxDF, file = "temp.csv", sep=",",
quote=F)
*** malloc: vm_allocate(size=8421376) failed (error code=3)
*** malloc[2889]: error: Can't allocate region
Error: vector memory exhausted (limit reached?)
*** malloc: vm_allocate(size=8421376) failed (error code=3)
*** malloc[2889]: error: Can't allocate region
> q()
*** malloc: vm_allocate(size=8421376) failed (error code=3)
*** malloc[2889]: error: Can't allocate region
*** malloc: vm_allocate(size=8421376) failed (error code=3)
*** malloc[2889]: error: Can't allocate region
Error in lazyLoadDBfetch(key, datafile, compressed, envhook) :
internal error in decompress1
>
------
I am running R 2.0.1 on MacOS X 10.3 with 1Gb ram.
How could I write such a matrix to a text file.
David
Please use a current version of R: they are much better at this! write.table was rewritten in R 2.1.0 to use *much* less memory. There _is_ a `R Data Import/Export Manual' that dicsusses this. write.table was designed to write `tables' (data frames) not matrices. write.matrix (package MASS) does a much better job of the latter, memory-wise, in particular allowing the use of blocks. That _is_ on the help page for write.table in your version of R. Assuming this is a numeric matrix, it is taking up 373MB of storage. It is not too surprising that you are having problems manipulating it on a 1GB machine: my 1GB machine ended up swapping just creating such a matrix. On Wed, 20 Jul 2005, David Ruau wrote:> Hi All, > > I want to print a square matrix of 7000 x 7000 into a text file. But I > got a error after few hours of computation... > -------- > > write.table(MyDistMxDF, file = "temp.csv", sep=",", quote=F) > *** malloc: vm_allocate(size=8421376) failed (error code=3) > *** malloc[2889]: error: Can't allocate region > Error: vector memory exhausted (limit reached?) > *** malloc: vm_allocate(size=8421376) failed (error code=3) > *** malloc[2889]: error: Can't allocate region > > q() > *** malloc: vm_allocate(size=8421376) failed (error code=3) > *** malloc[2889]: error: Can't allocate region > *** malloc: vm_allocate(size=8421376) failed (error code=3) > *** malloc[2889]: error: Can't allocate region > Error in lazyLoadDBfetch(key, datafile, compressed, envhook) : > internal error in decompress1 > > > ------ > I am running R 2.0.1 on MacOS X 10.3 with 1Gb ram. > How could I write such a matrix to a text file.-- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595
Thanks for your answers, I need to print this data frame into a .csv file to import it in WEKA. Do you have better solution? I quite a new user of WEKA I don't know if you can give it a binary file. I think you can but it will be complicated... David On Jul 20, 2005, at 15:07, jim holtman wrote:> You might have better luck if you have a loop that is processing one > row at a time. That means you will have to determine what the > formatting should be. With write.table it is trying to process the > entire array at once to determine the best formatting and it taking a > lot of memory (my guess at least 500MB). > > What are you going to do with a text file that large? Can you write > it out in binary if you are reading it in with another program? If > you are going to reread it with R, then 'save' would be a better > choice. > > On 7/20/05, David Ruau <David.Ruau at rwth-aachen.de> wrote: >> Hi All, >> >> I want to print a square matrix of 7000 x 7000 into a text file. But I >> got a error after few hours of computation... >> -------- >>> write.table(MyDistMxDF, file = "temp.csv", sep=",", quote=F) >> *** malloc: vm_allocate(size=8421376) failed (error code=3) >> *** malloc[2889]: error: Can't allocate region >> Error: vector memory exhausted (limit reached?) >> *** malloc: vm_allocate(size=8421376) failed (error code=3) >> *** malloc[2889]: error: Can't allocate region >>> q() >> *** malloc: vm_allocate(size=8421376) failed (error code=3) >> *** malloc[2889]: error: Can't allocate region >> *** malloc: vm_allocate(size=8421376) failed (error code=3) >> *** malloc[2889]: error: Can't allocate region >> Error in lazyLoadDBfetch(key, datafile, compressed, envhook) : >> internal error in decompress1 >>> >> ------ >> I am running R 2.0.1 on MacOS X 10.3 with 1Gb ram. >> How could I write such a matrix to a text file. >> >> David >> >> ______________________________________________ >> R-help at stat.math.ethz.ch mailing list >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide! >> http://www.R-project.org/posting-guide.html >> > > > -- > Jim Holtman > > What the problem you are trying to solve? > >