[R] rbind wastes memory

Douglas Bates
Mon, 30 May 2005 14:03:04 -0000

> Now I played around, but I couldn't find a solution. For example I
bound each dataframe
> step by step and removed the variables and cleared memory, but I still need
1140MB(!)
> RAM:
>
> ________ start example 2 __________
> tmpFile<- paste(myFile,'.tmp',sep="")
> load(myFile)
> ds.tmp	 <- ds
> save(ds.tmp, file=tmpFile, compress=T)
>
> for (Cycle in 1:10) {
> 	ds	<- NULL
> 	ds.tmp <- NULL
> 	rm(ds, ds.tmp)
> 	gc()
> 	load(tmpFile)
> 	load(myFile)
> 	ds.tmp	<- rbind(ds.tmp, ds)
> 	save(ds.tmp,file=tmpFile, compress=T)
> 	cat(Cycle,': ',object.size(ds),object.size(ds.tmp),'\n')
> }
> ________ end example 1 __________
>
>
> platform i386-pc-solaris2.8
> arch     i386
> os       solaris2.8
> system   i386, solaris2.8
> status
> major    1
> minor    9.1
> year     2004
> month    06
> day      21
> language R
>
>
>
>
> How can I avoid to run in that memory problem? Any ideas are very
appreciated.
> Thank you in advance & kind regards,

If you are going to look at the memory usage you should use gc(), and
perhaps repeated calls to gc(), before checking the memory footprint.
This will force a garbage collection.

Also, you will probably save memory by treating your data frames as
lists and concatenating them, then converting the result to a data frame.