This might be because I didn't get it right, but; I thought R would release memory back to the system as (big) objects get removed? Here is my platform (with 1Gb of RAM): platform sparc-sun-solaris2.8 arch sparc os solaris2.8 system sparc, solaris2.8 status major 1 minor 3.1 year 2001 month 08 day 31 language R A little example: Start a new section of R, with no objects:> ls()character(0)> gc()used (Mb) gc trigger (Mb) Ncells 187633 5.1 407500 10.9 Vcells 36061 0.3 786432 6.0>And the top commands shows: PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 18068 gardar 1 32 0 15M 12M sleep 0:02 0.01% R.bin Creating few objects:> x <- numeric(1e7) > gc()used (Mb) gc trigger (Mb) Ncells 187642 5.1 407500 10.9 Vcells 10036062 76.6 10394092 79.4> a <- x > gc()used (Mb) gc trigger (Mb) Ncells 187644 5.1 407500 10.9 Vcells 20036061 152.9 20394103 155.6> b <- x > gc()used (Mb) gc trigger (Mb) Ncells 187646 5.1 407500 10.9 Vcells 30036061 229.2 30394103 231.9>And at the end the top command shows: PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 18068 gardar 1 42 0 244M 241M sleep 0:08 3.47% R.bin Now, starting removing the objects:> rm(b) > gc()used (Mb) gc trigger (Mb) Ncells 187646 5.1 407500 10.9 Vcells 20036061 152.9 30394103 231.9> rm(a) > gc()used (Mb) gc trigger (Mb) Ncells 187652 5.1 407500 10.9 Vcells 10036073 76.6 30394103 231.9> rm(x) > gc()used (Mb) gc trigger (Mb) Ncells 187650 5.1 407500 10.9 Vcells 36073 0.3 24315282 185.6>But the top command shows: PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 18068 gardar 1 32 0 244M 241M sleep 0:09 0.17% R.bin A size of 244Mb (241Mb in memory). If I run gc() few times...> gc()used (Mb) gc trigger (Mb) Ncells 187657 5.1 407500 10.9 Vcells 36086 0.3 19452225 148.5> gc()used (Mb) gc trigger (Mb) Ncells 187657 5.1 407500 10.9 Vcells 36086 0.3 15561780 118.8>[RUNNING gc() FEW TIMES MORE]> gc()used (Mb) gc trigger (Mb) Ncells 187657 5.1 407500 10.9 Vcells 36086 0.3 786432 6.0>But the top command still shows: PID USERNAME THR PRI NICE SIZE RES STATE TIME CPU COMMAND 18068 gardar 1 32 0 244M 241M sleep 0:14 0.02% R.bin My question is; are gc() and unix top telling the same memory story? If not, which one is not getting it right? Cheers, Gardar ________________________________________________________ Gardar Johannesson Department of Statistics Ohio State University 304E Cockins Hall, 1958 Neil Av. Columbus, OH 43210 _________________________________________________________ -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
On Mon, 26 Nov 2001, Gardar Johannesson wrote:> This might be because I didn't get it right, but; I thought R would > release memory back to the system as (big) objects get removed? >You can give it back, but you can't force the OS to take it. This is from the comp.lang.c newsgroup's FAQ (http://www.eskimo.com/~scs/C-faq/top.html) --- Question 7.25 I have a program which mallocs and later frees a lot of memory, but memory usage (as reported by ps) doesn't seem to go back down. A: Most implementations of malloc/free do not return freed memory to the operating system (if there is one), but merely make it available for future malloc calls within the same program. ---- On the other hand, if the space isn't being used by R it will be paged out to virtual memory after a while, so it doesn't really matter whether it's reclaimed (unless you are very short on virtual memory). -thomas Thomas Lumley Asst. Professor, Biostatistics tlumley at u.washington.edu University of Washington, Seattle -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._
On Mon, 26 Nov 2001, Gardar Johannesson wrote:> This might be because I didn't get it right, but; I thought R would > release memory back to the system as (big) objects get removed? > > Here is my platform (with 1Gb of RAM): > > platform sparc-sun-solaris2.8 > arch sparc > os solaris2.8 > system sparc, solaris2.8 > status > major 1 > minor 3.1 > year 2001 > month 08 > day 31 > language R[...]> My question is; are gc() and unix top telling the same memory story? If > not, which one is not getting it right?No, so both are right in different ways. They are referring to very different things. gc() to the maximum amount of memory that R will use before garbage collection and the amount currently in use. top refers to the amount that the malloc system has marked as in use. In my experience on Solaris the latter very rarely goes down, but it does get swapped out. On my Linux system it goes down, but not by as much as it went up. R does release memory back to the malloc system: what happens thereafter is very machine (and even compiler) dependent. -- Brian D. Ripley, ripley at stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272860 (secr) Oxford OX1 3TG, UK Fax: +44 1865 272595 -.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.-.- r-help mailing list -- Read http://www.ci.tuwien.ac.at/~hornik/R/R-FAQ.html Send "info", "help", or "[un]subscribe" (in the "body", not the subject !) To: r-help-request at stat.math.ethz.ch _._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._._