Hi mailing list and R-core. Could someone from R-core please help me to create account in bugzilla? I would like to submit issue related to gc() to wishlist. Related context is here - https://stat.ethz.ch/pipermail/r-devel/2017-July/074715.html -- Regards Dmitriy Selivanov [[alternative HTML version deleted]]
>>>>> Dmitriy Selivanov <selivanov.dmitriy at gmail.com> >>>>> on Fri, 11 Aug 2017 17:33:31 +0400 writes:> Hi mailing list and R-core. Could someone from R-core please help me to > create account in bugzilla? I would like to submit issue related to gc() to > wishlist. I will create one. Your previous e-mails left me pretty clueless about what the problem is that you want to solve ... but maybe others understand better what you mean. Note that in the case of such a relatively sophisticated wish without a clear sign of a problem (in my view) chances are not high that anything will change, unless someone provides a (small footprint) patch towards the (R-devel aka "trunk") sources *and* reproducible R code that depicts the problem. Still: Thank you for trying to make R better by contributing with careful bug reports ! Best, Martin > Related context is here - > https://stat.ethz.ch/pipermail/r-devel/2017-July/074715.html > -- > Regards > Dmitriy Selivanov > [[alternative HTML version deleted]] > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel
Thanks Martin, I've received invitation and will create ticket soon. Regarding issue - basically the problem is that on operating systems which use glibc memory is not freed (R releases it, but system doesn't trim it). Setting corresponding environment variable (MALLOC_TRIM_THRESHOLD_) doesn't help, but manual call to `*malloc_trim*` does. Implication is following - for long-running jobs linux OOM killer can kill R process because highly-fragmented pieces of memory never returned back to OS. Very simple example: 1. I have 2gb of RAM on the machine. 2. I create large list of small objects which occupies 1.5 gig. 3. I remove it (even can manually call gc()) - "top" still shows 1.5 gig occupied. If I decide to recreate similar 1.5gig list - ram will be reused. 4. However if after removal of the list I will decide to create normal continuous integer/double vector of size 500+ mb - R will report that it can't allocate vector of such size. Hope this helped. 2017-08-11 18:00 GMT+04:00 Martin Maechler <maechler at stat.math.ethz.ch>:> >>>>> Dmitriy Selivanov <selivanov.dmitriy at gmail.com> > >>>>> on Fri, 11 Aug 2017 17:33:31 +0400 writes: > > > Hi mailing list and R-core. Could someone from R-core please help me > to > > create account in bugzilla? I would like to submit issue related to > gc() to > > wishlist. > > I will create one. > > Your previous e-mails left me pretty clueless about what the > problem is that you want to solve ... but maybe others > understand better what you mean. > > Note that in the case of such a relatively sophisticated wish > without a clear sign of a problem (in my view) > chances are not high that anything will change, unless someone > provides a (small footprint) patch towards the (R-devel aka > "trunk") sources *and* reproducible R code that depicts the > problem. > > Still: Thank you for trying to make R better by contributing with > careful bug reports ! > > Best, > Martin > > > > Related context is here - > > https://stat.ethz.ch/pipermail/r-devel/2017-July/074715.html > > > > -- > > Regards > > Dmitriy Selivanov > > > [[alternative HTML version deleted]] > > > ______________________________________________ > > R-devel at r-project.org mailing list > > https://stat.ethz.ch/mailman/listinfo/r-devel >-- Regards Dmitriy Selivanov [[alternative HTML version deleted]]
2017-08-11 16:00 GMT+02:00 Martin Maechler <maechler at stat.math.ethz.ch>:>>>>>> Dmitriy Selivanov <selivanov.dmitriy at gmail.com> >>>>>> on Fri, 11 Aug 2017 17:33:31 +0400 writes: > > > Hi mailing list and R-core. Could someone from R-core please help me to > > create account in bugzilla? I would like to submit issue related to gc() to > > wishlist. > > I will create one. > > Your previous e-mails left me pretty clueless about what the > problem is that you want to solve ... but maybe others > understand better what you mean. > > Note that in the case of such a relatively sophisticated wish > without a clear sign of a problem (in my view) > chances are not high that anything will change, unless someone > provides a (small footprint) patch towards the (R-devel aka > "trunk") sources *and* reproducible R code that depicts the > problem.How to reproduce it: a <- replicate(2e6, new.env()) # ~ 1.4 GB of memory rm(a) gc() # the R process still has the memory assigned I?aki