On 2/20/2015 4:04 AM, Konstantin Belousov wrote:> On Thu, Feb 19, 2015 at 04:04:53PM -0600, Matthew Grooms wrote: >> All, >> >> I have a multi-threaded program that runs on 10.1-RELEASE-p5. It starts >> out with a reasonable footprint but there is obviously a resource leak >> as the program uses substantially more memory over time ... >> >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 51560 rdj 3 20 0 46676K 7500K select 1 0:00 0.00% dialyd >> >> ... 24h later ... >> >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 51560 rdj 3 20 0 131M 27064K select 3 1:45 0.00% dialyd >> >> After a bit of debugging, I determined that it only happens when threads >> are created and then later destroyed. Valgrind thinks that the resources >> are being leaked from libthr itself ... > Threading library caches thread structures and some related objects. > This is needed to correctly handle kernel notifications about threads > exit and to avoid thread id reuse, besides usual argument for using cache > to improve creation speed. > > Also, the thread stacks are cached, each stack being 2MB probably accounts > for most of the memory usage columns in the top output above.Konstantin, Thanks for the reply. That seems reasonable. But surely these resources need to be reclaimed at some point after a thread gracefully exits. Otherwise any software that creates short lived threads will eventually run out of system resources and periodically require a restart. The test program I included does nothing but create threads that gracefully exit. Do you have another explanation as to why a program would indefinitely grow in size that way? Thanks, -Matthew
On Fri, 20 Feb 2015, Matthew Grooms wrote:> On 2/20/2015 4:04 AM, Konstantin Belousov wrote: >> Threading library caches thread structures and some related objects. >> This is needed to correctly handle kernel notifications about threads >> exit and to avoid thread id reuse, besides usual argument for using cache >> to improve creation speed. >> >> Also, the thread stacks are cached, each stack being 2MB probably accounts >> for most of the memory usage columns in the top output above. > > Konstantin, > > Thanks for the reply. That seems reasonable. But surely these resources need > to be reclaimed at some point after a thread gracefully exits. Otherwise any > software that creates short lived threads will eventually run out of system > resources and periodically require a restart. The test program I included > does nothing but create threads that gracefully exit. Do you have another > explanation as to why a program would indefinitely grow in size that way?When a thread is created, it will first try to reuse cached resources before allocating new resources. If you are creating 200 threads, for instance, try destroying those 200 threads, then create 200 new threads. You shouldn't see much change in resources, as libpthread should use the cached resources. If you see a double in the amount of resources used, then that would seem like a bug. -- DE