[dtrace-discuss] Getting total memory allocated so far

Jim Mauro
Thu, 03 Jan 2008 16:27:54 -0000

I hate to beat this to death, but it really sounds like you can make your
life simple and use prstat and pmap to track the information you want.

Heap memory usage tracking is tricky, since libc malloc may or may not
call into the kernel (sbrk) to satisfy a memory request. Also, tracking
malloc and keeping a running total in dtrace is dead easy, but tracking
free and correlating to how much space was freed is tricky, since
free takes an address, not a size. You could build an array of addresses
in a pid<PID:libc:malloc:return probe clause, but making use of it to
track free''s and sizes is tricky...

It really all gets down to the level of granularity and accuracy that you
require. prstat and pmap will provide the information you need with
reasonable accuracy - pmap will allow you to track heap segment sizes
(pmap -x <PID> | grep heap) - you could easily script something with
pmap and generate a summary report.

Finally, as Rickey pointed out, you could also write some C code that
uses procfs and grabs the desired data.

Thanks,
/jim


Lally Singh wrote:
> @Jeremy: yeah, I''m tracking how much memory it''s actually
using right
> now.. sbrk makes it complicated, but I figure that more advanced
> memory management may help, once I''ve measured how it''s
getting used
> today.  Via mmap() for large transient allocations and/or some
> smartness about allocating smartly to keep the working set of pages
> reasonable.
>
> @James: I saw that, but could I use it on a running program without
> stopping it?  The dtrace scripts don''t do much damage, and the
program
> still gets all its work done at ~60 Hz for its clients.  Single
> threaded, no less.
>
> Some more details: I was trying to keep this all a simple question,
> but with memory there rarely are simple questions.  My research is on
> the scalability of massively multiplayer online games.  So, I''ve
> written a load simulator which logs in from remote computers and
> starts playing the game.  I''ve got a bunch of dtrace scripts that
> measure the network, memory, and cpu usage as I add more simulated
> clients.  The scripts haven''t shown themselves to interfere
> significantly with performance (yet), but in exchange they put out
> data once a second, which another script reads & shoves into a db for
> me.
>
> I''d like to monitor memory usage without interfering with my other
> measurements.  Hence, I''d love to know something close to a global
var
> (which''d be great, malloc() already has to do some locking (for
normal
> malloc, haven''t seen umem), so why not?) or some way to quickly
> assemble a running total.  Right now all I''ve got is a pid
provider
> probe for malloc & free that reads the headers on the memory blocks
> they use.
>
> Thanks in advance everyone!
>
> On Jan 2, 2008 1:25 PM, James Carlson <james.d.carlson at sun.com>
wrote:
>
>> Lally Singh writes:
>>
>>> I''d like to track to total amount of memory the
application''s actually
>>> using right now.   More specifically, I guess these rules would do
it:
>>>
>>> 1. start at M=0
>>> 2. for each call to malloc(): set M += size of block returned by
malloc
>>> 3. for each call to free(): set M -= size of block freed.
>>>
>> Using "env LD_PRELOAD=libumem.so myapp" with mdb''s
umem support should
>> get you that (plus quite a bit more).
>>
>> --
>> James Carlson, Solaris Networking              <james.d.carlson at
sun.com>
>> Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442
2084
>> MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442
1677
>>
>>
>
>
>
>