dtrace discuss - Feb 2007 - Incorrect information showed by dtrace

I am using dtrace to check memory leaks in my code. I have put probe points for
malloc, calloc, realloc and free. It is basically for a telephony product that I
am working on.

I started dtrace after running my process. The output produced by dtrace for one
telephony call shows that one of my [b]initialization[/b] functions has been
invoked 295 times. But it is an initialization function and it gets called only
once during the lifetime of a phone call. I have verified that by adding a log
statement at the start of the function and checking the log output.

I am not sure if the output of dtrace is wrong or I am not probing the right way
?
 
 
This message posted from opensolaris.org

On 2/28/07, Nilendra Jain <nilendra at avaya.com>
wrote:
>
> I am using dtrace to check memory leaks in my code. I have put probe
> points for malloc, calloc, realloc and free. It is basically for a
telephony
> product that I am working on.
>
> I started dtrace after running my process. The output produced by dtrace
> for one telephony call shows that one of my [b]initialization[/b] functions
> has been invoked 295 times. But it is an initialization function and it
gets
> called only once during the lifetime of a phone call. I have verified that
> by adding a log statement at the start of the function and checking the log
> output.
>
> I am not sure if the output of dtrace is wrong or I am not probing the
> right way ?

more information is needed to diagnose this problem, for example if this is
a java or C++ or other object oriented language it is possible that it is
calling malloc and friends repeatedly in the background for hidden
constructors and copy constructors.

You can also print out the userstack (ustack())  on each call and see in
which context its being called so you can track down how its being called,
of course if they are all identical it could be a DTrace error.

James Dickens
uadmin.blogspot.com



This message posted from opensolaris.org
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070228/4b94b3df/attachment.html>

On 2/28/07, Nilendra Jain <nilendra at avaya.com>
wrote:
> I am using dtrace to check memory leaks in my code. I have put probe points
for malloc, calloc, realloc and free. It is basically for a telephony product
that I am working on.
>
> I started dtrace after running my process. The output produced by dtrace
for one telephony call shows that one of my [b]initialization[/b] functions has
been invoked 295 times. But it is an initialization function and it gets called
only once during the lifetime of a phone call. I have verified that by adding a
log statement at the start of the function and checking the log output.
>
> I am not sure if the output of dtrace is wrong or I am not probing the
right way ?
Any reason your not using libumem to look for memory leaks in your
code? I think that might be a better tool for this given job.

Thanks,
- Ryan
-- 
UNIX Administrator
http://prefetch.net

It is a C++ application. I have verified that this function is not calling
malloc and friends repeatedly. It is just allocating 2 structures.

This function is called at the start of every phone call. So logically there
should be just 2 instances of this function per call (one for each structure) in
the dtrace file. Surprisingly after one call I find that there are 295
occurrences of this function after one call and also after 10 calls the number
remains the same.

I am printing the userstack using ustack. In all the cases the stack is almost
the same, but the size of the memory allocated varies and also the pointer
address changes.
 
 
This message posted from opensolaris.org

I have already used libumem. Unfortunately all the leaks it is showing are
actually areas in code which holds initialization data, which will remain in
memory till the end of the process.

Another observation is that when I run my process under libumem, the memory
usage shoots to 255 MB in a very short time. But in normal scenarios it consumes
about 77 MB of memory.

I have also tried using watchmalloc library. But that has not helped either.
When I ran my process with watchmalloc library my process started consuming
about 62% CPU. Normally it shouldnt be taking any CPU, as it is dormant waiting
to answer a call. After reaching about 62% CPU usage, when I tried to make a
phone call to my process, it does not pick up the call and is sort of hung not
doing anything, but the CPU usage shoots up further to 70%.
 
 
This message posted from opensolaris.org

Some more information:
My process is running on Solaris 10, Update 3. We are using Sun Studio 11 for
building our code.
 
 
This message posted from opensolaris.org

>I have already used libumem. Unfortunately all the leaks it is showing
>are actually areas in code which holds initialization data, which will
>remain in memory till the end of the process.
Why do you think they''re not leaks?  libumem claims they are
not reachable if they are reported as leaks.  If you can''t
reach it, you might as well free it.

>Another observation is that when I run my process under libumem, the
>memory usage shoots to 255 MB in a very short time. But in normal
>scenarios it consumes about 77 MB of memory.
libumem allocates memory differently.



Watchmalloc is mostly useful when you have memory corruption issues.

Casper

Nilendra Jain wrote:
> It is a C++ application. I have verified that this function is not calling
malloc and friends repeatedly. It is just allocating 2 structures.
> 
> This function is called at the start of every phone call. So logically
there should be just 2 instances of this function per call (one for each
structure) in the dtrace file. Surprisingly after one call I find that there are
295 occurrences of this function after one call and also after 10 calls the
number remains the same.
> 
> I am printing the userstack using ustack. In all the cases the stack is
almost the same, but the size of the memory allocated varies and also the
pointer address changes.
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
Keep in mind that C++ does much magic... pass the -S
option to the compiler and examine the generated aseembler.
That''s really the only way to grok what''s happening.


- Bart


-- 
Bart Smaalders			Solaris Kernel Performance
barts at cyber.eng.sun.com		http://blogs.sun.com/barts