dtrace discuss - May 2009 - Collecting Long-Term Performance Data

I''m in the process of writing a daemon that uses libdtrace to collect
performance data continuously.  The only real problem I''ve run into is
figuring out how to limit the amount of data I''m collecting.

I obviously don''t need to know about every single IO or network event,
just a sampling.  The solution that I''m leaning towards is to have the
daemon start and stop dtrace instrumenting at regular intervals.

In order to avoid an appreciable performance impact from using dtrace
for this type of work, does it make sense to start and stop dtrace all
the time, or is it better to keep it running and throw away the data
points I don''t want?

-- 
Peter Jones, http://pmade.com
pmade inc.  Louisville, CO US

On 05/29/09 08:45, Peter Jones wrote:
> I''m in the process of writing a daemon that uses libdtrace to
collect
> performance data continuously.  The only real problem I''ve run
into is
> figuring out how to limit the amount of data I''m collecting.
> 
> I obviously don''t need to know about every single IO or network
event,
> just a sampling.  The solution that I''m leaning towards is to have
the
> daemon start and stop dtrace instrumenting at regular intervals.
> 
> In order to avoid an appreciable performance impact from using dtrace
> for this type of work, does it make sense to start and stop dtrace all
> the time, or is it better to keep it running and throw away the data
> points I don''t want?
wouldn''t it make more sense to use kstats for the long-term data 
collection? after all, that''s what they''re there for.

(I know, probably not what you wanted to hear ;-)

Michael
-- 
Michael Schuster 	http://blogs.sun.com/recursion
Recursion, n.: see ''Recursion''

Michael Schuster wrote:
> On 05/29/09 08:45, Peter Jones wrote:
>> I''m in the process of writing a daemon that uses libdtrace to
collect
>> performance data continuously.  The only real problem I''ve run
into is
>> figuring out how to limit the amount of data I''m collecting.
>>
>> I obviously don''t need to know about every single IO or
network event,
>> just a sampling.  The solution that I''m leaning towards is to
have the
>> daemon start and stop dtrace instrumenting at regular intervals.
>>
>> In order to avoid an appreciable performance impact from using dtrace
>> for this type of work, does it make sense to start and stop dtrace all
>> the time, or is it better to keep it running and throw away the data
>> points I don''t want?
Depends on the interval. Starting a DTrace script through libdtrace(3) 
has a slight delay, if that added time doesn''t interfere with your 
interval, I think you should stop/start DTrace to minimize the impact.
> wouldn''t it make more sense to use kstats for the long-term data 
> collection? after all, that''s what they''re there for.
> 
> (I know, probably not what you wanted to hear ;-)
> 
> Michael
I agree with Michael, but depends on whether the data you want is 
already exported through some interface or not.

Rafael

On Fri, May 29, 2009 at 09:45:26AM -0600, Peter Jones
wrote:
> I''m in the process of writing a daemon that uses libdtrace to
collect
> performance data continuously.  The only real problem I''ve run
into is
> figuring out how to limit the amount of data I''m collecting.
Have you considered aggregations?

Cheers,
- jonathan