dtrace discuss - Oct 2008 - RFE: if/else control flow in probes

Hi all,

There''s a dead thread about this back at
http://forums.sun.com/thread.jspa?threadID=5075157, but it would be really nice
to have support for if/else control flow in actions.

Reasons: 
1. The ternary operator ?: is already there (though it doesn''t let you
do anything with side effects)
2. The user can already write arbitrary and expensive non-cacheable predicates
if they want, so performance shouldn''t go down as a result.
3. For performance reasons it would be nice to push expensive non-cacheable
predicates inside the action so the cacheable ones can still do their job
efficiently (see http://docs.sun.com/app/docs/doc/817-6223/chp-perf-3?a=view)

For example, I have a dtrace script that prints out a message whenever a thread
gets preempted during certain (< 1usec long) critical sections. To avoid the
overhead of invoking dtrace every time they enter/exit (~6usec), at thread
startup they pass dtrace a pointer to a variable which they can then update
directly; dtrace checks it during context switches:

sched:::off-cpu
/ pid == $1 && self->user_ptr != 0 && *(int*)
copyin(self->usr_ptr, sizeof(int)) != 0 /
{
/* print notification message */
}

The first two predicates are cacheable and eliminate virtually all of the
threads in the system, while the third predicate is uncacheable and only
required occasionally. To minimize the cost of this probe it would be very nice
to be able to write:

sched:::off-cpu
/ pid == $1 && self->user_ptr != 0 /
{
if( *(int*) copyin(self->usr_ptr, sizeof(int)) != 0)
    /* print notification message */
}

Alternatively, a smarter compiler might push expensive non-cached predicates
inside the action without exposing any API changes to the user. That might
actually be better because then the user can write predicates without having to
worry quite so much about whether they''re cacheable or not.
--
This message posted from opensolaris.org

>From "Learning DTrace -- Part 3: Advanced Scripting and
Aggregations"
http://www.laurustech.com/tech_paper.asp?ID=102

"... DTrace requires that each probe clause produce a fixed-length trace
record. This means that the total number of bytes generated by all of
the recording actions (trace, printf, etc.), for one clause, must be
constant. Each clause can generate a different size record, but one
clause must generate the same size record each time it is invoked. If
the designers allowed flow control statements, the size of the trace
record could vary, depending on whether the if condition was true."

Chip
> -----Original Message-----
> From: dtrace-discuss-bounces at opensolaris.org [mailto:dtrace-discuss-
> bounces at opensolaris.org] On Behalf Of Ryan
> Sent: Wednesday, October 01, 2008 6:48 AM
> To: dtrace-discuss at opensolaris.org
> Subject: [dtrace-discuss] RFE: if/else control flow in probes
> 
> Hi all,
> 
> There''s a dead thread about this back at
> http://forums.sun.com/thread.jspa?threadID=5075157, but it would be
> really nice to have support for if/else control flow in actions.
> 
> Reasons:
> 1. The ternary operator ?: is already there (though it doesn''t let
you
> do anything with side effects)
> 2. The user can already write arbitrary and expensive non-cacheable
> predicates if they want, so performance shouldn''t go down as a
result.
> 3. For performance reasons it would be nice to push expensive non-
> cacheable predicates inside the action so the cacheable ones can still
> do their job efficiently (see http://docs.sun.com/app/docs/doc/817-
> 6223/chp-perf-3?a=view)
> 
> For example, I have a dtrace script that prints out a message whenever
> a thread gets preempted during certain (< 1usec long) critical
> sections. To avoid the overhead of invoking dtrace every time they
> enter/exit (~6usec), at thread startup they pass dtrace a pointer to a
> variable which they can then update directly; dtrace checks it during
> context switches:
> 
> sched:::off-cpu
> / pid == $1 && self->user_ptr != 0 && *(int*)
copyin(self->usr_ptr,
> sizeof(int)) != 0 /
> {
> /* print notification message */
> }
> 
> The first two predicates are cacheable and eliminate virtually all of
> the threads in the system, while the third predicate is uncacheable
and
> only required occasionally. To minimize the cost of this probe it
would
> be very nice to be able to write:
> 
> sched:::off-cpu
> / pid == $1 && self->user_ptr != 0 /
> {
> if( *(int*) copyin(self->usr_ptr, sizeof(int)) != 0)
>     /* print notification message */
> }
> 
> Alternatively, a smarter compiler might push expensive non-cached
> predicates inside the action without exposing any API changes to the
> user. That might actually be better because then the user can write
> predicates without having to worry quite so much about whether
they''re
> cacheable or not.
> --
> This message posted from opensolaris.org
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org

> "... DTrace requires that each probe clause produce a fixed-length
trace
> record. This means that the total number of bytes generated by all of
> the recording actions (trace, printf, etc.), for one clause, must be
> constant. Each clause can generate a different size record, but one
> clause must generate the same size record each time it is invoked. If
> the designers allowed flow control statements, the size of the trace
> record could vary, depending on whether the if condition was true."
> Chip
Is this really true (that clauses cause different-sized records)? Why
don''t you simply, automatically null (or space) pad the trace record of
the smaller if condition(s)?

Or, in the case of user-based functions (which could also screw up this
requirement) require that the user pass an argument to the his/her function
indicating how many bytes that it will return?

Paul
--
This message posted from opensolaris.org

> Is this really true (that clauses cause different-sized records)? Why
don''t
> you simply, automatically null (or space) pad the trace record of the
smaller
> if condition(s)?
Yes it''s true. Read through the header comments in dtrace.c for more on
why.

Adam

-- 
Adam Leventhal, Fishworks                     http://blogs.sun.com/ahl

>> Is this really true (that clauses cause different-sized records)? Why
don''t
>> you simply, automatically null (or space) pad the trace record of the
smaller
>> if condition(s)?
> Yes it''s true. Read through the header comments in dtrace.c for
more on why.
> Adam
Could you possibly give some pointers to finding the source code for dtrace.c?
Last time I checked, they weren''t online.

Anyways, my point was that if you force them to be the same size, like a union,
they are the same size. Is there something you are seeing that I''m
missing here?

Paul
--
This message posted from opensolaris.org

> Could you possibly give some pointers to finding the source code for
dtrace.c?
> Last time I checked, they weren''t online.
Like the rest of OpenSolaris, you can find the source code on opensolaris.org.

... and you might want to check more throroughly or more frequently: DTrace
has been open source since January 2005. :-)

Adam

-- 
Adam Leventhal, Fishworks                     http://blogs.sun.com/ahl

>> Like the rest of OpenSolaris, you can find the source code on
opensolaris.org.
> ... and you might want to check more throroughly or more frequently: DTrace
> has been open source since January 2005. :-)
> Adam
Fair enough - went to opensolaris.org, clicked on ''source
browser'', looked for either an ''all'' or a
''dtrace'' project, didn''t find it. There is a
dtrace-doc however, which doesn''t have source.

so.. am I looking in the wrong place? If not, how do I see this file?

Paul

(
you didn''t answer my question, btw.. do you see anything unworkable
with adding a predicate to conditionals and/or user functions to force them to
have the same size trace record?
)
--
This message posted from opensolaris.org

> Fair enough - went to opensolaris.org, clicked on ''source
browser'', looked for
> either an ''all'' or a ''dtrace'' project,
didn''t find it. There is a dtrace-doc
> however, which doesn''t have source.
Google, "dtrace.c"
> you didn''t answer my question, btw.. do you see anything
unworkable with
> adding a predicate to conditionals and/or user functions to force them to
have
> the same size trace record?
You''re asking a deep question involving the internals of DTrace. Start
with
the code.

Adam

-- 
Adam Leventhal, Fishworks                     http://blogs.sun.com/ahl

> Fair enough - went to opensolaris.org, clicked on ''source
browser'', looked for either an ''all'' or a
''dtrace'' project, didn''t find it. There is a
dtrace-doc however, which doesn''t have source.
> 
> so.. am I looking in the wrong place? If not, how do I see this file?
The file is here:

 
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/dtrace/dtrace.c

More DTrace-related files can be found by searching for "dtrace" in
the
file path.
> (
> you didn''t answer my question, btw.. do you see anything
unworkable with adding a predicate to conditionals and/or user functions to
force them to have the same size trace record?
> )
To the degree that this parses, I think it''s safe to say that finding
the
source is the least of the problems. ;)

	- Bryan

--------------------------------------------------------------------------
Bryan Cantrill, Sun Microsystems Fishworks.       http://blogs.sun.com/bmc

> To the degree that this parses, I think it''s safe to say that
finding the
> source is the least of the problems. ;)
> - Bryan
Argh... yes, I realize that the previous sentence was compressed a little too
much for comfort, but no, it''s not line noise..

I''m going off the statement:

"... DTrace requires that each probe clause produce a fixed-length trace
record. This means that the total number of bytes generated by all of
the recording actions (trace, printf, etc.), for one clause, must be
constant. Each clause can generate a different size record, but one
clause must generate the same size record each time it is invoked. If
the designers allowed flow control statements, the size of the trace
record could vary, depending on whether the if condition was true."


The way I parse this is that this limitation makes the following not work

if (cond) { printf("a"); } else { printf("aa"); }

because the first creates a trace 1 byte long, the second 2 bytes.

My suggestion to make this work was to force the trace record to be 2 bytes long
(null padding or truncating the end as necessary)  no matter what  condition was
reached:

{
    printf.length = 2;
    if (cond) { printf("a"); } else { printf("aa"); }
}

or better yet, have the dtrace interpreter figure this length out if it can, and
warn if it can''t. This would allow for a much more expressive dtrace,
and would answer everyone''s request for conditionals and user
functions.

Paul
--
This message posted from opensolaris.org

> Argh... yes, I realize that the previous sentence was compressed a little
too much for comfort, but no, it''s not line noise..
> 
> I''m going off the statement:
> 
> "... DTrace requires that each probe clause produce a fixed-length
trace
> record. This means that the total number of bytes generated by all of
> the recording actions (trace, printf, etc.), for one clause, must be
> constant. Each clause can generate a different size record, but one
> clause must generate the same size record each time it is invoked. If
> the designers allowed flow control statements, the size of the trace
> record could vary, depending on whether the if condition was true."
> 
> 
> The way I parse this is that this limitation makes the following not work
> 
> if (cond) { printf("a"); } else { printf("aa"); }
> 
> because the first creates a trace 1 byte long, the second 2 bytes.
Well, this is a bad example, as these actually store the same amount
of data.  Regardless, quantity of data is actually the easier bit
of this problem; it is type of data that is more challenging.

That is:

  if (cond) { ustack(); } else { printf("Cond is false.\n");
printa(@); }

Is it possible to solve this?  Yes, of course -- it''s just software
after
all.  But does it make the system substantially more complicated for what
amounts to syntactic sugar?  Yes, again.  And by the way:  yes, you
can envision ways to tell lies to get this to work (e.g., by putting
each potential execution in its own clause with an appropriate
predicate), but again, it''s complicated and (like many lies), the truth
will leak which will require patches of yet more lies.

We have what we have because it''s simple in both model and
implementation --
and there is (in my opinion) tremendous value in simplicity wherever it
can be reasonably had. 

	- Bryan

--------------------------------------------------------------------------
Bryan Cantrill, Sun Microsystems Fishworks.       http://blogs.sun.com/bmc

> Is it possible to solve this? Yes, of course -- it''s just software
after
> all. But does it make the system substantially more complicated for what
> amounts to syntactic sugar? Yes, again. And by the way: yes, you
> can envision ways to tell lies to get this to work (e.g., by putting
> each potential execution in its own clause with an appropriate
> predicate), but again, it''s complicated and (like many lies), the
truth
> will leak which will require patches of yet more lies.
Yes, and so the question is whether or not the ''lies'' (I
prefer compiler directives/features ;-)) are worth the effort, in the sense that
they promote end-user efficiency. And a simple pragma that tells a predicate how
many bytes to return is not all that onerous from a end-user''s point of
view. (and from a dtrace-programming perspective I would guess, too, if done
correctly, ie: not too fancy)

So I put it to you that they are worth it, simply because the
''lies'' as you say allow for first party, and third party
pluggable modules for dtrace, *written in dtrace*.

This last point IMO is a big deal. Yes, you can have code generation GUIs that
generate d scripts, and I suppose you could have a language on top of d which
generates an underlying d script (d with conditionals), but those are far bigger
lies from the end-user''s point of view - what you lose in the process
is both simplicity and clarity.

I''d love to be able to program reusable objects for d - it increases
the bang for my programming buck, and the GUI/uber-d approach is bad in the long
run because it locks people into third party solutions (including their own),
leads to read-only scripts (have you ever tweaked outputted code, only to find
that that tweak makes it into production?), and in general is a *bad thing*.

So limiting this stunts what could otherwise be a great win for consumers of
your product, and a community that could write modules for you. And it limits
how user friendly dtrace could be.

Just MO.

Paul
--
This message posted from opensolaris.org