dtrace discuss - Feb 2008 - Project proposal for enhanced ustack/jstack

Hello,

The Hotspot JVM Runtime group would like to put forth a project proposal 
to improve the capabilities of DTrace''s ustack()/jstack() actions to 
better handle non-standard user stack frames.  This email will describe 
the background, the problem, and our proposed solution.  We''re 
interested in getting a sponsor for this work to help us with the 
process of getting the code into the solaris codebase, and we''re very 
interested in getting any design reviews or feedback on our approach. 
If there''s already a process in place for how to conduct projects like 
this, please let us know so that we can pick it up and go with it.

** Background

The dtrace ustack() command provides a mechanism for the tracer to view 
the user-land call stack of a particular thread.  It walks the call 
stack of the thread, examining each frame and providing a one-line 
summary.  A problem, however, exists when the frames on the stack do not 
have native ABI layout, which can occur for example, in Java 
applications run by the Hotspot JVM. In the JVM, Java-level frames are 
stored on the user call stack, but have a VM-defined frame layout.

The ustack helper mechanism helps solve this problem by allowing the
application to define and register a helper D routine which, if present,
will be invoked by the ustack() command to decode a frame into a single
string which can be displayed as part of the stack trace.

** The Problem

The Hotspot server compiler creates highly optimized code that is 
difficult to impossible to trace with the current ustack() or jstack() 
mechanisms.  One reason for this is that the code generated by the 
server compiler makes use of the frame pointer register as a general 
purpose register (on x86 architectures), thus a straightforward walking 
of the thread''s call stack will not work.  The other complication is 
that server compiler aggressively inlines code, so a single call record 
on the stack may in fact contain a number of Java activations.  In both 
cases, the information such as the frame size and the virtual frame 
information is encoded in the VM data structures, it just needs to be 
retrieved and used.

** Our Proposal

We propose to add two new "well-known" helpers to the ustack/jstack 
mechanism.  The first we''re tentatively calling 
"dtrace:helper:ustack:fixframe".  The fixframe helper, if it exists, 
would be invoked by dtrace from within dtrace_getufpstack() if it 
encounters a frame it cannot parse (due to an invalid frame pointer 
register, for instance).  The fixframe helper, defined by the 
application being traced, would return new values for the registers, 
which would allow dtrace to continue the stack walk.

To solve the problem of inlined frames requires a two-pronged approach.
Firstly, dtrace needs a way to determine if there are virtual inlined 
frames in a call frame, and then needs a way to display them.  We 
propose adding a additional variable value returned from ustack:helper 
invocation, named ''vframe_count''.  If the vframe_count is
greater than
0, then that indicates that the frame in question contains multiple 
virtual frames.  DTrace would then loop over the vframe_count (likely to 
some hard-set maximum limit) and invoke a new helper, 
''dtrace:helper:ustack:vstack'', which would be provided by the 
application.  The helper would be invoked with a variable containing the 
loop index, ''vframe_index'', and would return a string that
describes
that particular virtual (inlined) frame, similar to the traditional 
ustack helper.

We believe that these modifications to the ustack mechanism are safe,
efficient, backwards-compatible, and minimally-intrusive.  They would 
provide some flexibility to any traced application or system which uses 
non-standard stack frame layouts in their execution.  Though these 
modifications were designed with the Hotspot JVM server compiler in 
mind, we believe they would be generally useful by other dynamic 
language systems.

We would appreciate any feedback to this plan.  If there''s anything
that
looks unreasonable, please speak up now since we''d very much not like
to
spend time implementing this, and then run into friction when it''s time
to integrate into the codebase.  On that note, we''d like to reiterate 
our request for a sponsor that would help guide us through the design 
and implementation process.  And of course, if anyone else is looking 
for something to do and would like to help out, we would not turn you down!

--
The Hotspot Runtime Team