dtrace discuss - Oct 2007 - post-mortem access to dtrace variables?

If I declare a global variable such as an associative array in a .d
script, and the system crashes while the script is running, is it possible
to retrieve the contents of that variable from the crash dump?

-- 
meem

Hey Peter,
> If I declare a global variable such as an associative array in a .d
> script, and the system crashes while the script is running, is it possible
> to retrieve the contents of that variable from the crash dump?
The data is certainly there -- it just depends on how badly you want to
get to it. ;)  Unfortunately, it''s not terribly easy to get to, but it
would be much easier if (for example) your D program only had a single
dynamic variable.

To show how to do this for the simple case, take the following program:

   dtrace -n BEGIN''{foobar["bippityboppity"] =
"boo"}''

First, find the dtrace_state_t that corresponds to this enabling (on
a live system in my example, from the dump in your case):

  > ::dtrace_state
              ADDR MINOR             PROC NAME                         FILE
  ffffff01d66ec780     2 ffffff01d4e17c70 akd              ffffff01cf5085e8
  ffffff01d6b3b540     3 ffffff01ea5e28d8 dtrace           ffffff01d899a318
 
In my case, it''s the second enabling.  Now you need to find the address
of the dtrace_dstate_t embedded in the dts_vstate:

  > ffffff01d6b3b540::print -at dtrace_state_t dts_vstate
  {
      ffffff01d6b3b578 dtrace_state_t *dts_vstate.dtvs_state = ...
      ...
      ffffff01d6b3b5a8 int dts_vstate.dtvs_nlocals = 0
-->   ffffff01d6b3b5b0 dtrace_dstate_t dts_vstate.dtvs_dynvars = {
          ffffff01d6b3b5b0 void *dtds_base = 0xffffff07e2ae1000
          ffffff01d6b3b5b8 size_t dtds_size = 0x100000
          ...
      }
  }

That''s your dynamic variable state.  Now you can walk all of your
dynamic
variables by feeding that to the dtrace_dynvar walker.  One problem with
the walker is that it gets fooled by the presence of DTRACE_DYNHASH_SINK
(I should have updated it when I fixed 6379717), so you''ll need to
shove
this walker in a pipeline to get to your allocated dynamic variables:

  > ffffff01d6b3b5b0::walk dtrace_dynvar | ::grep
".!=dtrace_dynhash_sink"

In my case, that yields just the one variable:

  ffffff07e2afe1c0

Print that out as a dtrace_dynvar_t:

  > ffffff07e2afe1c0::print -at dtrace_dynvar_t
  {
      ffffff07e2afe1c0 uint64_t dtdv_hashval = 0x89c0258fdf3058e7
      ffffff07e2afe1c8 struct dtrace_dynvar *dtdv_next = dtrace_dynhash_sink
      ffffff07e2afe1d0 void *dtdv_data = 0xffffff07e2afe210
      ffffff07e2afe1d8 dtrace_tuple_t dtdv_tuple = {
          ffffff07e2afe1d8 uint32_t dtt_nkeys = 0x2
          ffffff07e2afe1dc uint32_t dtt_pad = 0
          ffffff07e2afe1e0 dtrace_key_t [1] dtt_key = [
              {
                  ffffff07e2afe1e0 uint64_t dttk_value = 0xffffff07e2afe200
                  ffffff07e2afe1e8 uint64_t dttk_size = 0xf
              }
          ]
      }
  }

As this may suggest, your data is in dtdv_data; your keys are in dtdv_tuple.

So, in our case, the key is here:

  > 0xffffff07e2afe200,10::dump
                     \/ 1 2 3  4 5 6 7  8 9 a b  c d e f  v123456789abcdef
  ffffff07e2afe200:  62697070 69747962 6f707069 74790000  bippityboppity..

And the data is here:

  > 0xffffff07e2afe210,10::dump
                     \/ 1 2 3  4 5 6 7  8 9 a b  c d e f  v123456789abcdef
  ffffff07e2afe210:  626f6f00 00000000 00000000 00000000  boo.............

Apologies that this isn''t easier (a fancy dcmd would be most welcome
here), but that should allow you to at least get to your data...

	- Bryan

--------------------------------------------------------------------------
Bryan Cantrill, Solaris Kernel Development.       http://blogs.sun.com/bmc

> 
 > > If I declare a global variable such as an associative array in a .d
 > > script, and the system crashes while the script is running, is it
possible
 > > to retrieve the contents of that variable from the crash dump?
 > 
 > The data is certainly there -- it just depends on how badly you want to
 > get to it. ;)  Unfortunately, it''s not terribly easy to get to,
but it
 > would be much easier if (for example) your D program only had a single
 > dynamic variable.

Excellent --  thanks for the info, it''s a great help.

--
meem