Robert Milkowski
2006-Mar-01 08:58 UTC
[dtrace-discuss] Getting all current dtrace consumers
Hello dtrace-discuss,
I belive it would be useful if one could get listing of all current
run dtrace scripts with basic info (who did run it, what time, what
probes are enabled, etc.). Additionally with easy way of stopping
selected script (killing user process probably should do it).
We''ve got some production systems where sys-admins and developers
are running their own scripts at the same time.
--
Best regards,
Robert mailto:rmilkowski at task.gda.pl
http://milek.blogspot.com
Robert Milkowski wrote:> Hello dtrace-discuss, > > I belive it would be useful if one could get listing of all current > run dtrace scripts with basic info (who did run it, what time, what > probes are enabled, etc.). Additionally with easy way of stopping > selected script (killing user process probably should do it). > > We''ve got some production systems where sys-admins and developers > are running their own scripts at the same time. >I would start with the ::dtrace_state dcmd in mdb: # echo "::help dtrace_state" |mdb -k NAME dtrace_state - print active DTrace consumers SYNOPSIS addr ::dtrace_state DESCRIPTION Given a dtrace_state_t structure, displays all consumers, or "<anonymous>" if the consumer is anonymous. If no state structure is provided, iterates over all state structures. Addresses in ADDR column may be provided to ::dtrace to obtain dtrace(1M)-like output for in-kernel DTrace data. ATTRIBUTES Target: kvm Module: dtrace Interface Stability: Unstable # echo "::dtrace_state" |mdb -k ADDR MINOR PROC NAME FILE ffffffff815b0000 2 ffffffff837ac4d0 dtrace ffffffff84cc8b68 From thee you''ll need to drill down into the proc_t structure to get pids, uids, etc. Jim
Hi Robert,
The may be other ways of doing this but you can use mdb(1). To get a
list of all consumers currently running you can use the ::dtrace_state
dcmd:
# mdb -k
Loading modules: [ unix krtld genunix specfs dtrace ufs ip sctp usba
random fcp fctl nca lofs nfs sppp crypto ptm ipc logindmux fcip md ]
> ::dtrace_state
ADDR MINOR PROC NAME FILE
ffffffff87a38800 2 ffffffff877d7d18 dtrace ffffffff8988dee8
ffffffff8998b000 3 ffffffff877dc898 dtrace ffffffff8876d510
You could then use the proc_t in whatever way you want:
> ::dtrace_state ! tail +2 | awk ''{print $3}'' > /tmp/l
> ::cat /tmp/l |::ps
S PID PPID PGID SID UID FLAGS ADDR NAME
R 9718 9711 9711 9711 0 0x42004000 ffffffff877d7d18 dtrace
R 9768 9764 9764 9764 0 0x42004000 ffffffff877dc898 dtrace
To dredge out the probes that are enabled by a given consumer you can
use the dtrace_state_t pointer (obtained from ::dtrace_state above) and
first
pull out the pointer to the ecbs (enabling control blocks) and the
number of ecb''s.
We can then extract the probe information from each ecb:
> ffffffff87a38800::print dtrace_state_t dts_ecbs dts_necbs
dts_ecbs = 0xffffffff89011600
dts_necbs = 0x8
> 0xffffffff89011600::array uintptr_t 8 | ::print uintptr_t |::grep
".!=0" |::pr
int dtrace_ecb_t dte_probe | ::grep ".!=0"|::print dtrace_probe_t
dtpr_func dtpr
_name
dtpr_func = 0xffffffff812671f8 "resume"
dtpr_name = 0xffffffff84adddb8 "on-cpu"
dtpr_func = 0xffffffff8de618c0 "resume_from_zombie"
dtpr_name = 0xffffffff86732478 "on-cpu"
dtpr_func = 0xffffffff8894cb00 "resume_from_intr"
dtpr_name = 0xffffffff86732468 "on-cpu"
dtpr_func = 0xffffffff86b8e1b0 ""
dtpr_name = 0xffffffff86bf46b8 "ERROR"
You could obviously script this up to make it pretty.
Cheers.
Jon.
>Hello dtrace-discuss,
>
> I belive it would be useful if one could get listing of all current
> run dtrace scripts with basic info (who did run it, what time, what
> probes are enabled, etc.). Additionally with easy way of stopping
> selected script (killing user process probably should do it).
>
> We''ve got some production systems where sys-admins and developers
> are running their own scripts at the same time.
>
>
>
CHIAN-PHON LIN
2006-Mar-01 17:02 UTC
[dtrace-discuss] No attach/init/setup in anonymous tracing
Hi,
I tried simple anonymous tracing steps. But did not
yield attach/init/setup. Am I missing something?
Below are the output:
# nm -n /platform/sun4u/kernel/drv/sparcv9/pcf8584 | grep FUNC | grep -v
UNDEF | sed ''s/^.*|//''
_fini
_info
_init
pcf8584_acquire
pcf8584_attach
pcf8584_bbn_ready
pcf8584_bus_ctl
pcf8584_close
pcf8584_detach
pcf8584_dip_to_addr
pcf8584_doattach
pcf8584_dodetach
pcf8584_do_polled_io
pcf8584_error
pcf8584_free_regs
pcf8584_get_s0
pcf8584_get_s1
pcf8584_give_up
pcf8584_init
pcf8584_initchild
pcf8584_intr
pcf8584_ioctl
pcf8584_monitor_mode
pcf8584_open
pcf8584_print
pcf8584_process
pcf8584_put_s0
pcf8584_put_s1
pcf8584_release
pcf8584_reportdev
pcf8584_resume
pcf8584_select_bus
pcf8584_setup_regs
pcf8584_suspend
pcf8584_take_over
pcf8584_transfer
pcf8584_type_to_state
pcf8584_uninitchild
# dtrace -Aqn ''pcf8584::entry{@[probefunc] = count();}''
dtrace: cleaned up old anonymous enabling in /kernel/drv/dtrace.conf
dtrace: cleaned up forceload directives in /etc/system
dtrace: saved anonymous enabling in /kernel/drv/dtrace.conf
dtrace: added forceload directives to /etc/system
dtrace: run update_drv(1M) or reboot to enable changes
# reboot
Connection to dt88-115 closed.
sr1-umpk-07{cplin}118: rsh dt88-115 -l root
Password:
Last login: Wed Mar 1 08:11:25 from sr1-umpk-07.SFB
Sun Microsystems Inc. SunOS 5.10 Generic Patch Nov. 17, 2005
SunOS Internal Development: gk 2005-11-17 [G10U1B19-051117-daily]
bfu''ed from
/net/on10-patch.sfbay/export/pkgs_archives/daily_builds/G10U1B19-051117-daily/archives/sparc/nightly
on 2006-01-25
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
# dtrace -ae
pcf8584_get_s0 2
pcf8584_type_to_state 2
pcf8584_initchild 8
pcf8584_error 11
pcf8584_intr 11
pcf8584_process 11
pcf8584_dip_to_addr 22
pcf8584_bus_ctl 24
pcf8584_acquire 500
pcf8584_transfer 500
pcf8584_bbn_ready 501
pcf8584_select_bus 501
pcf8584_release 505
pcf8584_put_s1 507
pcf8584_put_s0 509
pcf8584_get_s1 512
pcf8584_print 1040
#
--
************************************************
* C P Lin, Common Technology Project Lead. *
* Sun Microsystems Inc. *
* E-Mail: c.lin at sun.com *
* Address: 4150 Network Circle, M/S UMPK12-330 *
* Santa Clara, CA 95054 *
* Phone: 650/352-4967 Fax: 650/786-7816 *
************************************************
Hi,
Is it possible that a return occurs when making a call? For example,
unix:sfmmu_hat_exit() calls mutex_exit at offset 0xc (ie. 12) and we
know call is not the same thing as ret. However, tracing return of
sfmmu_hat_exit also yield return offset 0xc (ie. 12). Below are the
outputs:
# /usr/ccs/bin/dis -F sfmmu_hat_exit /platform/sun4u/kernel/sparcv9/unix
**** DISASSEMBLER ****
disassembly for /platform/sun4u/kernel/sparcv9/unix
section .text
sfmmu_hat_exit()
sfmmu_hat_exit: 90 90 00 08 orcc %g0, %o0, %o0
sfmmu_hat_exit+0x4: 02 60 00 04 be,pn %xcc, sfmmu_hat_exit+0x14
sfmmu_hat_exit+0x8: 82 10 00 0f mov %o7, %g1
sfmmu_hat_exit+0xc: 40 00 42 b8 call mutex_exit
sfmmu_hat_exit+0x10: 9e 10 00 01 mov %g1, %o7
sfmmu_hat_exit+0x14: 81 c3 e0 08 retl
sfmmu_hat_exit+0x18: 01 00 00 00 nop
# dtrace -qn ''unix:sfmmu_hat_exit:return{@[probefunc, arg0] =
count();}''
^C
sfmmu_hat_exit 20 27
sfmmu_hat_exit 12 754
#
--
************************************************
* C P Lin, Common Technology Project Lead. *
* Sun Microsystems Inc. *
* E-Mail: c.lin at sun.com *
* Address: 4150 Network Circle, M/S UMPK12-330 *
* Santa Clara, CA 95054 *
* Phone: 650/352-4967 Fax: 650/786-7816 *
************************************************
You''ve identified a situation known as a tail-call. This is where a
function
effectively recycles its own stack frame for the called function. DTrace
reports a tail-call as a return and then a call to the other function. See
the fbt Provider chapter in the Solaris Dynamic Tracing Guide for more
information:
http://docs.sun.com/app/docs/doc/817-6223/6mlkidljt?a=view
Tail-call Optimization
When one function ends by calling another function, the compiler
can engage in tail-call optimization, in which the function being
called reuses the caller''s stack frame. This procedure is most
commonly used in the SPARC architecture, where the compiler reuses
the caller''s register window in the function being called in order
to minimize register window pressure.
...
Adam
On Wed, Mar 01, 2006 at 11:05:33AM -0800, CHIAN-PHON LIN
wrote:> Hi,
>
> Is it possible that a return occurs when making a call? For example,
> unix:sfmmu_hat_exit() calls mutex_exit at offset 0xc (ie. 12) and we
> know call is not the same thing as ret. However, tracing return of
> sfmmu_hat_exit also yield return offset 0xc (ie. 12). Below are the
> outputs:
>
> # /usr/ccs/bin/dis -F sfmmu_hat_exit /platform/sun4u/kernel/sparcv9/unix
> **** DISASSEMBLER ****
>
>
> disassembly for /platform/sun4u/kernel/sparcv9/unix
>
> section .text
> sfmmu_hat_exit()
> sfmmu_hat_exit: 90 90 00 08 orcc %g0, %o0, %o0
> sfmmu_hat_exit+0x4: 02 60 00 04 be,pn %xcc, sfmmu_hat_exit+0x14
> sfmmu_hat_exit+0x8: 82 10 00 0f mov %o7, %g1
> sfmmu_hat_exit+0xc: 40 00 42 b8 call mutex_exit
> sfmmu_hat_exit+0x10: 9e 10 00 01 mov %g1, %o7
> sfmmu_hat_exit+0x14: 81 c3 e0 08 retl
> sfmmu_hat_exit+0x18: 01 00 00 00 nop
> # dtrace -qn ''unix:sfmmu_hat_exit:return{@[probefunc, arg0] =
count();}''
> ^C
>
> sfmmu_hat_exit 20 27
> sfmmu_hat_exit 12 754
> #
>
> --
> ************************************************
> * C P Lin, Common Technology Project Lead. *
> * Sun Microsystems Inc. *
> * E-Mail: c.lin at sun.com *
> * Address: 4150 Network Circle, M/S UMPK12-330 *
> * Santa Clara, CA 95054 *
> * Phone: 650/352-4967 Fax: 650/786-7816 *
> ************************************************
>
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
--
Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl
On Wed, Mar 01, 2006 at 11:05:33AM -0800, CHIAN-PHON LIN wrote:> Hi, > > Is it possible that a return occurs when making a call? For example, > unix:sfmmu_hat_exit() calls mutex_exit at offset 0xc (ie. 12) and we > know call is not the same thing as ret. However, tracing return of > sfmmu_hat_exit also yield return offset 0xc (ie. 12). Below are the > outputs: > > # /usr/ccs/bin/dis -F sfmmu_hat_exit /platform/sun4u/kernel/sparcv9/unix > **** DISASSEMBLER **** > > > disassembly for /platform/sun4u/kernel/sparcv9/unix > > section .text > sfmmu_hat_exit() > sfmmu_hat_exit: 90 90 00 08 orcc %g0, %o0, %o0 > sfmmu_hat_exit+0x4: 02 60 00 04 be,pn %xcc, sfmmu_hat_exit+0x14 > sfmmu_hat_exit+0x8: 82 10 00 0f mov %o7, %g1 > sfmmu_hat_exit+0xc: 40 00 42 b8 call mutex_exit > sfmmu_hat_exit+0x10: 9e 10 00 01 mov %g1, %o7^^^ this replaces the return address for mutex_exit with the return address from sfmmu_hat_exit().> sfmmu_hat_exit+0x14: 81 c3 e0 08 retl > sfmmu_hat_exit+0x18: 01 00 00 00 nop > # dtrace -qn ''unix:sfmmu_hat_exit:return{@[probefunc, arg0] = count();}'' > ^C > > sfmmu_hat_exit 20 27 > sfmmu_hat_exit 12 754 > #Cheers, - jonathan -- Jonathan Adams, Solaris Kernel Development