John Zolnowsky x69422/408-404-5064
2008-May-20 18:47 UTC
[dtrace-discuss] entry does not match any probes
In investigating a hang in kcfd, I wanted to find the call path to libxml2''s xmlInitParser() routine. So, under snv89, I invoked dtrace: # dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' \ -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid2794::xmlInitParser:entry does not match any probes elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. ... Specifying the libxml2.so.2 module doesn''t help. Removing the function name altogether does result in a probe at libxml2.so.2`xmlInitParser, but I have to search through an exhaustive list of all function calls. How do I narrow my probe specification? Thanks -JZ
Hi, On Tue, May 20, 2008 at 11:47:09AM -0700, John Zolnowsky x69422/408-404-5064 wrote:> In investigating a hang in kcfd, I wanted to find the call path to > libxml2''s xmlInitParser() routine. So, under snv89, I invoked dtrace: > > # dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' \ > -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' > dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid2794::xmlInitParser:entry does not match any probes > elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed.Running the same on my Nevada 81 does: $ dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' dtrace: description ''pid$target::xmlInitParser:entry '' matched 1 probe elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. format: rsa_sha1. signer: O=Sun Microsystems Inc, OU=Solaris Cryptographic Framework, CN=SunOS 5.10. signed on: 12. ledna 2008 19:54:01 CET. dtrace: pid 14760 has exited CPU ID FUNCTION:NAME 0 76158 xmlInitParser:entry libxml2.so.2`xmlInitParser libxml2.so.2`xmlParseDocument+0x1c libxml2.so.2`xmlDoRead+0x6b libxml2.so.2`xmlCtxtReadFile+0x55 libkmf.so.1`kmf_get_policy+0x63 libkmf.so.1`kmf_set_policy+0x6a libkmf.so.1`kmf_initialize+0xf1 libkmf.so.1`KMF_Initialize+0x21 libelfsign.so.1`elfcertlib_init+0x2f libelfsign.so.1`elfsign_begin+0xac elfsign`getelfobj+0x1c elfsign`do_verify+0x20 elfsign`main+0x568 elfsign`_start+0x7a Couldn''t you mistype the name ?> Specifying the libxml2.so.2 module doesn''t help. Removing the function > name altogether does result in a probe at libxml2.so.2`xmlInitParser, > but I have to search through an exhaustive list of all function calls. > How do I narrow my probe specification?You can always do ''-o logfile'' and then open the file in editor, where searching for string is easy. HTH -- Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 193 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20080520/c63a548e/attachment.bin>
Sean McGrath - Sun Microsystems Ireland
2008-May-20 23:10 UTC
[dtrace-discuss] entry does not match any probes
John Zolnowsky x69422/408-404-5064 stated: < In investigating a hang in kcfd, I wanted to find the call path to < libxml2''s xmlInitParser() routine. So, under snv89, I invoked dtrace: < < # dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' \ < -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' < dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid2794::xmlInitParser:entry does not match any probes < elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. < ... < < Specifying the libxml2.so.2 module doesn''t help. Removing the function < name altogether does result in a probe at libxml2.so.2`xmlInitParser, < but I have to search through an exhaustive list of all function calls. < How do I narrow my probe specification? Adding the -Z flag may help. dtrace(1M): ''Permit probe descriptions that match zero probes.'' Without -Z, same error as you. $ dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1 '' dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid3654::xmlInitParser:entry does not match any probes elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. format: rsa_sha1. signer: O=Sun Microsystems Inc, OU=Solaris Cryptographic Framework, CN=SunOS 5.10. signed on: Tue Apr 29 10:25:51 2008. With -Z the probe fires, even though matched 0 probes. $ dtrace -Z -n ''pid$target::xmlInitParser:entry { ustack(); }'' -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1 '' dtrace: description ''pid$target::xmlInitParser:entry '' matched 0 probes elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. format: rsa_sha1. signer: O=Sun Microsystems Inc, OU=Solaris Cryptographic Framework, CN=SunOS 5.10. signed on: Tue Apr 29 10:25:51 2008. dtrace: pid 3656 has exited CPU ID FUNCTION:NAME 0 79006 xmlInitParser:entry libxml2.so.2`xmlInitParser libxml2.so.2`xmlParseDocument+0x1c libxml2.so.2`xmlDoRead+0x6b libxml2.so.2`xmlCtxtReadFile+0x55 libkmf.so.1`kmf_get_policy+0x63 libkmf.so.1`kmf_set_policy+0x6a libkmf.so.1`kmf_initialize+0xf1 libkmf.so.1`KMF_Initialize+0x21 libelfsign.so.1`elfcertlib_init+0x2f libelfsign.so.1`elfsign_begin+0xac elfsign`getelfobj+0x22 elfsign`do_verify+0x20 elfsign`main+0x568 elfsign`_start+0x7a < < Thanks -JZ < _______________________________________________ < dtrace-discuss mailing list < dtrace-discuss at opensolaris.org -- Sean. .
John Zolnowsky x69422/408-404-5064
2008-May-20 23:31 UTC
[dtrace-discuss] entry does not match any probes
Sean, Thanks for the tip. This lends credence to my suspicion that the change in behavior came in snv83 with the direct bindings change, which also changed most libraries to lazyload. As a result, most libraries are now not yet loaded when dtrace does its initial probe scan, and hence the descriptions for libraries will now usually fail. Perhaps the description of the -Z flag should note its particular value for plugins and lazyloaded libraries. -JZ> From Sean.McGrath at Sun.COM Tue May 20 16:11:30 2008 > > John Zolnowsky x69422/408-404-5064 stated: > < In investigating a hang in kcfd, I wanted to find the call path to > < libxml2''s xmlInitParser() routine. So, under snv89, I invoked dtrace: > < > < # dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' \ > < -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' > < dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid2794::xmlInitParser:entry does not match any probes > < elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. > < ... > < > < Specifying the libxml2.so.2 module doesn''t help. Removing the function > < name altogether does result in a probe at libxml2.so.2`xmlInitParser, > < but I have to search through an exhaustive list of all function calls. > < How do I narrow my probe specification? > > Adding the -Z flag may help. > dtrace(1M): ''Permit probe descriptions that match zero probes.'' > > Without -Z, same error as you. > > $ dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1 '' > dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid3654::xmlInitParser:entry does not match any probes > elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. > format: rsa_sha1. > signer: O=Sun Microsystems Inc, OU=Solaris Cryptographic Framework, CN=SunOS 5.10. > signed on: Tue Apr 29 10:25:51 2008. > > With -Z the probe fires, even though matched 0 probes. > > $ dtrace -Z -n ''pid$target::xmlInitParser:entry { ustack(); }'' -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1 '' > dtrace: description ''pid$target::xmlInitParser:entry '' matched 0 probes > elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. > format: rsa_sha1. > signer: O=Sun Microsystems Inc, OU=Solaris Cryptographic Framework, CN=SunOS 5.10. > signed on: Tue Apr 29 10:25:51 2008. > dtrace: pid 3656 has exited > CPU ID FUNCTION:NAME > 0 79006 xmlInitParser:entry > libxml2.so.2`xmlInitParser > libxml2.so.2`xmlParseDocument+0x1c > libxml2.so.2`xmlDoRead+0x6b > libxml2.so.2`xmlCtxtReadFile+0x55 > libkmf.so.1`kmf_get_policy+0x63 > libkmf.so.1`kmf_set_policy+0x6a > libkmf.so.1`kmf_initialize+0xf1 > libkmf.so.1`KMF_Initialize+0x21 > libelfsign.so.1`elfcertlib_init+0x2f > libelfsign.so.1`elfsign_begin+0xac > elfsign`getelfobj+0x22 > elfsign`do_verify+0x20 > elfsign`main+0x568 > elfsign`_start+0x7a > > > > > < > < Thanks -JZ > < _______________________________________________ > < dtrace-discuss mailing list > < dtrace-discuss at opensolaris.org > > -- > Sean. > . >
> Thanks for the tip. > > This lends credence to my suspicion that the change in behavior came in > snv83 with the direct bindings change, which also changed most > libraries to lazyload. As a result, most libraries are now not yet > loaded when dtrace does its initial probe scan, and hence the > descriptions for libraries will now usually fail.Does it help if you export LD_NOLAZYLOAD ? -- Vlad -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 193 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20080521/94d532ea/attachment.bin>
> Does it help if you export LD_NOLAZYLOAD ?Yes, that also allows the probe description to match and fire. -JZ -- This message posted from opensolaris.org
Hey John, Can you send me the output of that dtrace(1M) invocation with ''-x debug'' added to the command-line? Thanks. - ahl On Tue, May 20, 2008 at 11:47:09AM -0700, John Zolnowsky x69422/408-404-5064 wrote:> In investigating a hang in kcfd, I wanted to find the call path to > libxml2''s xmlInitParser() routine. So, under snv89, I invoked dtrace: > > # dtrace -n ''pid$target::xmlInitParser:entry { ustack(); }'' \ > -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' > dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid2794::xmlInitParser:entry does not match any probes > elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. > ... > > Specifying the libxml2.so.2 module doesn''t help. Removing the function > name altogether does result in a probe at libxml2.so.2`xmlInitParser, > but I have to search through an exhaustive list of all function calls. > How do I narrow my probe specification? > > Thanks -JZ > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org-- Adam Leventhal, Fishworks http://blogs.sun.com/ahl
John Zolnowsky x69422/408-404-5064
2008-May-21 17:08 UTC
[dtrace-discuss] entry does not match any probes
> From ahl at basura.sf.fishpong.com Wed May 21 09:45:55 2008 > > Hey John, > > Can you send me the output of that dtrace(1M) invocation with ''-x debug'' added > to the command-line? Thanks. > > - ahlThis is just generic snv_89, so you can run your own experiments. For this request, here ya go .... -JZ (ksh)# dtrace -x debug -n ''pid$target::xmlInitParser:entry { ustack(); }'' -c ''elfsign verify -v /usr/lib/security/pkcs11_kernel.so.1'' libdtrace DEBUG: pid 22664: proc stopped showing 6/3 libdtrace DEBUG: pid 22664: hit breakpoint at ff3c6980 (1) libdtrace DEBUG: pid 22664: rtld event RD_DLACTIVITY type=3 state 2 libdtrace DEBUG: pid 22664: proc stopped showing 6/3 libdtrace DEBUG: pid 22664: hit breakpoint at ff3c6980 (2) libdtrace DEBUG: pid 22664: rtld event RD_DLACTIVITY type=3 state 1 libdtrace DEBUG: pid 22664: proc stopped showing 6/3 libdtrace DEBUG: pid 22664: hit breakpoint at ff3c69d4 (1) libdtrace DEBUG: pid 22664: rtld event RD_PREINIT type=1 state 0 libdtrace DEBUG: breakpoints disabled libdtrace DEBUG: created pid 22664 libdtrace DEBUG: library /usr/lib/dtrace/iscsi.d sorted (1/2) libdtrace DEBUG: library /usr/lib/dtrace/errno.d sorted (3/4) libdtrace DEBUG: library /usr/lib/dtrace/io.d sorted (5/6) libdtrace DEBUG: library /usr/lib/dtrace/nfs.d sorted (8/9) libdtrace DEBUG: library /usr/lib/dtrace/ip.d sorted (7/10) libdtrace DEBUG: library /usr/lib/dtrace/net.d sorted (11/12) libdtrace DEBUG: library /usr/lib/dtrace/procfs.d sorted (13/14) libdtrace DEBUG: library /usr/lib/dtrace/regs.d sorted (15/16) libdtrace DEBUG: library /usr/lib/dtrace/sched.d sorted (17/18) libdtrace DEBUG: library /usr/lib/dtrace/signal.d sorted (19/20) libdtrace DEBUG: library /usr/lib/dtrace/sysevent.d sorted (21/22) libdtrace DEBUG: library /usr/lib/dtrace/unistd.d sorted (23/24) libdtrace DEBUG: typedef syseventinfo_t added as id 32798 libdtrace DEBUG: typedef syseventchaninfo_t added as id 32801 libdtrace DEBUG: type struct cpuinfo added as id 32803 libdtrace DEBUG: typedef cpuinfo_t added as id 32815 libdtrace DEBUG: typedef conninfo_t added as id 32846 libdtrace DEBUG: typedef nfsv4opinfo_t added as id 36245 libdtrace DEBUG: typedef nfsv4cbinfo_t added as id 36247 libdtrace DEBUG: typedef nfsv3opinfo_t added as id 36264 libdtrace DEBUG: typedef nfsv3oparg_t added as id 36266 libdtrace DEBUG: typedef bufinfo_t added as id 36270 libdtrace DEBUG: typedef devinfo_t added as id 36273 libdtrace DEBUG: typedef fileinfo_t added as id 36276 libdtrace DEBUG: typedef iscsiinfo_t added as id 36279 libdtrace DEBUG: typedef uiscsiproto_t added as id 36281 libdtrace DEBUG: typedef uiscsiproto64_t added as id 36283 libdtrace DEBUG: typedef uiscsiproto32_t added as id 36285 libdtrace DEBUG: typedef iscsicmd_t added as id 36293 libdtrace DEBUG: typedef uiscsicmd_t added as id 36295 libdtrace DEBUG: grabbed pid 22664 (cached) libdtrace DEBUG: /usr/bin/elfsign stret 0 0 0 0 libdtrace DEBUG: /lib/ld.so.1 stret 0 0 0 0 libdtrace DEBUG: /lib/libc.so.1 stret ff237900 ff237938 ff237970 ff237970 dtrace: invalid probe specifier pid$target::xmlInitParser:entry { ustack(); }: probe description pid22664::xmlInitParser:entry does not match any probes libdtrace DEBUG: releasing pid 22664 libdtrace DEBUG: breakpoints enabled libdtrace DEBUG: breakpoints enabled libdtrace DEBUG: dt_buf_destroy(section headers): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(string table): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(loadable data): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(unloadable data): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(probe data): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(probe args): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(probe offs): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(probe is-enabled offs): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(probe rels): size=512 resizes=0 libdtrace DEBUG: dt_buf_destroy(xlate members): size=512 resizes=0 (super-ksh)# elfsign: verification of /usr/lib/security/pkcs11_kernel.so.1 passed. ...