John Rice
2007-Feb-13 16:39 UTC
[dtrace-discuss] USDT probes in Mozilla - can we run dtrace -G against a shared library
Hi - we are looking into adding dtrace probes into mozilla. One of the problems is that this is a large project with lots of shared libraries. It would simplify integration of probes if we could just build the shared libs with the embedded probes [using appropriate header from dtrace -h] and then before we do the final link to create the binary, do the dtrace -G to convert all our undef probe symbols into the appropriate ignore ones. Simple test seems to say that we cannot. As a test we took Bart''s example USDT, split it into two files and generated a shared library for it, but could not get dtrace -G to swallow it: $ make simple_probes.o dtrace -G -s simple_probes.d simple1.so dtrace: failed to link script simple_probes: an error was encountered while processing simple1.so make: *** [simple_probes.o] Error 1 Anyone any ideas? Of course may have screwed up on my shared lib creation ... JR http://blogs.sun.com/barts/entry/putting_user_defined_dtrace_probe -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Makefile URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/0dd51f22/attachment.ksh> -------------- next part -------------- A non-text attachment was scrubbed... Name: simple.c Type: text/x-csrc Size: 411 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/0dd51f22/attachment.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: simple1.c Type: text/x-csrc Size: 2255 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/0dd51f22/attachment-0001.bin>
Bart Smaalders
2007-Feb-13 17:19 UTC
[dtrace-discuss] USDT probes in Mozilla - can we run dtrace -G against a shared library
John Rice wrote:> Hi - we are looking into adding dtrace probes into mozilla. > > One of the problems is that this is a large project with lots of shared > libraries. It would simplify integration of probes if we could just > build the shared libs with the embedded probes [using appropriate header > from dtrace -h] and then before we do the final link to create the > binary, do the dtrace -G to convert all our undef probe symbols into the > appropriate ignore ones. > > Simple test seems to say that we cannot. As a test we took Bart''s > example USDT, split it into two files and generated a shared library for > it, but could not get dtrace -G to swallow it: > > $ make simple_probes.o > dtrace -G -s simple_probes.d simple1.so > dtrace: failed to link script simple_probes: an error was encountered > while processing simple1.so > make: *** [simple_probes.o] Error 1 > > Anyone any ideas? Of course may have screwed up on my shared lib > creation ... > > JR > > http://blogs.sun.com/barts/entry/putting_user_defined_dtrace_probe >The easiest thing to do here is to place each library''s probes in each library separately. Collect all the .o files that belong in a library into a single .o w/ ld -r and then use dtrace on it. I modified the example to put the probes into the shared library; since there was only a single .o file I use it w/o the ld -r. - Bart -- Bart Smaalders Solaris Kernel Performance barts at cyber.eng.sun.com http://blogs.sun.com/barts -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Makefile URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/b1e3a4d7/attachment.ksh> -------------- next part -------------- A non-text attachment was scrubbed... Name: simple.c Type: text/x-csrc Size: 411 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/b1e3a4d7/attachment.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: simple1.c Type: text/x-csrc Size: 2273 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/b1e3a4d7/attachment-0001.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: simple_probes.d Type: text/x-dsrc Size: 67 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20070213/b1e3a4d7/attachment-0002.bin>
John Rice
2007-Feb-13 22:29 UTC
[dtrace-discuss] USDT probes in Mozilla - can we run dtrace -G against a shared library
Thanks Bart - we can do this, but we where trying to reduce the amount of work someone who wants to add a new probe to mozilla would have to do, once we put in the basic infrastruture. Just getting them to add in a probe to a top level probe.d file, add the call to the probe to their .cpp file which includes the top level probe.h file, then run a top level make would have been ideal. But to make this happen we''d have had to been able to process the .so''s containign probes with dtrace -G in the top level make. Otherwise they have to modify the sub component makefiles, which we where trying to avoid :( JR Bart Smaalders wrote:> John Rice wrote: >> Hi - we are looking into adding dtrace probes into mozilla. >> >> One of the problems is that this is a large project with lots of >> shared libraries. It would simplify integration of probes if we could >> just build the shared libs with the embedded probes [using >> appropriate header from dtrace -h] and then before we do the final >> link to create the binary, do the dtrace -G to convert all our undef >> probe symbols into the appropriate ignore ones. >> >> Simple test seems to say that we cannot. As a test we took Bart''s >> example USDT, split it into two files and generated a shared library >> for it, but could not get dtrace -G to swallow it: >> >> $ make simple_probes.o >> dtrace -G -s simple_probes.d simple1.so >> dtrace: failed to link script simple_probes: an error was encountered >> while processing simple1.so >> make: *** [simple_probes.o] Error 1 >> >> Anyone any ideas? Of course may have screwed up on my shared lib >> creation ... >> >> JR >> >> http://blogs.sun.com/barts/entry/putting_user_defined_dtrace_probe >> > > > The easiest thing to do here is to place each library''s > probes in each library separately. Collect all the .o files > that belong in a library into a single .o w/ ld -r and then > use dtrace on it. > > I modified the example to put the probes into the shared library; > since there was only a single .o file I use it w/o the ld -r. > > - Bart > > ------------------------------------------------------------------------ > > > CFLAGS=-Kpic > CC=cc > > DTRACE=dtrace > > default: simple > > simple_probes.o: simple_probes.d simple1.o > $(DTRACE) -G -s simple_probes.d simple1.o > > simple1.so: simple1.o simple_probes.o > $(CC) -o simple1.so -G simple1.o simple_probes.o -ztext -lc > > simple: simple.o simple1.so > $(CC) -o simple simple.o -R`pwd` simple1.so > > ------------------------------------------------------------------------ > > #include <stdio.h> > #include <sys/sdt.h> > > extern int foo(int argc, char *argv[]); > > /* > * simple example of defining sdt probes in a trivial program > * Sdt probes can often completely replace debug levels, optional > * log files, etc, in daemons... you can leverage the power of dtrace > * to make your server/application more readily debuggable. > */ > > int > main(int argc, char *argv[]) > { > return foo(argc, argv); > } > > > ------------------------------------------------------------------------ > > #include <stdio.h> > #include <sys/sdt.h> > #include <ctype.h> > /* > * simple example of defining sdt probes in a trivial program > * Sdt probes can often completely replace debug levels, optional > * log files, etc, in daemons... you can leverage the power of dtrace > * to make your server/application more readily debuggable. > */ > > int > foo(int argc, char *argv[]) > { > int i; > int characters, lines, words; > characters = lines = words = 0; > > > while (1) { > > if ((i = getchar()) == EOF) { > /* > * here we specify the name of the module, > * the name of the probe (modulo mapping > * ''__'' to ''-'') and pass in the parameter to be > * traced which in this case is the number of > * lines seen so far. > */ > DTRACE_PROBE1(simple, saw__line, lines); > break; > } > > characters++; > > if (i == ''\n'') { > lines++; > DTRACE_PROBE1(simple, saw__line, lines); > continue; > } > > if (isblank(i)) /* eating white space */ > continue; > > words++; /* in a word now */ > > while (1) { > > if ((i = getchar()) == EOF) { > DTRACE_PROBE1(simple, saw__word, words); > break; > } > > characters++; > > if (i == ''\n'') { /* EOL? ends word implicitly */ > DTRACE_PROBE1(simple, saw__word, words); > lines++; > DTRACE_PROBE1(simple, saw__line, lines); > break; > } > > if (isblank(i)) { /* white space ends words too */ > DTRACE_PROBE1(simple, saw__word, words); > break; > } > } > } > > printf("%8d %8d %8d\n", lines, words, characters); > > exit(0); > } > > > ------------------------------------------------------------------------ > > provider simple { > probe saw__word(int); > probe saw__line(int); > }; >
Adam Leventhal
2007-Feb-18 18:29 UTC
[dtrace-discuss] USDT probes in Mozilla - can we run dtrace -G against a shared library
Hi John, I want to make sure I understand the issue here: mozilla has a bazillion shared objects and several of them are going to need DTrace probes. You want to let developers add probes to any one of them without any exotic linking steps and you don''t want to have a .d file with a provider definition for each shared object. You could have a single .d file, but dtrace -G will fail if a given probe doesn''t exist. Is that correct? Adam On Tue, Feb 13, 2007 at 10:29:00PM +0000, John Rice wrote:> Thanks Bart - we can do this, but we where trying to reduce the amount > of work someone who wants to add a new probe to mozilla would have to > do, once we put in the basic infrastruture. > > Just getting them to add in a probe to a top level probe.d file, add the > call to the probe to their .cpp file which includes the top level > probe.h file, then run a top level make would have been ideal. But to > make this happen we''d have had to been able to process the .so''s > containign probes with dtrace -G in the top level make. Otherwise they > have to modify the sub component makefiles, which we where trying to > avoid :( > > JR > > Bart Smaalders wrote: > >John Rice wrote: > >>Hi - we are looking into adding dtrace probes into mozilla. > >> > >>One of the problems is that this is a large project with lots of > >>shared libraries. It would simplify integration of probes if we could > >>just build the shared libs with the embedded probes [using > >>appropriate header from dtrace -h] and then before we do the final > >>link to create the binary, do the dtrace -G to convert all our undef > >>probe symbols into the appropriate ignore ones. > >> > >>Simple test seems to say that we cannot. As a test we took Bart''s > >>example USDT, split it into two files and generated a shared library > >>for it, but could not get dtrace -G to swallow it: > >> > >>$ make simple_probes.o > >>dtrace -G -s simple_probes.d simple1.so > >>dtrace: failed to link script simple_probes: an error was encountered > >>while processing simple1.so > >>make: *** [simple_probes.o] Error 1 > >> > >>Anyone any ideas? Of course may have screwed up on my shared lib > >>creation ... > >> > >>JR > >> > >>http://blogs.sun.com/barts/entry/putting_user_defined_dtrace_probe > >> > > > > > >The easiest thing to do here is to place each library''s > >probes in each library separately. Collect all the .o files > >that belong in a library into a single .o w/ ld -r and then > >use dtrace on it. > > > >I modified the example to put the probes into the shared library; > >since there was only a single .o file I use it w/o the ld -r. > > > >- Bart > > > >------------------------------------------------------------------------ > > > > > >CFLAGS=-Kpic > >CC=cc > > > >DTRACE=dtrace > > > >default: simple > > > >simple_probes.o: simple_probes.d simple1.o > > $(DTRACE) -G -s simple_probes.d simple1.o > > > >simple1.so: simple1.o simple_probes.o > > $(CC) -o simple1.so -G simple1.o simple_probes.o -ztext -lc > > > >simple: simple.o simple1.so > > $(CC) -o simple simple.o -R`pwd` simple1.so > > > >------------------------------------------------------------------------ > > > >#include <stdio.h> > >#include <sys/sdt.h> > > > >extern int foo(int argc, char *argv[]); > > > >/* > > * simple example of defining sdt probes in a trivial program > > * Sdt probes can often completely replace debug levels, optional > > * log files, etc, in daemons... you can leverage the power of dtrace > > * to make your server/application more readily debuggable. > > */ > > > >int > >main(int argc, char *argv[]) > >{ > > return foo(argc, argv); > >} > > > > > >------------------------------------------------------------------------ > > > >#include <stdio.h> > >#include <sys/sdt.h> > >#include <ctype.h> > >/* > > * simple example of defining sdt probes in a trivial program > > * Sdt probes can often completely replace debug levels, optional > > * log files, etc, in daemons... you can leverage the power of dtrace > > * to make your server/application more readily debuggable. > > */ > > > >int > >foo(int argc, char *argv[]) > >{ > > int i; > > int characters, lines, words; > > characters = lines = words = 0; > > > > > > while (1) { > > > > if ((i = getchar()) == EOF) { > > /* > > * here we specify the name of the module, > > * the name of the probe (modulo mapping > > * ''__'' to ''-'') and pass in the parameter to be > > * traced which in this case is the number of > > * lines seen so far. > > */ > > DTRACE_PROBE1(simple, saw__line, lines); > > break; > > } > > > > characters++; > > > > if (i == ''\n'') { > > lines++; > > DTRACE_PROBE1(simple, saw__line, lines); > > continue; > > } > > > > if (isblank(i)) /* eating white space */ > > continue; > > > > words++; /* in a word now */ > > > > while (1) { > > > > if ((i = getchar()) == EOF) { > > DTRACE_PROBE1(simple, saw__word, words); > > break; > > } > > > > characters++; > > > > if (i == ''\n'') { /* EOL? ends word implicitly */ > > DTRACE_PROBE1(simple, saw__word, words); > > lines++; > > DTRACE_PROBE1(simple, saw__line, lines); > > break; > > } > > > > if (isblank(i)) { /* white space ends words too */ > > DTRACE_PROBE1(simple, saw__word, words); > > break; > > } > > } > > } > > > > printf("%8d %8d %8d\n", lines, words, characters); > > > > exit(0); > >} > > > > > >------------------------------------------------------------------------ > > > >provider simple { > > probe saw__word(int); > > probe saw__line(int); > >}; > > > > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org-- Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl
John Rice
2007-Feb-19 15:15 UTC
[dtrace-discuss] USDT probes in Mozilla - can we run dtrace -G against a shared library
Adam - yep think you''ve got it. We want to make it as easy as possible to add probes to any of the mozilla sub projects, so what to do. Seems we can''t process at the .so level with dtrace -G, so we have no choice but to process at the subcomponent level and process generated .o''s which have probe definitions. Here''s what we have so far for the proposal we are submitting: We will have a single top level .d file into which to place the probes, then developers who want to add new probes to a subcomponent will need to modify the makefile for the subproject so it will create the appropriate <sublib>_probes.o to link against when building this particular .so. We''ve done it already for the mozilla/layer subcomponent so folks have a template to follow. This shouldn''t really be a big deal if probes are added to different subcomponents and we update the makefiles, but would be nice not to have to :( I''ll post a link to the proposal later on today that has all the gory details. If you can suggest anything simpler fire ahead :) Cheers, JR Adam Leventhal wrote:> Hi John, > > I want to make sure I understand the issue here: mozilla has a bazillion > shared objects and several of them are going to need DTrace probes. You want > to let developers add probes to any one of them without any exotic linking > steps and you don''t want to have a .d file with a provider definition for > each shared object. You could have a single .d file, but dtrace -G will fail > if a given probe doesn''t exist. Is that correct? > > Adam > > On Tue, Feb 13, 2007 at 10:29:00PM +0000, John Rice wrote: > >> Thanks Bart - we can do this, but we where trying to reduce the amount >> of work someone who wants to add a new probe to mozilla would have to >> do, once we put in the basic infrastruture. >> >> Just getting them to add in a probe to a top level probe.d file, add the >> call to the probe to their .cpp file which includes the top level >> probe.h file, then run a top level make would have been ideal. But to >> make this happen we''d have had to been able to process the .so''s >> containign probes with dtrace -G in the top level make. Otherwise they >> have to modify the sub component makefiles, which we where trying to >> avoid :( >> >> JR >> >> Bart Smaalders wrote: >> >>> John Rice wrote: >>> >>>> Hi - we are looking into adding dtrace probes into mozilla. >>>> >>>> One of the problems is that this is a large project with lots of >>>> shared libraries. It would simplify integration of probes if we could >>>> just build the shared libs with the embedded probes [using >>>> appropriate header from dtrace -h] and then before we do the final >>>> link to create the binary, do the dtrace -G to convert all our undef >>>> probe symbols into the appropriate ignore ones. >>>> >>>> Simple test seems to say that we cannot. As a test we took Bart''s >>>> example USDT, split it into two files and generated a shared library >>>> for it, but could not get dtrace -G to swallow it: >>>> >>>> $ make simple_probes.o >>>> dtrace -G -s simple_probes.d simple1.so >>>> dtrace: failed to link script simple_probes: an error was encountered >>>> while processing simple1.so >>>> make: *** [simple_probes.o] Error 1 >>>> >>>> Anyone any ideas? Of course may have screwed up on my shared lib >>>> creation ... >>>> >>>> JR >>>> >>>> http://blogs.sun.com/barts/entry/putting_user_defined_dtrace_probe >>>> >>>> >>> The easiest thing to do here is to place each library''s >>> probes in each library separately. Collect all the .o files >>> that belong in a library into a single .o w/ ld -r and then >>> use dtrace on it. >>> >>> I modified the example to put the probes into the shared library; >>> since there was only a single .o file I use it w/o the ld -r. >>> >>> - Bart >>> >>> ------------------------------------------------------------------------ >>> >>> >>> CFLAGS=-Kpic >>> CC=cc >>> >>> DTRACE=dtrace >>> >>> default: simple >>> >>> simple_probes.o: simple_probes.d simple1.o >>> $(DTRACE) -G -s simple_probes.d simple1.o >>> >>> simple1.so: simple1.o simple_probes.o >>> $(CC) -o simple1.so -G simple1.o simple_probes.o -ztext -lc >>> >>> simple: simple.o simple1.so >>> $(CC) -o simple simple.o -R`pwd` simple1.so >>> >>> ------------------------------------------------------------------------ >>> >>> #include <stdio.h> >>> #include <sys/sdt.h> >>> >>> extern int foo(int argc, char *argv[]); >>> >>> /* >>> * simple example of defining sdt probes in a trivial program >>> * Sdt probes can often completely replace debug levels, optional >>> * log files, etc, in daemons... you can leverage the power of dtrace >>> * to make your server/application more readily debuggable. >>> */ >>> >>> int >>> main(int argc, char *argv[]) >>> { >>> return foo(argc, argv); >>> } >>> >>> >>> ------------------------------------------------------------------------ >>> >>> #include <stdio.h> >>> #include <sys/sdt.h> >>> #include <ctype.h> >>> /* >>> * simple example of defining sdt probes in a trivial program >>> * Sdt probes can often completely replace debug levels, optional >>> * log files, etc, in daemons... you can leverage the power of dtrace >>> * to make your server/application more readily debuggable. >>> */ >>> >>> int >>> foo(int argc, char *argv[]) >>> { >>> int i; >>> int characters, lines, words; >>> characters = lines = words = 0; >>> >>> >>> while (1) { >>> >>> if ((i = getchar()) == EOF) { >>> /* >>> * here we specify the name of the module, >>> * the name of the probe (modulo mapping >>> * ''__'' to ''-'') and pass in the parameter to be >>> * traced which in this case is the number of >>> * lines seen so far. >>> */ >>> DTRACE_PROBE1(simple, saw__line, lines); >>> break; >>> } >>> >>> characters++; >>> >>> if (i == ''\n'') { >>> lines++; >>> DTRACE_PROBE1(simple, saw__line, lines); >>> continue; >>> } >>> >>> if (isblank(i)) /* eating white space */ >>> continue; >>> >>> words++; /* in a word now */ >>> >>> while (1) { >>> >>> if ((i = getchar()) == EOF) { >>> DTRACE_PROBE1(simple, saw__word, words); >>> break; >>> } >>> >>> characters++; >>> >>> if (i == ''\n'') { /* EOL? ends word implicitly */ >>> DTRACE_PROBE1(simple, saw__word, words); >>> lines++; >>> DTRACE_PROBE1(simple, saw__line, lines); >>> break; >>> } >>> >>> if (isblank(i)) { /* white space ends words too */ >>> DTRACE_PROBE1(simple, saw__word, words); >>> break; >>> } >>> } >>> } >>> >>> printf("%8d %8d %8d\n", lines, words, characters); >>> >>> exit(0); >>> } >>> >>> >>> ------------------------------------------------------------------------ >>> >>> provider simple { >>> probe saw__word(int); >>> probe saw__line(int); >>> }; >>> >>> >> _______________________________________________ >> dtrace-discuss mailing list >> dtrace-discuss at opensolaris.org >> > >