dtrace discuss - Aug 2005 - Problem with probes defined in static libraries?

I spent a while earlier adding an SDT provider to ruby (purely to learn 
how to do it.)

Ruby puts most of the interpreter in a static library 
(libruby-static.a), unless configured --enable-shared.

If my probe is in this static library, it is not visible after linking 
the ruby executable (I''m running Nevada b19, i86pc/i386)

However, if I configure it with --enable-shared, the probe is visible.
Also, if I hack the makefile such that rather than linking 
libruby-static.a into the executable, it links the objects directly, the 
probe is also visible.

I put together a small test case, which is available at 
http://richlowe.net/~richlowe/dtrace-testcase.tgz

It builds 3 executables from the same objects
(directly linked, static lib, shared lib).

$ make
$ dtrace -lPtestcase''$target'' -c ./test-nolib    # No library
ID   PROVIDER            MODULE           FUNCTION NAME
44539 testcase14216    test-nolib         testfunc testprobe
$ dtrace -lPtestcase''$target'' -c ./test-dynlib   # Shared
library
ID   PROVIDER            MODULE           FUNCTION NAME
44539 testcase14218    libtestcase.so     testfunc testprobe
$ dtrace -lPtestcase''$target'' -c ./test-statlib  # Static
library
ID   PROVIDER            MODULE           FUNCTION NAME
dtrace: failed to match testcase14220:::: No probe matches description

It''s entirely possible I''m doing something stupid, or missing
something
obvious, but if I am I don''t know what it is :)

For anyone interested, the patch for ruby is available at 
http://richlowe.net/~richlowe/patches/ruby-1.8.2-dtrace.diff
if configured with --enable-shared --enable-dtrace, the provider does 
seem to actually work, but the patch is fairly horrific.

(patch(1) seems to have trouble applying the diff, because of the file 
addition, not sure what I can do about that...
if you edit the diff to remove the ''dtrace.d'' hunk, and create
that file
by hand, it applies fine).

There''s a kludge to only call the probes if the the class and method 
names can be converted to strings, and if the source file name is not 
NULL.  The ruby interpreter seems to call into ruby a lot before a 
script is actually started, and call various things that don''t have 
names which otherwise would mean we passed bad pointers to the probes.

If anybody wants to take this and run with it, feel free, I don''t know 
enough about the internals of ruby to get much further.

(forgot to CC dtrace-discuss)

On 8/16/05, Richard Lowe <richlowe at richlowe.net>
wrote:
> I spent a while earlier adding an SDT provider to ruby (purely to learn
> how to do it.)
> 
> Ruby puts most of the interpreter in a static library
> (libruby-static.a), unless configured --enable-shared.
> 
> If my probe is in this static library, it is not visible after linking
> the ruby executable (I''m running Nevada b19, i86pc/i386)
> 
> However, if I configure it with --enable-shared, the probe is visible.
> Also, if I hack the makefile such that rather than linking
> libruby-static.a into the executable, it links the objects directly, the
> probe is also visible.
This is only a guess, but you might try linking with ''-z
allextract'';
the dtrace stuff might not look like it is being referenced, and be
dropped out of the executable, which could cause this problem.

Cheers,
- jonathan

Hey Rich,

On Tue, Aug 16, 2005 at 10:59:44PM -0400, Richard Lowe
wrote:
> I spent a while earlier adding an SDT provider to ruby (purely to learn 
> how to do it.)
Spicey!
> Ruby puts most of the interpreter in a static library 
> (libruby-static.a), unless configured --enable-shared.
> 
> If my probe is in this static library, it is not visible after linking 
> the ruby executable (I''m running Nevada b19, i86pc/i386)
> 
> However, if I configure it with --enable-shared, the probe is visible.
> Also, if I hack the makefile such that rather than linking 
> libruby-static.a into the executable, it links the objects directly, the 
> probe is also visible.
This is actually a symptom of some larger strangeness with respect to
archives and .init processing.  The key is that archives are not like shared
objects -- they are really more a collection of objects that happen to be
represented as a single file in the file system.  (That is, ar(1) and tar(1)
are closer in spirit than you might realize.)  And in particular, when you
link against an archive, it _only_ pulls in the _object files_ that you
need.  This means that if you need a symbol found in some object, that whole
object gets linked into the resulting object -- .init section and all.  If
an object isn''t needed, that object isn''t linked at all -- and
any .init and
.fini sections in that object are silently ignored.  The Linkers and
Libraries manual actually warns about this (if opaquely) in "Initialization
and Termination Sections":

  Initialization and termination code, spread throughout several
  relocatable objects, can result in different behavior when included in
  an archive library or shared object. The link-edit of an application
  that uses this archive might extract only a fraction of the objects
  contained in the archive.  These objects might provide only a portion
  of the initialization and termination code spread throughout the
  members of the archive. At runtime, only this portion of code is
  executed. The same application built against the shared object will
  have all the accumulated initialization and termination code executed
  when the dependency is loaded at runtime.

  (See http://docs.sun.com/app/docs/doc/817-1984/6mhm7pl17?q=init&a=view)

And that, in a nutshell, is what''s nailing you here.  When we generate
the
.o with the probe definition in it (testprovider.o in your case), we very
much rely on the .init section for that object being currectly processed and
run.  (That''s how your probes make themselves known to the system.) 
But
when you toss testprovider.o in an archive, there is no reason for that
object to be pulled out of the archive for linking against main.o -- so it
never goes along for the ride, the .init section in testprovider.o is never
run, and your probes are never created.  Fortunately, solving this problem
isn''t too bad:  when you generate the .o that contains the provider,
turn
around and link that new object right back against the object(s) that you
passed to dtrace(1M) to form a new, larger glommed object.  Then use _that_
object to create the archive.  To put this in terms of your test case,
you want to change the build procedure for libtestcase.a to be:

  dtrace -G -32 -o testprovider.o -s testprovider.d testfunc.o
  ld -r -o glommedobj.o testprovider.o testfunc.o
  ar rcu libtestcase.a glommedobj.o

Doing this yields a libtestcase.a that can be correctly used as an archive.
So it''s actually not so bad -- but oh boy does it need to be
documented...

	- Bryan

--------------------------------------------------------------------------
Bryan Cantrill, Solaris Kernel Development.       http://blogs.sun.com/bmc

Bryan Cantrill wrote:
> Hey Rich,
> 
> On Tue, Aug 16, 2005 at 10:59:44PM -0400, Richard Lowe wrote:
> 
>>Ruby puts most of the interpreter in a static library 
>>(libruby-static.a), unless configured --enable-shared.
>>
>>If my probe is in this static library, it is not visible after linking 
>>the ruby executable (I''m running Nevada b19, i86pc/i386)
>>
>>However, if I configure it with --enable-shared, the probe is visible.
>>Also, if I hack the makefile such that rather than linking 
>>libruby-static.a into the executable, it links the objects directly, the
>>probe is also visible.
> 
> 
> This is actually a symptom of some larger strangeness with respect to
> archives and .init processing.  The key is that archives are not like
shared
> objects -- they are really more a collection of objects that happen to be
> represented as a single file in the file system.  (That is, ar(1) and
tar(1)
> are closer in spirit than you might realize.)  And in particular, when you
> link against an archive, it _only_ pulls in the _object files_ that you
> need.  This means that if you need a symbol found in some object, that
whole
> object gets linked into the resulting object -- .init section and all.  If
> an object isn''t needed, that object isn''t linked at all
-- and any .init and
> .fini sections in that object are silently ignored.  The Linkers and
> Libraries manual actually warns about this (if opaquely) in
"Initialization
> and Termination Sections":
> 
>   Initialization and termination code, spread throughout several
>   relocatable objects, can result in different behavior when included in
>   an archive library or shared object. The link-edit of an application
>   that uses this archive might extract only a fraction of the objects
>   contained in the archive.  These objects might provide only a portion
>   of the initialization and termination code spread throughout the
>   members of the archive. At runtime, only this portion of code is
>   executed. The same application built against the shared object will
>   have all the accumulated initialization and termination code executed
>   when the dependency is loaded at runtime.
> 
>   (See
http://docs.sun.com/app/docs/doc/817-1984/6mhm7pl17?q=init&a=view)
> 
> And that, in a nutshell, is what''s nailing you here.  When we
generate the
> .o with the probe definition in it (testprovider.o in your case), we very
> much rely on the .init section for that object being currectly processed
and
> run.  (That''s how your probes make themselves known to the
system.)  But
> when you toss testprovider.o in an archive, there is no reason for that
> object to be pulled out of the archive for linking against main.o -- so it
> never goes along for the ride, the .init section in testprovider.o is never
> run, and your probes are never created.  Fortunately, solving this problem
> isn''t too bad:  when you generate the .o that contains the
provider, turn
> around and link that new object right back against the object(s) that you
> passed to dtrace(1M) to form a new, larger glommed object.  Then use _that_
> object to create the archive.  To put this in terms of your test case,
> you want to change the build procedure for libtestcase.a to be:
Ah ha!
So when dtrace -G replaces the actual call to the probe with a nop, the 
only reference into the provider object vanishes, and it''s not linked.

That makes sense (now) :)
>   dtrace -G -32 -o testprovider.o -s testprovider.d testfunc.o
>   ld -r -o glommedobj.o testprovider.o testfunc.o
>   ar rcu libtestcase.a glommedobj.o
> 
> Doing this yields a libtestcase.a that can be correctly used as an archive.
Indeed, it does.  Thank you!

An updated patch against ruby-1.8.2 is at 
http://richlowe.net/~richlowe/patches/ruby-1.8.2-dtrace.diff
and now works when configured without --enable-shared
(though the associated make-goo isn''t pretty).