dtrace discuss - Jan 2010 - dtrace -G: unused provider screws up other provider?

This drove me nuts until a colleague helped me nail down a small test case. 
I''m posting here before filing a bug in case I''m somehow
overlooking something that would justify what I''m seeing.  This is on
Solaris, both S10 and OpenSolaris (whatever release is snv_111b), by the way.

I have defined two providers, named A and B, and I have a program that uses
either a probe from each or only from one.  If it uses both, everything works as
expected.  If it uses only one, that probe disappears.  Somehow, adding the
second, unused provider to the "dtrace -G" seems to make the probe for
the first provider go away.

Here''s the test program, main.c:

#include "providerA.h"
#include "providerB.h"

int main() {
   A_HELLO();
#ifdef USE_B
   B_BYE();
#endif
   return 0;
}

With both probes in place, I get this:

$ dtrace -h -s providerA.d
$ dtrace -h -s providerB.d
$ cc -o main.o -c -DUSE_B main.c
$ dtrace -G -s providerA.d -s providerB.d -o providers.o main.o
$ cc -o program main.o providers.o
$ dtrace -l -P ''A$target'' -P ''B$target'' -c
./program
  ID   PROVIDER            MODULE                          FUNCTION NAME
63692     A12109           program                              main hello
63693     B12109           program                              main bye

Just as expected.  With only one probe in place, I get this:

$ dtrace -h -s providerA.d
$ dtrace -h -s providerB.d
$ cc -o main.o -c main.c
$ dtrace -G -s providerA.d -s providerB.d -o providers.o main.o
$ cc -o program main.o providers.o
$ dtrace -l -P ''A$target'' -P ''B$target'' -c
./program
  ID   PROVIDER            MODULE                          FUNCTION NAME
dtrace: failed to match A12161:::: No probe matches description
dtrace: failed to match B12161:::: No probe matches description

I expect the B probe to be missing, but what happened to the A probe?

The providers are trivial:
$ cat providerA.d providerB.d
provider A {
   probe hello();
};
provider B {
   probe bye();
};

Any clues?
Thanks.
joe

Hi Joe,

Thanks for the good test case.

Yes, this seems to be a bug. It appears that we end up generating
malformed DOF when unreferenced providers are included as in your
example. This will mean that we reject the DOF when it''s loaded
into the kernel and you therefore don''t get any probes. I''ve
logged:

6918387: DOF rejected if unreferenced USDT providers are included

I''m a bit short of time at the minute to look at this but let
me know if it''s causing major problems.

Thanks.

Jon.

> This drove me nuts until a colleague helped me nail down a small test case.
I''m posting here before filing a bug in case I''m somehow
overlooking something that would justify what I''m seeing.  This is on
Solaris, both S10 and OpenSolaris (whatever release is snv_111b), by the way.
> 
> I have defined two providers, named A and B, and I have a program that uses
either a probe from each or only from one.  If it uses both, everything works as
expected.  If it uses only one, that probe disappears.  Somehow, adding the
second, unused provider to the "dtrace -G" seems to make the probe for
the first provider go away.
> 
> Here''s the test program, main.c:
> 
> #include "providerA.h"
> #include "providerB.h"
> 
> int main() {
>    A_HELLO();
> #ifdef USE_B
>    B_BYE();
> #endif
>    return 0;
> }
> 
> With both probes in place, I get this:
> 
> $ dtrace -h -s providerA.d
> $ dtrace -h -s providerB.d
> $ cc -o main.o -c -DUSE_B main.c
> $ dtrace -G -s providerA.d -s providerB.d -o providers.o main.o
> $ cc -o program main.o providers.o
> $ dtrace -l -P ''A$target'' -P ''B$target''
-c ./program
>   ID   PROVIDER            MODULE                          FUNCTION NAME
> 63692     A12109           program                              main hello
> 63693     B12109           program                              main bye
> 
> Just as expected.  With only one probe in place, I get this:
> 
> $ dtrace -h -s providerA.d
> $ dtrace -h -s providerB.d
> $ cc -o main.o -c main.c
> $ dtrace -G -s providerA.d -s providerB.d -o providers.o main.o
> $ cc -o program main.o providers.o
> $ dtrace -l -P ''A$target'' -P ''B$target''
-c ./program
>   ID   PROVIDER            MODULE                          FUNCTION NAME
> dtrace: failed to match A12161:::: No probe matches description
> dtrace: failed to match B12161:::: No probe matches description
> 
> I expect the B probe to be missing, but what happened to the A probe?
> 
> The providers are trivial:
> $ cat providerA.d providerB.d
> provider A {
>    probe hello();
> };
> provider B {
>    probe bye();
> };
> 
> Any clues?
> Thanks.
> joe
> 
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org

On Jan 20, 2010, at 7:31 AM, Jon Haslam wrote:
> Hi Joe,
> 
> Thanks for the good test case.
> 
> Yes, this seems to be a bug. It appears that we end up generating
> malformed DOF when unreferenced providers are included as in your
> example. This will mean that we reject the DOF when it''s loaded
> into the kernel and you therefore don''t get any probes.
I''ve logged:
> 
> 6918387: DOF rejected if unreferenced USDT providers are included
> 
> I''m a bit short of time at the minute to look at this but let
> me know if it''s causing major problems.
> 
> Thanks.
> 
> Jon.
Well, this is a bit of a nuisance.  I''m trying to work around the
problem with probes in project static libraries by using the "ld -r"
trick suggested by Alan Coopersmith, but at executable link time rather than at
library creation time.  (See earlier thread "More fun with USDT and static
libraries on Solaris")  That means I need to process each partially linked
executable with all the providers that it uses.  My plan had been to have a
single provider file for each library, and just put all of those on the dtrace
-G line.

It looks like I need to do two things to work around this issue:

1) Don''t include multiple providers in a single .d file, unless I am
certain that either all of them are used or none of them are.  That''s
unlikely, so I''ll probably need to have one .d file per provider.

2) Examine the executable before running dtrace -G to determine which providers
are actually used, and include only those D files on the command line.

I was already needing a script to do linking anyway, so the second part is just
more work for it to do, assuming that I can rely on "nm -u | grep
__dtrace" to identify the probes that are actually used.  It''s
unfortunate that the two approaches interact to make the task just a bit
messier.

Do you see any obvious problems with this before I invest some effort in trying
it out?

Thanks.
joe