dtrace discuss - Oct 2007 - What are .dynstr dtrace strings and why they change?

Hello,

I was comparing two workspaces using wsdiff and noticed unexpected changes in 
libc which looked like these:

lib/libc.so.1
NOTE: ELF .dynstr difference detected.

1427,1428c1427,1428
< 0026220   t  \0   $   d   t   r   a   c   e   1   5   0   7   3   3   .
< 	        74002464        74726163        65313530        3733332e
---
> 0026220   t  \0   $   d   t   r   a   c   e   1   4   9   4   2   0   .
> 	        74002464        74726163        65313439        3432302e
1433,1434c1433,1434
< 0026300  \0   $   d   t   r   a   c   e   1   5   0   7   3   3   .   m
< 	        00246474        72616365        31353037        33332e6d
---
> 0026300  \0   $   d   t   r   a   c   e   1   4   9   4   2   0   .   m
> 	        00246474        72616365        31343934        32302e6d
1439,1442c1439,1442
< 0026360   u   p   _   a   l   l  \0   $   d   t   r   a   c   e   1   5
< 	        75705f61        6c6c0024        64747261        63653135
< 0026400   0   7   3   3   .   m   u   t   e   x   _   t   r   y   l   o
< 	        30373333        2e6d7574        65785f74        72796c6f
---
> 0026360   u   p   _   a   l   l  \0   $   d   t   r   a   c   e   1   4
> 	        75705f61        6c6c0024        64747261        63653134
> 0026400   9   4   2   0   .   m   u   t   e   x   _   t   r   y   l   o
> 	        39343230        2e6d7574        65785f74        72796c6f
...

What are these and why they can show up?

- akolb

If you have USDT probes in static functions, the dtrace -G invocation will
generate these symbols. They be should be scoped local in the final binary
(I''m surprised that they''re showing up in the dynstr --
that''s probably a
bug in the linker).

Once the following bug is putback, it will be possible for DTrace to mark
these temporary symbols so that they don''t show up in the final binary
at
all:

  6602451 new symbol visibilities required: EXPORTED, SINGLETON and ELIMINATE

Adam

On Tue, Oct 02, 2007 at 05:53:17PM -0700, Alexander Kolbasov
wrote:
> Hello,
> 
> I was comparing two workspaces using wsdiff and noticed unexpected changes
in
> libc which looked like these:
> 
> lib/libc.so.1
> NOTE: ELF .dynstr difference detected.
> 
> 1427,1428c1427,1428
> < 0026220   t  \0   $   d   t   r   a   c   e   1   5   0   7   3   3  
.
> < 	        74002464        74726163        65313530        3733332e
> ---
> > 0026220   t  \0   $   d   t   r   a   c   e   1   4   9   4   2   0  
.
> > 	        74002464        74726163        65313439        3432302e
> 1433,1434c1433,1434
> < 0026300  \0   $   d   t   r   a   c   e   1   5   0   7   3   3   .  
m
> < 	        00246474        72616365        31353037        33332e6d
> ---
> > 0026300  \0   $   d   t   r   a   c   e   1   4   9   4   2   0   .  
m
> > 	        00246474        72616365        31343934        32302e6d
> 1439,1442c1439,1442
> < 0026360   u   p   _   a   l   l  \0   $   d   t   r   a   c   e   1  
5
> < 	        75705f61        6c6c0024        64747261        63653135
> < 0026400   0   7   3   3   .   m   u   t   e   x   _   t   r   y   l  
o
> < 	        30373333        2e6d7574        65785f74        72796c6f
> ---
> > 0026360   u   p   _   a   l   l  \0   $   d   t   r   a   c   e   1  
4
> > 	        75705f61        6c6c0024        64747261        63653134
> > 0026400   9   4   2   0   .   m   u   t   e   x   _   t   r   y   l  
o
> > 	        39343230        2e6d7574        65785f74        72796c6f
> 
> ...
> 
> What are these and why they can show up?
> 
> - akolb
> 
> 
> _______________________________________________
> dtrace-discuss mailing list
> dtrace-discuss at opensolaris.org
-- 
Adam Leventhal, FishWorks                        http://blogs.sun.com/ahl

Adam,

thank you for the explanation. What would cause these symbols to change? The 
only thing that I changed between build was the introduce parallelism between 
uts and lib build plus additional parallelism in uts build.

- akolb
> If you have USDT probes in static functions, the dtrace -G invocation will
> generate these symbols. They be should be scoped local in the final binary
> (I''m surprised that they''re showing up in the dynstr --
that''s probably a
> bug in the linker).
> 
> Once the following bug is putback, it will be possible for DTrace to mark
> these temporary symbols so that they don''t show up in the final
binary at
> all:
> 
>   6602451 new symbol visibilities required: EXPORTED, SINGLETON and
ELIMINATE
> 
> Adam
> 
> On Tue, Oct 02, 2007 at 05:53:17PM -0700, Alexander Kolbasov wrote:
> > Hello,
> > 
> > I was comparing two workspaces using wsdiff and noticed unexpected
changes in
> > libc which looked like these:
> > 
> > lib/libc.so.1
> > NOTE: ELF .dynstr difference detected.
> > 
> > 1427,1428c1427,1428
> > < 0026220   t  \0   $   d   t   r   a   c   e   1   5   0   7   3  
3   .
> > < 	        74002464        74726163        65313530        3733332e
> > ---
> > > 0026220   t  \0   $   d   t   r   a   c   e   1   4   9   4   2  
0   .
> > > 	        74002464        74726163        65313439        3432302e
> > 1433,1434c1433,1434
> > < 0026300  \0   $   d   t   r   a   c   e   1   5   0   7   3   3  
.   m
> > < 	        00246474        72616365        31353037        33332e6d
> > ---
> > > 0026300  \0   $   d   t   r   a   c   e   1   4   9   4   2   0  
.   m
> > > 	        00246474        72616365        31343934        32302e6d
> > 1439,1442c1439,1442
> > < 0026360   u   p   _   a   l   l  \0   $   d   t   r   a   c   e  
1   5
> > < 	        75705f61        6c6c0024        64747261        63653135
> > < 0026400   0   7   3   3   .   m   u   t   e   x   _   t   r   y  
l   o
> > < 	        30373333        2e6d7574        65785f74        72796c6f
> > ---
> > > 0026360   u   p   _   a   l   l  \0   $   d   t   r   a   c   e  
1   4
> > > 	        75705f61        6c6c0024        64747261        63653134
> > > 0026400   9   4   2   0   .   m   u   t   e   x   _   t   r   y  
l   o
> > > 	        39343230        2e6d7574        65785f74        72796c6f
> > 
> > ...
> > 
> > What are these and why they can show up?
> > 
> > - akolb
> > 
> > 
> > _______________________________________________
> > dtrace-discuss mailing list
> > dtrace-discuss at opensolaris.org
> 
> -- 
> Adam Leventhal, FishWorks                        http://blogs.sun.com/ahl
>

> thank you for the explanation. What would cause these symbols to change?
The
> only thing that I changed between build was the introduce parallelism
between
> uts and lib build plus additional parallelism in uts build.
dtrace -G chooses a randomly generated prefix for the symbols. They should
change every time a workspace is built.

Adam

-- 
Adam Leventhal, FishWorks                        http://blogs.sun.com/ahl

Adam Leventhal wrote:
> If you have USDT probes in static functions, the dtrace -G invocation will
> generate these symbols. They be should be scoped local in the final binary
> (I''m surprised that they''re showing up in the dynstr --
that''s probably a
> bug in the linker).
  I''m betting that these are local symbols referenced by the
.SUNW_ldynsym
section, which is an allocable section, and therefore uses the dynstr:

	ali at rtld% elfdump -sN.SUNW_ldynsym /lib/libc.so | grep dtrace
	      [12]  0x000a7202 0x0000038f  FUNC LOCL  H    0 .text         
$dtrace9391.
	      [66]  0x000a6360 0x00000260  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [134]  0x000a7a30 0x0000024d  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [349]  0x000a6a50 0x00000139  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [368]  0x000a6b90 0x00000111  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [473]  0x000a5e40 0x000001ed  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [601]  0x000a6ff0 0x00000073  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [655]  0x000a6646 0x0000026a  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [684]  0x000a76d0 0x00000172  FUNC LOCL  H    0 .text         
$dtrace9391.
	     [762]  0x00000000 0x00000000  FILE LOCL  D    0 ABS           
dtrace_data.

.SUNW_ldynsym lets us report names for static functions instead of hex addresses
when looking at stripped objects. For example, ''pstack'' uses
this, via libproc.
For more about this:

     PSARC 2006/526 SHT_SUNW_LDYNSYM - default local symbol addition
     4934427 runtime linker should load up static symbol names visible to
dladdr()


> 
> Once the following bug is putback, it will be possible for DTrace to mark
> these temporary symbols so that they don''t show up in the final
binary at
> all:
> 
>   6602451 new symbol visibilities required: EXPORTED, SINGLETON and
ELIMINATE
>
I believe Rod filed the RTI for this today, so we''re getting close.

- Ali

Ali Bahrami <Ali.Bahrami at Sun.COM> wrote:

    Ali>  I''m betting that these are local symbols referenced by the
.SUNW_ldynsym
    Ali> section, which is an allocable section, and therefore uses the
    Ali> dynstr:

Are these randomly generated during each build or they are
deterministic? Does it mean that wsdiff will always pick these up as a
change in libc?

- Akolb

    Ali> 	ali at rtld% elfdump -sN.SUNW_ldynsym /lib/libc.so | grep dtrace
    Ali> 	      [12]  0x000a7202 0x0000038f  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	      [66]  0x000a6360 0x00000260  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [134]  0x000a7a30 0x0000024d  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [349]  0x000a6a50 0x00000139  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [368]  0x000a6b90 0x00000111  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [473]  0x000a5e40 0x000001ed  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [601]  0x000a6ff0 0x00000073  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [655]  0x000a6646 0x0000026a  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [684]  0x000a76d0 0x00000172  FUNC LOCL  H    0 .text         
$dtrace9391.
    Ali> 	     [762]  0x00000000 0x00000000  FILE LOCL  D    0 ABS           
dtrace_data.

    Ali> .SUNW_ldynsym lets us report names for static functions instead of
hex addresses
    Ali> when looking at stripped objects. For example,
''pstack'' uses this, via libproc.
    Ali> For more about this:

    Ali>     PSARC 2006/526 SHT_SUNW_LDYNSYM - default local symbol addition
    Ali>     4934427 runtime linker should load up static symbol names
visible to dladdr()



    >> 
    >> Once the following bug is putback, it will be possible for DTrace
to mark
    >> these temporary symbols so that they don''t show up in the
final binary at
    >> all:
    >> 
    >> 6602451 new symbol visibilities required: EXPORTED, SINGLETON and
ELIMINATE
    >> 

    Ali> I believe Rod filed the RTI for this today, so we''re
getting close.

    Ali> - Ali

Alexander Kolbasov wrote:
> Ali Bahrami <Ali.Bahrami at Sun.COM> wrote:
> 
>     Ali>  I''m betting that these are local symbols referenced
by the .SUNW_ldynsym
>     Ali> section, which is an allocable section, and therefore uses the
>     Ali> dynstr:
> 
> Are these randomly generated during each build or they are
> deterministic? Does it mean that wsdiff will always pick these up as a
> change in libc?
> 
> - Akolb

Unless I''m missing something (certainly possible), Adam already covered
the answers to both questions.

 > dtrace -G chooses a randomly generated prefix for the symbols. They should
 > change every time a workspace is built.

This accounts for the fact that the names change.

My bit was to explain why those names are showing up in the dynstr
at all --- because we have a "local dynamic" symbol table, and
those local function symbols are ending up in it. The linker is doing
what it ought to do in this situation.

And, the symbols will disappear once

   6602451 new symbol visibilities required: EXPORTED, SINGLETON and 	
	  ELIMINATE

gets integrated, and then ''dtrace -G'' gets modified to use
the ELIMINATE feature. So wsdiff will be picking up these as changes
in libc until those two things happen.

- Ali

On Wed, 2007-10-03 at 01:15 +0000, Adam Leventhal wrote:
> > thank you for the explanation. What would cause these symbols to
change? The
> > only thing that I changed between build was the introduce parallelism
between
> > uts and lib build plus additional parallelism in uts build.
> 
> dtrace -G chooses a randomly generated prefix for the symbols. They should
> change every time a workspace is built.
But my inner two-year-old compels me to ask:

Why?

if they were actually random, the values in the examples were just large
enough that collisions would occur frequently enough to bother people
but be annoyingly hard to reproduce.

So I cheated.  (I read the libdtrace code; it looks like this is in
usr/src/libdtrace/common/dt_link.c).  

Assuming I found the right place, it''s not actually random.  

It uses ftok(path, 0) on the file; that ftok invocation produces a
24-bit integer based on 12 bits of device and 12 bits of inode number.
This is actually better than a truly random value that small as it makes
collisions less likely, at least on small filesystems. 

I (and, well, anyone who has to use wsdiff) have this thing about build
reproduceability -- it''s best if the same source tree produces the same
objects.  Can we do something more collision-resistant but deterministic
so as to not cause these sorts of false positives for wsdiff?

					- Bill

On Wed, Oct 03, 2007 at 08:46:08AM -0400, Bill Sommerfeld
wrote:
> I (and, well, anyone who has to use wsdiff) have this thing about build
> reproduceability -- it''s best if the same source tree produces the
same
> objects.  Can we do something more collision-resistant but deterministic
> so as to not cause these sorts of false positives for wsdiff?
First, for people who are wondering what the hell wsdiff is, it''s a
tool
that we use at Sun to determine the binary impact of our changes mainly
to see what binaries need to be included in a Solaris patch.

As I mentioned (and Ali reiterated), the fix for this is to exclude those
symbols from the final binary altogether.

Adam

-- 
Adam Leventhal, FishWorks                        http://blogs.sun.com/ahl

On Wed, 2007-10-03 at 16:28 +0000, Adam Leventhal wrote:
> First, for people who are wondering what the hell wsdiff is, it''s
a tool
> that we use at Sun to determine the binary impact of our changes mainly
> to see what binaries need to be included in a Solaris patch.
This may give some people the impression that this is a sun-private
tool.  It''s not.

wsdiff is one of the ON consolidation build tools; source is in
usr/src/tools/scripts; and there are options to "nightly" to run it
after a build.

It''s generally available for anyone to use to understand how far their
source changes percolate out into the result of the build.

						- Bill