dtrace discuss - Oct 2009 - Question on function tracing using pid provider on stripped programs

Hello,

I tried this script: http://www.brendangregg.com/DTrace/dappprof (using pid
provider to
construct call graph by tracing function entries/returns), on an ELF binary
with symbol
table (.symtab) stripped away. The Dtrace script was still able to generates
the call
graph with symbolic function names and addresses, for example:

....
  874/1:       450       .      .         ->
ld.so.1:stravl_insert(0xD17C2679, 0x0, 0x0)
  874/1:       450       .      .           -> ld.so.1:pnavl_create(0x18,
0x0, 0x0)
  874/1:       450       .      .             -> ld.so.1:malloc(0x14, 0x0,
0x0)
  874/1:       450       .      .               -> ld.so.1:split(0xD17FEB78,
0x18, 0x0)
  874/1:       450       0      0               <- ld.so.1:split = 79
  874/1:       450       0      0             <- ld.so.1:malloc = 212
  874/1:       450       .      .             ->
ld.so.1:avl_create(0xD17FEB88, 0xD17D2EC0, 0x18)
  874/1:       450       0      0             <- ld.so.1:avl_create = 32
  874/1:       450       0      0           <- ld.so.1:pnavl_create = 66
  874/1:       450       .      .           -> ld.so.1:strlen(0xD17C2679,
0x0, 0x0)
  874/1:       450       0      0           <- ld.so.1:strlen = 85
....

I am really curious and interested to know how the pid provider can figure
out the symbolic
information of the called functions from a stripped program. Could someone
help me
demystify the mechanism?

Thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20091007/44e8191c/attachment.html>

> I am really curious and interested to know how the pid provider can  
> figure out the symbolic
> information of the called functions from a stripped program. Could  
> someone help me
> demystify the mechanism?
First, stripping a program is the bane of debugging since it deprives  
the
debugger of a bunch of symbolic information. However, even a stripped  
program
must contain some symbolic information for doing dynamic linker lookups.
For example, if a program calls a function in a library that symbol name
will survive stripping.

Do a nm -D on your program to see that ''dynamic'' symbol table.

Adam

--
Adam Leventhal, Fishworks                        http://blogs.sun.com/ahl

Ling wrote:
> Hello,
> 
> I tried this script: http://www.brendangregg.com/DTrace/dappprof (using 
> pid provider to
> construct call graph by tracing function entries/returns), on an ELF 
> binary with symbol
> table (.symtab) stripped away. The Dtrace script was still able to 
> generates the call
> graph with symbolic function names and addresses, for example:
> 
> ....
>   874/1:       450       .      .         ->
ld.so.1:stravl_insert(0xD17C2679, 0x0, 0x0)
>   874/1:       450       .      .           ->
ld.so.1:pnavl_create(0x18, 0x0, 0x0)
>   874/1:       450       .      .             -> ld.so.1:malloc(0x14,
0x0, 0x0)
>   874/1:       450       .      .               ->
ld.so.1:split(0xD17FEB78, 0x18, 0x0)
>   874/1:       450       0      0               <- ld.so.1:split = 79
>   874/1:       450       0      0             <- ld.so.1:malloc = 212
>   874/1:       450       .      .             ->
ld.so.1:avl_create(0xD17FEB88, 0xD17D2EC0, 0x18)
>   874/1:       450       0      0             <- ld.so.1:avl_create = 32
>   874/1:       450       0      0           <- ld.so.1:pnavl_create = 66
>   874/1:       450       .      .           ->
ld.so.1:strlen(0xD17C2679, 0x0, 0x0)
>   874/1:       450       0      0           <- ld.so.1:strlen = 85
> ....
> 
> I am really curious and interested to know how the pid provider can 
> figure out the symbolic
> information of the called functions from a stripped program. Could 
> someone help me
> demystify the mechanism?
> 
> Thanks in advance.
> 
> 

    If I''m reading this correctly, DTrace is reporting on symbols found
in the runtime linker (ld.so.1), which is not stripped. Stripping your
object has no effect on the other system objects it is linked to, so
it''s
not surprising that this works.

However, you will find that Solaris can figure out most symbols of interest
in your stripped object as well. As Adam points out, this comes from the dynamic
symbol table, and the related SUNW_ldynsym symbol table, neither of which
can be stripped. If the .symbtab is gone, the system is able to use these
instead, and will be able to resolve all functions, as well as global data.

Here are some things from my blog that you might help you understand
more about the underpinnings:

     http://blogs.sun.com/ali/entry/inside_elf_symbol_tables
     http://blogs.sun.com/ali/entry/what_is_sunw_ldynsym
     http://blogs.sun.com/ali/entry/which_solaris_files_are_stripped

I don''t think you should strip your objects though --- there''s
little
or any benefit, and it does hurt observability.

Ali

Thanks, Adam!
I am aware of the dynamic symbol section and relocation table in ELF. I have
a picture of how
dynamic loader relocates and patches dynamic functions. What I am curious is
that how can
DTrace still get the static function names (those functions built into the
program itself, or static
functions)?  I have a hypothesis that .hash section is used since it is
normally available.

-Ling

On Thu, Oct 8, 2009 at 12:14 PM, Adam Leventhal <ahl at eng.sun.com>
wrote:
> I am really curious and interested to know how the pid provider can figure
>> out the symbolic
>> information of the called functions from a stripped program. Could
someone
>> help me
>> demystify the mechanism?
>>
>
> First, stripping a program is the bane of debugging since it deprives the
> debugger of a bunch of symbolic information. However, even a stripped
> program
> must contain some symbolic information for doing dynamic linker lookups.
> For example, if a program calls a function in a library that symbol name
> will survive stripping.
>
> Do a nm -D on your program to see that ''dynamic'' symbol
table.
>
> Adam
>
> --
> Adam Leventhal, Fishworks                        http://blogs.sun.com/ahl
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20091008/100b70c7/attachment.html>

Thank you Ali!
Yes, you are right, the example I gave was not stripped.
By reading your blog I am now  much more clear about the symbol tables in
ELF.
Therefore, DTrace obtains the local function names and address via
SUNW_ldynsym symbol table when .symtab section is gone ?

A very simple example is main( ) call foo( ), foo( ) is a local function in
the program. I stripped the
.symtab of this program, then I use DTrace and got the following result.

CALL(args)          = return
  -> _start(0x8047E50, 0x0, 0x8047E57)
    -> __fsr(0x1, 0x8047D8C, 0x8047D94)
    <- __fsr = 125
    -> main(0x1, 0x8047D8C, 0x8047D94)
      -> foo(0xA, 0x8047D64, 0xD17C6F04)
      <- foo = 10
      -> foo(0xA, 0x8047D64, 0xD17C6F04)
      <- foo = 10
    <- main = 83

Thanks!
Ling

>   If I''m reading this correctly, DTrace is reporting on symbols
found
> in the runtime linker (ld.so.1), which is not stripped. Stripping your
> object has no effect on the other system objects it is linked to, so
it''s
> not surprising that this works.
>
> However, you will find that Solaris can figure out most symbols of interest
> in your stripped object as well. As Adam points out, this comes from the
> dynamic
> symbol table, and the related SUNW_ldynsym symbol table, neither of which
> can be stripped. If the .symbtab is gone, the system is able to use these
> instead, and will be able to resolve all functions, as well as global data.
>
> Here are some things from my blog that you might help you understand
> more about the underpinnings:
>
>    http://blogs.sun.com/ali/entry/inside_elf_symbol_tables
>    http://blogs.sun.com/ali/entry/what_is_sunw_ldynsym
>    http://blogs.sun.com/ali/entry/which_solaris_files_are_stripped
>
> I don''t think you should strip your objects though ---
there''s little
> or any benefit, and it does hurt observability.
>
> Ali
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20091012/1346b878/attachment.html>

Ling wrote:
> Thank you Ali!
> Yes, you are right, the example I gave was not stripped.
> By reading your blog I am now  much more clear about the symbol tables 
> in ELF.
> Therefore, DTrace obtains the local function names and address via 
> SUNW_ldynsym symbol table when .symtab section is gone ?
> 
    Yes, that''s right. The code for that is actually in libproc (which
DTrace uses). That means that the ptools (pstack in particular), mdb, and
other libproc users all get the same treatment without having to do
anything special.

- Ali

Thank you so much Ali!

-Ling

On Mon, Oct 12, 2009 at 11:45 AM, Ali Bahrami <Ali.Bahrami at sun.com>
wrote:
> Ling wrote:
>
>> Thank you Ali!
>> Yes, you are right, the example I gave was not stripped.
>> By reading your blog I am now  much more clear about the symbol tables
in
>> ELF.
>> Therefore, DTrace obtains the local function names and address via
>> SUNW_ldynsym symbol table when .symtab section is gone ?
>>
>>
>   Yes, that''s right. The code for that is actually in libproc
(which
> DTrace uses). That means that the ptools (pstack in particular), mdb, and
> other libproc users all get the same treatment without having to do
> anything special.
>
> - Ali
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/dtrace-discuss/attachments/20091012/08a9c684/attachment.html>