llvm dev - Apr 2016 - Writing a test for gcov style coverage crashing after dlclose

Hi Everyone,

I have uploaded a patch that allows one to successfully gather gcov/gcda
coverage information on programs which unload shared libraries. It¹s a
simple fix, just adding a few COMPILER_RT_VISIBILITY (i.e.
__attribute__((visibility("hidden")))) in GCDAProfiling.c.

Now, I¹d like to include a test program to demonstrate the fix. AFAICT,
there seems to be a single test for
"-fprofile­arcs/-ftest-coverage/-femit-coverage-data", namely:
tools/clang/test/CodeGen/code-coverage.c. (To be fair, there are some
tests related to the testing of the command line options themselves.)
There seems to be no tests that actually execute the instrumented
executables to gather coverage data.

- Is that a fair assessment of the current situation ?
- Should I create a new test directory such as
llvm/projects/compiler-rt/test/gcov (or gcda) to put my test case ?

Guidance would be appreciated.
Benoit

‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹

See: https://llvm.org/bugs/show_bug.cgi?id=27224


Bug 27224 <https://llvm.org/bugs/show_bug.cgi?id=27224> - gcov /
gcda-based profiling crashes when shared libraries are unloaded (dlclose())


I believe that GCDA profiling functions should be declared as hidden, i.e.
symbols that are not exported by shared libraries. This would allow the
GCDA based profiling to work correctly with code using dlopen/dlclose to
load and unload shared libraries.

Each shared library must have its own copy of the following GCDA functions:

     - __gcov_flush
     - llvm_delete_flush_function_list
     - llvm_delete_writeout_function_list
     - llvm_gcda_emit_arcs
     - llvm_gcda_emit_function
     - llvm_gcda_end_file
     - llvm_gcda_increment_indirect_counter
     - llvm_gcda_start_file
     - llvm_gcda_summary_info
     - llvm_gcov_init
     - llvm_register_flush_function
     - llvm_register_writeout_function
     - llvm_writeout_files

Defining these function as "hidden" prevents a shared library from
exporting them in its interface. The end-result is that each profiled
shared library will be guaranteed to get its own set of the GCDA functions
extracted from libclang_rt.profile.a.

These GCDA functions are referencing the static writeout_fn and flush_fn
lists. These lists are traversed when a shared library is unloaded or the
program exits through an atexit() function. It is important that each
shared library gets its own set of these lists so that the proper .gcda
files are updated when a library is unloaded.

Furthermore, if a shared library doesn't get its own set of GCDA
functions and associated lists, it will leave dangling pointers to its own
"writeout" and "flush" functions in the static writeout_fn
and flush_fn
lists of another shared library. These dangling pointers lead to crashes
when the other shared library is unloaded or when the program exits.



‹‹‹‹‹‹‹‹‹‹‹‹‹‹


Benoit Belley
Sr Principal Developer
M&E-Product Development Group
Autodesk, Inc.
10 Duke Street
Montreal, Quebec, Canada H3C 2L7
www.autodesk.com <http://www.autodesk.com/>

Hi Benoit,
> Now, I¹d like to include a test program to demonstrate the fix. AFAICT,
> there seems to be a single test for
> "-fprofile­arcs/-ftest-coverage/-femit-coverage-data", namely:
> tools/clang/test/CodeGen/code-coverage.c. (To be fair, there are some
> tests related to the testing of the command line options themselves.)
> There seems to be no tests that actually execute the instrumented
> executables to gather coverage data.
> 
> - Is that a fair assessment of the current situation ?
Yes, I think that's fair.

> - Should I create a new test directory such as
> llvm/projects/compiler-rt/test/gcov (or gcda) to put my test case ?
I think such a test could live in, e.g, compiler-rt/test/profile/gcda-$foo.

The integration tests for profiling instrumentation and code coverage also
live in this directory.

thanks
vedant

> 
> Guidance would be appreciated.
> Benoit
> 
> ‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹
> 
> See: https://llvm.org/bugs/show_bug.cgi?id=27224
> 
> 
> Bug 27224 <https://llvm.org/bugs/show_bug.cgi?id=27224> - gcov /
> gcda-based profiling crashes when shared libraries are unloaded (dlclose())
> 
> 
> I believe that GCDA profiling functions should be declared as hidden, i.e.
> symbols that are not exported by shared libraries. This would allow the
> GCDA based profiling to work correctly with code using dlopen/dlclose to
> load and unload shared libraries.
> 
> Each shared library must have its own copy of the following GCDA functions:
> 
>     - __gcov_flush
>     - llvm_delete_flush_function_list
>     - llvm_delete_writeout_function_list
>     - llvm_gcda_emit_arcs
>     - llvm_gcda_emit_function
>     - llvm_gcda_end_file
>     - llvm_gcda_increment_indirect_counter
>     - llvm_gcda_start_file
>     - llvm_gcda_summary_info
>     - llvm_gcov_init
>     - llvm_register_flush_function
>     - llvm_register_writeout_function
>     - llvm_writeout_files
> 
> Defining these function as "hidden" prevents a shared library
from
> exporting them in its interface. The end-result is that each profiled
> shared library will be guaranteed to get its own set of the GCDA functions
> extracted from libclang_rt.profile.a.
> 
> These GCDA functions are referencing the static writeout_fn and flush_fn
> lists. These lists are traversed when a shared library is unloaded or the
> program exits through an atexit() function. It is important that each
> shared library gets its own set of these lists so that the proper .gcda
> files are updated when a library is unloaded.
> 
> Furthermore, if a shared library doesn't get its own set of GCDA
> functions and associated lists, it will leave dangling pointers to its own
> "writeout" and "flush" functions in the static
writeout_fn and flush_fn
> lists of another shared library. These dangling pointers lead to crashes
> when the other shared library is unloaded or when the program exits.
> 
> 
> 
> ‹‹‹‹‹‹‹‹‹‹‹‹‹‹
> 
> 
> Benoit Belley
> Sr Principal Developer
> M&E-Product Development Group
> Autodesk, Inc.
> 10 Duke Street
> Montreal, Quebec, Canada H3C 2L7
> www.autodesk.com <http://www.autodesk.com/>
> 
> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Excellent! Thanks for the guidance. I’ll follow your suggestion.


Cheers,
Benoit

On 16-04-07 20:03, "vsk at apple.com on behalf of Vedant Kumar"
<vsk at apple.com> wrote:
>Hi Benoit,
>
>> Now, I¹d like to include a test program to demonstrate the fix. AFAICT,
>> there seems to be a single test for
>> "-fprofile­arcs/-ftest-coverage/-femit-coverage-data",
namely:
>> tools/clang/test/CodeGen/code-coverage.c. (To be fair, there are some
>> tests related to the testing of the command line options themselves.)
>> There seems to be no tests that actually execute the instrumented
>> executables to gather coverage data.
>> 
>> - Is that a fair assessment of the current situation ?
>
>Yes, I think that's fair.
>
>
>> - Should I create a new test directory such as
>> llvm/projects/compiler-rt/test/gcov (or gcda) to put my test case ?
>
>I think such a test could live in, e.g,
>compiler-rt/test/profile/gcda-$foo.
>
>The integration tests for profiling instrumentation and code coverage also
>live in this directory.
>
>thanks
>vedant
>
>
>> 
>> Guidance would be appreciated.
>> Benoit
>> 
>> ‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹‹
>> 
>> See: https://llvm.org/bugs/show_bug.cgi?id=27224
>> 
>> 
>> Bug 27224 <https://llvm.org/bugs/show_bug.cgi?id=27224> - gcov /
>> gcda-based profiling crashes when shared libraries are unloaded
>>(dlclose())
>> 
>> 
>> I believe that GCDA profiling functions should be declared as hidden,
>>i.e.
>> symbols that are not exported by shared libraries. This would allow the
>> GCDA based profiling to work correctly with code using dlopen/dlclose
to
>> load and unload shared libraries.
>> 
>> Each shared library must have its own copy of the following GCDA
>>functions:
>> 
>>     - __gcov_flush
>>     - llvm_delete_flush_function_list
>>     - llvm_delete_writeout_function_list
>>     - llvm_gcda_emit_arcs
>>     - llvm_gcda_emit_function
>>     - llvm_gcda_end_file
>>     - llvm_gcda_increment_indirect_counter
>>     - llvm_gcda_start_file
>>     - llvm_gcda_summary_info
>>     - llvm_gcov_init
>>     - llvm_register_flush_function
>>     - llvm_register_writeout_function
>>     - llvm_writeout_files
>> 
>> Defining these function as "hidden" prevents a shared library
from
>> exporting them in its interface. The end-result is that each profiled
>> shared library will be guaranteed to get its own set of the GCDA
>>functions
>> extracted from libclang_rt.profile.a.
>> 
>> These GCDA functions are referencing the static writeout_fn and
flush_fn
>> lists. These lists are traversed when a shared library is unloaded or
>>the
>> program exits through an atexit() function. It is important that each
>> shared library gets its own set of these lists so that the proper .gcda
>> files are updated when a library is unloaded.
>> 
>> Furthermore, if a shared library doesn't get its own set of GCDA
>> functions and associated lists, it will leave dangling pointers to its
>>own
>> "writeout" and "flush" functions in the static
writeout_fn and flush_fn
>> lists of another shared library. These dangling pointers lead to
crashes
>> when the other shared library is unloaded or when the program exits.
>> 
>> 
>> 
>> ‹‹‹‹‹‹‹‹‹‹‹‹‹‹
>> 
>> 
>> Benoit Belley
>> Sr Principal Developer
>> M&E-Product Development Group
>> Autodesk, Inc.
>> 10 Duke Street
>> Montreal, Quebec, Canada H3C 2L7
>> www.autodesk.com <http://www.autodesk.com/>
>> 
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>