llvm dev - Mar 2018 - llvm-cov: Combined report for multiple executables

Hi! I am trying to get a combined coverage report from multiple
executables. Looking at earlier discussions [1, 2], it looks like this
is supposed to work. I am having some difficulty getting this to work
as I would expect it to work, however. Following is a simple case to
explain:

////////// shared.h
#include <string>

void Print1(const std::string& msg);
void Print2(const std::string& msg);
////////// end

////////// shared.cc
#include <iostream>
#include <string>

void Print1(const std::string& msg) {
  std::cout << "1" << msg;
}

void Print2(const std::string& msg) {
  std::cout << "2" << msg;
}
////////// end

////////// first.cc
#include "shared.h"

int main() {
  Print1("First\n");
  return 0;
}
////////// end

////////// second.cc
#include "shared.h"

int main() {
  Print2("Second\n");
  return 0;
}
////////// end

I use the following commands to build 'first' and 'second'
binaries:
  clang++ -O0 -c -fprofile-instr-generate -fcoverage-mapping shared.cc
-o ./out/shared.o
  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o first.cc -o ./out/first
  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
./out/shared.o second.cc -o ./out/second

I then use the following to run the binaries:
  LLVM_PROFILE_FILE='./out/first.profraw' ./out/first
  LLVM_PROFILE_FILE='./out/second.profraw' ./out/second

I then use the following commands to show the report:
  llvm-profdata merge -sparse ./out/first.profraw ./out/second.profraw
-o ./out/all.profdata
  llvm-cov show ./out/second -object ./out/first -instr-profile
./out/all.profdata

But the output only contains:

second.cc:
    1|       |#include "shared.h"
    2|       |
    3|      2|int main() {
    4|      2|  Print2("Second\n");
    5|      2|  return 0;
    6|      2|}

shared.cc:
    1|       |#include <iostream>
    2|       |#include <string>
    3|       |
    4|      1|void Print1(const std::string& msg) {
    5|      1|  std::cout << "1" << msg;
    6|      1|}
    7|       |
    8|      1|void Print2(const std::string& msg) {
    9|      1|  std::cout << "2" << msg;
   10|      1|}

The output does not include anything for first.cc If I run the last
command with ./out/second and ./out/first swapped, then the output
only contains first.cc and shared.cc, but does not have second.cc

Is my expectation that the report should include both first.cc and
second.cc correct? If it is, then I am probably holding something
wrong? Any suggestions on what I should try?

[1]
https://groups.google.com/forum/#!msg/llvm-dev/11mEQr1t2ys/9SOI39v-AAAJ;context-place=forum/llvm-dev
[2] https://reviews.llvm.org/D25086#change-SchnPUmKhFiN

Thank you!
Sadrul

Hi Sadrul,
> On Mar 8, 2018, at 7:40 PM, Sadrul Chowdhury via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Hi! I am trying to get a combined coverage report from multiple
> executables. Looking at earlier discussions [1, 2], it looks like this
> is supposed to work. I am having some difficulty getting this to work
> as I would expect it to work, however. Following is a simple case to
> explain:
> 
> ////////// shared.h
> #include <string>
> 
> void Print1(const std::string& msg);
> void Print2(const std::string& msg);
> ////////// end
> 
> ////////// shared.cc
> #include <iostream>
> #include <string>
> 
> void Print1(const std::string& msg) {
>  std::cout << "1" << msg;
> }
> 
> void Print2(const std::string& msg) {
>  std::cout << "2" << msg;
> }
> ////////// end
> 
> ////////// first.cc
> #include "shared.h"
> 
> int main() {
>  Print1("First\n");
>  return 0;
> }
> ////////// end
> 
> ////////// second.cc
> #include "shared.h"
> 
> int main() {
>  Print2("Second\n");
>  return 0;
> }
> ////////// end
> 
> I use the following commands to build 'first' and 'second'
binaries:
>  clang++ -O0 -c -fprofile-instr-generate -fcoverage-mapping shared.cc
> -o ./out/shared.o
>  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
> ./out/shared.o first.cc -o ./out/first
>  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
> ./out/shared.o second.cc -o ./out/second
> 
> I then use the following to run the binaries:
>  LLVM_PROFILE_FILE='./out/first.profraw' ./out/first
>  LLVM_PROFILE_FILE='./out/second.profraw' ./out/second
> 
> I then use the following commands to show the report:
>  llvm-profdata merge -sparse ./out/first.profraw ./out/second.profraw
> -o ./out/all.profdata
>  llvm-cov show ./out/second -object ./out/first -instr-profile
> ./out/all.profdata
> 
> But the output only contains:
> 
> second.cc:
>    1|       |#include "shared.h"
>    2|       |
>    3|      2|int main() {
>    4|      2|  Print2("Second\n");
>    5|      2|  return 0;
>    6|      2|}
> 
> shared.cc:
>    1|       |#include <iostream>
>    2|       |#include <string>
>    3|       |
>    4|      1|void Print1(const std::string& msg) {
>    5|      1|  std::cout << "1" << msg;
>    6|      1|}
>    7|       |
>    8|      1|void Print2(const std::string& msg) {
>    9|      1|  std::cout << "2" << msg;
>   10|      1|}
> 
> The output does not include anything for first.cc If I run the last
> command with ./out/second and ./out/first swapped, then the output
> only contains first.cc and shared.cc, but does not have second.cc
> 
> Is my expectation that the report should include both first.cc and
> second.cc correct? If it is, then I am probably holding something
> wrong? Any suggestions on what I should try?
The coverage tool currently has limitations which require it to assume that the
one definition rule holds over all of the tool inputs. That means that if you
have multiple definitions of the same symbol (in this case "main"),
the coverage tool will only report one of them.

You might try reorganizing the code so that there's only one function called
main.

best,
vedant

> 
> [1]
https://groups.google.com/forum/#!msg/llvm-dev/11mEQr1t2ys/9SOI39v-AAAJ;context-place=forum/llvm-dev
> [2] https://reviews.llvm.org/D25086#change-SchnPUmKhFiN
> 
> Thank you!
> Sadrul
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Fri, Mar 9, 2018 at 6:07 PM, Vedant Kumar <vsk at apple.com>
wrote:
> Hi Sadrul,
>
>> On Mar 8, 2018, at 7:40 PM, Sadrul Chowdhury via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>
>> Hi! I am trying to get a combined coverage report from multiple
>> executables. Looking at earlier discussions [1, 2], it looks like this
>> is supposed to work. I am having some difficulty getting this to work
>> as I would expect it to work, however. Following is a simple case to
>> explain:
>>
>> ////////// shared.h
>> #include <string>
>>
>> void Print1(const std::string& msg);
>> void Print2(const std::string& msg);
>> ////////// end
>>
>> ////////// shared.cc
>> #include <iostream>
>> #include <string>
>>
>> void Print1(const std::string& msg) {
>>  std::cout << "1" << msg;
>> }
>>
>> void Print2(const std::string& msg) {
>>  std::cout << "2" << msg;
>> }
>> ////////// end
>>
>> ////////// first.cc
>> #include "shared.h"
>>
>> int main() {
>>  Print1("First\n");
>>  return 0;
>> }
>> ////////// end
>>
>> ////////// second.cc
>> #include "shared.h"
>>
>> int main() {
>>  Print2("Second\n");
>>  return 0;
>> }
>> ////////// end
>>
>> I use the following commands to build 'first' and
'second' binaries:
>>  clang++ -O0 -c -fprofile-instr-generate -fcoverage-mapping shared.cc
>> -o ./out/shared.o
>>  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
>> ./out/shared.o first.cc -o ./out/first
>>  clang++ -O0 -fprofile-instr-generate -fcoverage-mapping
>> ./out/shared.o second.cc -o ./out/second
>>
>> I then use the following to run the binaries:
>>  LLVM_PROFILE_FILE='./out/first.profraw' ./out/first
>>  LLVM_PROFILE_FILE='./out/second.profraw' ./out/second
>>
>> I then use the following commands to show the report:
>>  llvm-profdata merge -sparse ./out/first.profraw ./out/second.profraw
>> -o ./out/all.profdata
>>  llvm-cov show ./out/second -object ./out/first -instr-profile
>> ./out/all.profdata
>>
>> But the output only contains:
>>
>> second.cc:
>>    1|       |#include "shared.h"
>>    2|       |
>>    3|      2|int main() {
>>    4|      2|  Print2("Second\n");
>>    5|      2|  return 0;
>>    6|      2|}
>>
>> shared.cc:
>>    1|       |#include <iostream>
>>    2|       |#include <string>
>>    3|       |
>>    4|      1|void Print1(const std::string& msg) {
>>    5|      1|  std::cout << "1" << msg;
>>    6|      1|}
>>    7|       |
>>    8|      1|void Print2(const std::string& msg) {
>>    9|      1|  std::cout << "2" << msg;
>>   10|      1|}
>>
>> The output does not include anything for first.cc If I run the last
>> command with ./out/second and ./out/first swapped, then the output
>> only contains first.cc and shared.cc, but does not have second.cc
>>
>> Is my expectation that the report should include both first.cc and
>> second.cc correct? If it is, then I am probably holding something
>> wrong? Any suggestions on what I should try?
>
> The coverage tool currently has limitations which require it to assume that
the one definition rule holds over all of the tool inputs. That means that if
you have multiple definitions of the same symbol (in this case
"main"), the coverage tool will only report one of them.
Ah, I see. If I add some functions other than main() in first.cc and
second.cc, those all do correctly show up (except one of the 'main'
functions show no-coverage). That makes sense. Thank you very much for
the quick reply!

Sadrul
>
> You might try reorganizing the code so that there's only one function
called main.
>
> best,
> vedant
>
>
>>
>> [1]
https://groups.google.com/forum/#!msg/llvm-dev/11mEQr1t2ys/9SOI39v-AAAJ;context-place=forum/llvm-dev
>> [2] https://reviews.llvm.org/D25086#change-SchnPUmKhFiN
>>
>> Thank you!
>> Sadrul
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>