llvm dev - May 2019 - How data is laid out in default.profraw when doing profiling?

Hi all,

  I'm now working on llvm-cov for a new target and have a problem here. 
  Because of some reasons, users do not stub in the main function and  after
running elf file, they cannot get a default profraw. Now they want to construct
a default profraw manually but don't know how data is laid out in `default
profraw` file.
  We found a struct ProfDataIOVec in InstrProfilingWriter.c in compiler-rt and
followed this layout.
```
  ProfDataIOVec IOVec[] = {
      {&Header, sizeof(__llvm_profile_header), 1},
      {DataBegin, sizeof(__llvm_profile_data), DataSize},
      {CountersBegin, sizeof(uint64_t), CountersSize},
      {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
      {Zeroes, sizeof(uint8_t), Padding}};
```
  This helped us successfully passed some small cases(not know for sure whether
the coverage result is true, but we can see coverage report). But when we added
more cases, `llvm-cov show -arch=xxx ...` gave us an assertion when entering
fuction ` loadFunctionRecord ` in CoverageMapping.cpp. The OrigFuncName is empty
when inserting into FunctionNames. Seems the llvm-cov cannot find a function
name. I think it's because we didn't construct a correct
default.profraw. So how data is laid out in default.profraw when doing
profiling? Any help will be appreciated.
  A small tip which may help. We are using coverage tool for a new target so
when building compiler-rt, we use InstrProfilingPlatformOther.c

Best,
Ruobin

I would recommend reusing the existing code for writing those files, if
possible. If atexit() doesn't work in your environment, you can still invoke
the routines explicitly.
https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#using-the-profiling-runtime-without-static-initializers
describes this.

If llvm-cov isn't working, the most likely cause is that there's
something wrong with your ELF file.  Are you sure your ELF file matches the
profile data, and has all the relevant __llvm_* sections?

-Eli
> -----Original Message-----
> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of
xuruobin via
> llvm-dev
> Sent: Wednesday, May 15, 2019 8:27 PM
> To: llvm-dev at lists.llvm.org
> Cc: Yuchao (Michael) <michael.yuchao at huawei.com>
> Subject: [EXT] [llvm-dev] How data is laid out in default.profraw when
doing
> profiling?
> 
> Hi all,
> 
>   I'm now working on llvm-cov for a new target and have a problem here.
>   Because of some reasons, users do not stub in the main function and 
after
> running elf file, they cannot get a default profraw. Now they want to
construct a
> default profraw manually but don't know how data is laid out in
`default
> profraw` file.
>   We found a struct ProfDataIOVec in InstrProfilingWriter.c in compiler-rt
and
> followed this layout.
> ```
>   ProfDataIOVec IOVec[] = {
>       {&Header, sizeof(__llvm_profile_header), 1},
>       {DataBegin, sizeof(__llvm_profile_data), DataSize},
>       {CountersBegin, sizeof(uint64_t), CountersSize},
>       {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t), NamesSize},
>       {Zeroes, sizeof(uint8_t), Padding}};
> ```
>   This helped us successfully passed some small cases(not know for sure
whether
> the coverage result is true, but we can see coverage report). But when we
added
> more cases, `llvm-cov show -arch=xxx ...` gave us an assertion when
entering
> fuction ` loadFunctionRecord ` in CoverageMapping.cpp. The OrigFuncName is
> empty when inserting into FunctionNames. Seems the llvm-cov cannot find a
> function name. I think it's because we didn't construct a correct
default.profraw.
> So how data is laid out in default.profraw when doing profiling? Any help
will be
> appreciated.
>   A small tip which may help. We are using coverage tool for a new target
so
> when building compiler-rt, we use InstrProfilingPlatformOther.c
> 
> Best,
> Ruobin
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi Eli,

    For the first problem, I will try reuse the existing code.
    For the second problem that my llvm-cov isn't working, I compared elf
file with original one(no -fprofile-instr-generate  -fcoverage-mapping flag). 5
new sections were introduced. __llvm_prf_names, __llvm_prf_cnts,
__llvm_prf_data, __llvm_prf_vnds and __llvm_covmap. Does the sequence of 5
sections matter or are there more sections I need to use?

Best,
Ruobin
> -----Original Message-----
> From: Eli Friedman [mailto:efriedma at quicinc.com]
> Sent: Friday, May 17, 2019 2:44 AM
> To: xuruobin <xuruobin at huawei.com>; llvm-dev <llvm-dev at
lists.llvm.org>
> Cc: Yuchao (Michael) <michael.yuchao at huawei.com>
> Subject: RE: [llvm-dev] How data is laid out in default.profraw when doing
> profiling?
> 
> I would recommend reusing the existing code for writing those files, if
> possible. If atexit() doesn't work in your environment, you can still
invoke the
> routines explicitly.
> https://clang.llvm.org/docs/SourceBasedCodeCoverage.html#using-the-
> profiling-runtime-without-static-initializers describes this.
> 
> If llvm-cov isn't working, the most likely cause is that there's
something
> wrong with your ELF file.  Are you sure your ELF file matches the profile
data,
> and has all the relevant __llvm_* sections?
> 
> -Eli
> 
> > -----Original Message-----
> > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of
xuruobin
> > via llvm-dev
> > Sent: Wednesday, May 15, 2019 8:27 PM
> > To: llvm-dev at lists.llvm.org
> > Cc: Yuchao (Michael) <michael.yuchao at huawei.com>
> > Subject: [EXT] [llvm-dev] How data is laid out in default.profraw when
> > doing profiling?
> >
> > Hi all,
> >
> >   I'm now working on llvm-cov for a new target and have a problem
here.
> >   Because of some reasons, users do not stub in the main function and
> > after running elf file, they cannot get a default profraw. Now they
> > want to construct a default profraw manually but don't know how
data
> > is laid out in `default profraw` file.
> >   We found a struct ProfDataIOVec in InstrProfilingWriter.c in
> > compiler-rt and followed this layout.
> > ```
> >   ProfDataIOVec IOVec[] = {
> >       {&Header, sizeof(__llvm_profile_header), 1},
> >       {DataBegin, sizeof(__llvm_profile_data), DataSize},
> >       {CountersBegin, sizeof(uint64_t), CountersSize},
> >       {SkipNameDataWrite ? NULL : NamesBegin, sizeof(uint8_t),
NamesSize},
> >       {Zeroes, sizeof(uint8_t), Padding}}; ```
> >   This helped us successfully passed some small cases(not know for
> > sure whether the coverage result is true, but we can see coverage
> > report). But when we added more cases, `llvm-cov show -arch=xxx ...`
> > gave us an assertion when entering fuction ` loadFunctionRecord ` in
> > CoverageMapping.cpp. The OrigFuncName is empty when inserting into
> > FunctionNames. Seems the llvm-cov cannot find a function name. I think
> it's because we didn't construct a correct default.profraw.
> > So how data is laid out in default.profraw when doing profiling? Any
> > help will be appreciated.
> >   A small tip which may help. We are using coverage tool for a new
> > target so when building compiler-rt, we use
> > InstrProfilingPlatformOther.c
> >
> > Best,
> > Ruobin
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev