llvm dev - May 2020 - [EXTERNAL] How to get branch coverage by using 'source-based code coverage'

Hi, llvm/clang experts

I need to get the branch coverage for some testing code. But i found gcov
can't give a expected coverage which may
count some 'hidden branch' in (See stackoverflow answer
<https://stackoverflow.com/questions/42003783/lcov-gcov-branch-coverage-with-c-producing-branches-all-over-the-place>).
Instead, I turn to use clang and the 'source-based code coverage'
feature
may be a right choice. But i can't find anything to describe branch
coverage explicitly on the official site
<https://clang.llvm.org/docs/SourceBasedCodeCoverage.html>.

So how to use clang to get branch coverage by utilizing
'source-based...'
feature?  And is *region coverage *equivalent to branch coverage?
(I tested on a simple program and region coverage seems pretty close to
branch coverage)

Any help is highly appreciated

Thanks,
Ted Xie
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200427/d9aad1e1/attachment-0001.html>

Hi Ted,

Presently, branch coverage is not supported in clang source-based code coverage,
but I am working on implementing branch condition coverage right now.  It will
track True/False branches for each leaf-level condition, including within
Boolean expressions comprised of logical operators (“&&”, “||”). 
Technically, this condition-based coverage is more granular than how “branch
coverage” is often defined (tracking on that control flow decisions allow for
all regions to be hit), and is closer to the granularity that GCOV provides, but
it’s anchored to the source code, won’t count “hidden branches”, and isn’t
negatively impacted by optimization (although the reverse isn’t true: coverage
instrumentation may impact optimization).

My work is a step closer to getting us toward Modified Condition/Decision
Coverage (MC/DC).  I hope to upstream my work within the next few months, so
unfortunately it won’t help you in the short-term.  Region coverage may be
enough for what you need if you can ascertain based on the coverage that
control-flow decisions have evaluated to both true and false, but I’ll let
others comment here with suggestions.

-Alan Phipps

From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of LoveZhi
Xie via llvm-dev
Sent: Sunday, April 26, 2020 11:36 AM
To: llvm-dev at lists.llvm.org
Subject: [EXTERNAL] [llvm-dev] How to get branch coverage by using
'source-based code coverage'

Hi, llvm/clang experts

I need to get the branch coverage for some testing code. But i found gcov
can't give a expected coverage which may
count some 'hidden branch' in (See stackoverflow
answer<https://stackoverflow.com/questions/42003783/lcov-gcov-branch-coverage-with-c-producing-branches-all-over-the-place>).
Instead, I turn to use clang and the 'source-based code coverage'
feature
may be a right choice. But i can't find anything to describe branch coverage
explicitly on the official
site<https://clang.llvm.org/docs/SourceBasedCodeCoverage.html>.

So how to use clang to get branch coverage by utilizing
'source-based...' feature?  And is region coverage equivalent to branch
coverage?
(I tested on a simple program and region coverage seems pretty close to branch
coverage)

Any help is highly appreciated

Thanks,
Ted Xie


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200428/a0cd3cc6/attachment.html>

Hi, Alan

Really very excited to receive your email and sorry to be slow replying, it
has been exceptionally busy over the last few days ;(

Your explanation made the problem clear to me. So gcov branch coverage
should be called condition coverage and clang region coverage
is branch coverage in fact(also known as *decision/C1*), right?

And llvm/clang will support all the following coverage criteria in future

   - Line coverage
   - Function coverage
   - Branch coverage/region coverage
   - Condition coverage
   - Modified Condition/Decision Coverage

If there is any misunderstanding here, please point it out.

Thanks,
Ted Xie



Phipps, Alan <a-phipps at ti.com> 于2020年4月28日周二 下午10:14写道：
> Hi Ted,
>
>
>
> Presently, branch coverage is not supported in clang source-based code
> coverage, but I am working on implementing branch condition coverage right
> now.  It will track True/False branches for each leaf-level condition,
> including within Boolean expressions comprised of logical operators
(“&&”,
> “||”).  Technically, this condition-based coverage is more granular than
> how “branch coverage” is often defined (tracking on that control flow
> decisions allow for all regions to be hit), and is closer to the
> granularity that GCOV provides, but it’s anchored to the source code, won’t
> count “hidden branches”, and isn’t negatively impacted by optimization
> (although the reverse isn’t true: coverage instrumentation may impact
> optimization).
>
>
>
> My work is a step closer to getting us toward Modified Condition/Decision
> Coverage (MC/DC).  I hope to upstream my work within the next few months,
> so unfortunately it won’t help you in the short-term.  Region coverage may
> be enough for what you need if you can ascertain based on the coverage that
> control-flow decisions have evaluated to both true and false, but I’ll let
> others comment here with suggestions.
>
>
>
> -Alan Phipps
>
>
>
> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of
*LoveZhi
> Xie via llvm-dev
> *Sent:* Sunday, April 26, 2020 11:36 AM
> *To:* llvm-dev at lists.llvm.org
> *Subject:* [EXTERNAL] [llvm-dev] How to get branch coverage by using
> 'source-based code coverage'
>
>
>
> Hi, llvm/clang experts
>
>
>
> I need to get the branch coverage for some testing code. But i found gcov
> can't give a expected coverage which may
>
> count some 'hidden branch' in (See stackoverflow answer
>
<https://stackoverflow.com/questions/42003783/lcov-gcov-branch-coverage-with-c-producing-branches-all-over-the-place>).
> Instead, I turn to use clang and the 'source-based code coverage'
feature
>
> may be a right choice. But i can't find anything to describe branch
> coverage explicitly on the official site
> <https://clang.llvm.org/docs/SourceBasedCodeCoverage.html>.
>
>
>
> So how to use clang to get branch coverage by utilizing
'source-based...'
> feature?  And is *region coverage *equivalent to branch coverage?
>
> (I tested on a simple program and region coverage seems pretty close to
> branch coverage)
>
>
>
> Any help is highly appreciated
>
>
>
> Thanks,
>
> Ted Xie
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200504/664917ab/attachment.html>