llvm dev - Oct 2014 - [LLVMdev] Recent changes in -gmlt break sample profiling

On Fri, Oct 24, 2014 at 4:06 PM, Xinliang David Li <xinliangli at
gmail.com>
wrote:
> Diego,
>
> I think sampleFDO needs to be designed in a way which can protect itself
> from future breakage like this. The roots in the unnecessary dependency of
> sample FDO on gmlt setting. It is totally reasonable to tune debug binary
> size by changes like this.
>
> The right way is to fix this is to introduce an internal -g<...> flag
for
> use by sampleFDO -- it will have a fixed definition of what needs to be
> emitted.
>
FWIW, I strongly disagree.

The more modes we have, the harder everything will be to support and keep
track of. The wide variety of modes used for debug information is already
really challenging to support and maintain. We shouldn't make it harder,
and less-used flags seem like the wrong direction.


At a higher level, I don't think "sample profiling" or
"asan" are good ways
to design a set of debug information that we want emitted in a particular
mode. Instead, we should look at what fundament information a collection of
tools need access to. Both sample profiling, the sanitizers, and crash
backtraces need access to a very minimal amount of information consisting
of line tables, and that is how we designed '-gline-tables-only' (the
LLVM
flag name).

And I think that both this design and Dave's change are totally fine. There
was only one thing we missed: a very particular use case for line tables
that had a particular usage pattern. The problem here is that we don't have
any profile *collection* tests in LLVM. There are some reasons for that
(its hard, etc) but we could probably work on improving it. But the correct
path is the one Dave and Diego identified -- the information *is* still
there, we just need a more clever way of extracting it. And (in addition)
we should probably add some testing strategy for this. =]
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141026/7d4946e7/attachment.html>

The fundamental questions we need to answer are the following:
1) what is -gmlt option designed for? debugging, profiling, autofdo. Do we
expect more use cases for -gmlt?
2) can gmlt's behavior be standardized? The meaning of 'minimal'
really
varies depending on the target use of the information. What is minimally
enough today may become not enough tomorrow if there is a new target use
case identified.
3) Do we have regression tests for other well established use cases, such
as asan?
4) When we need to add more debug info for -gmlt in the future for
enhancement of one of the existing use cases, is it considered a memory and
object size regression and get rejected?

For now, if the minimal debug info (after stripping out DW_TAG_subprogram
etc) already contains enough information for our need, that will be fine.
What if not (now, or in the future)?

David


On Sun, Oct 26, 2014 at 5:44 PM, Chandler Carruth <chandlerc at
google.com>
wrote:
>
> On Fri, Oct 24, 2014 at 4:06 PM, Xinliang David Li <xinliangli at
gmail.com>
> wrote:
>
>> Diego,
>>
>> I think sampleFDO needs to be designed in a way which can protect
itself
>> from future breakage like this. The roots in the unnecessary dependency
of
>> sample FDO on gmlt setting. It is totally reasonable to tune debug
binary
>> size by changes like this.
>>
>> The right way is to fix this is to introduce an internal -g<...>
flag for
>> use by sampleFDO -- it will have a fixed definition of what needs to be
>> emitted.
>>
>
> FWIW, I strongly disagree.
>
> The more modes we have, the harder everything will be to support and keep
> track of. The wide variety of modes used for debug information is already
> really challenging to support and maintain. We shouldn't make it
harder,
> and less-used flags seem like the wrong direction.
>
>
> At a higher level, I don't think "sample profiling" or
"asan" are good
> ways to design a set of debug information that we want emitted in a
> particular mode. Instead, we should look at what fundament information a
> collection of tools need access to. Both sample profiling, the sanitizers,
> and crash backtraces need access to a very minimal amount of information
> consisting of line tables, and that is how we designed
'-gline-tables-only'
> (the LLVM flag name).
>
> And I think that both this design and Dave's change are totally fine.
> There was only one thing we missed: a very particular use case for line
> tables that had a particular usage pattern. The problem here is that we
> don't have any profile *collection* tests in LLVM. There are some
reasons
> for that (its hard, etc) but we could probably work on improving it. But
> the correct path is the one Dave and Diego identified -- the information
> *is* still there, we just need a more clever way of extracting it. And (in
> addition) we should probably add some testing strategy for this. =]
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141026/b2c686d5/attachment.html>

On Sun, Oct 26, 2014 at 7:23 PM, Xinliang David Li <xinliangli at
gmail.com>
wrote:
> The fundamental questions we need to answer are the following:
>
FWIW, there was a longer discussion on the lists when the actual option was
added. I'll try to relay my memory of that discussion, but you might need
to track it down or involve some of the other people who participated in
it. Honestly, this thread is probably not the best place to rehash the
entire design of this feature...

> 1) what is -gmlt option designed for? debugging, profiling, autofdo. Do we
> expect more use cases for -gmlt?
>
It was designed for all users of debug information that merely need
location information rather than variable, type, or other debug
information. We expect these to at least include any and everything that is
primarily symbolizing backtraces and analyzing those. I think sample-based
profiling and PGO is definitely in-scope.

2) can gmlt's behavior be standardized? The meaning of 'minimal'
really
> varies depending on the target use of the information. What is minimally
> enough today may become not enough tomorrow if there is a new target use
> case identified.
>
Again, the Clang flag is not '-gmlt'. It is
'-gline-tables-only'. But I
think both are clear in their intent... Perhaps we could add some
documentation for the flag, but I don't think we can or need to try to
standardize this... It's a pragmatic thing and should continue to be driven
by pragmatic decisions.

> 3) Do we have regression tests for other well established use cases, such
> as asan?
>
Both the ASan and TSan test suites should cover this, yes.

> 4) When we need to add more debug info for -gmlt in the future for
> enhancement of one of the existing use cases, is it considered a memory and
> object size regression and get rejected?
>
This is really open ended and impossible to answer. It would depend
entirely on the nature of the enhancement and the impact it had on the
debug info size. It would likely be discussed and a decision made on the
lists.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20141026/e5d49197/attachment.html>