llvm dev - May 2017 - Enable STATISTIC all the time again?

Right now, the LLVM Statistic class does not increment values if we are in
non-debug mode, unless you define LLVM_ENABLE_STATS (which, sadly, is also
not available from cmake).

Before i go and add it to cmake, i ran numbers.

I can't find a single testcase, large or small, where enabling statistics
all the time isn't completely noise.

It looks like it was disabled in march  2013.
https://reviews.llvm.org/D486

The only discussion i can find in the same time period is around fastisel
using a lot of per-instruction stats, and it causing some slow-down.

However, I can't find any actual data/testcases in this discussion at all,
and disabling stats globally is actually fairly annoying for performance
work.
:(

Does anyone have any testcases where it is actually slow that i can look at?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170531/da674683/attachment.html>

+Jan, the original contributor (from Chromium), in case he's got any
context that might be useful.

On Wed, May 31, 2017 at 7:39 AM Daniel Berlin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Right now, the LLVM Statistic class does not increment values if we are in
> non-debug mode, unless you define LLVM_ENABLE_STATS (which, sadly, is also
> not available from cmake).
>
> Before i go and add it to cmake, i ran numbers.
>
> I can't find a single testcase, large or small, where enabling
statistics
> all the time isn't completely noise.
>
> It looks like it was disabled in march  2013.
> https://reviews.llvm.org/D486
>
> The only discussion i can find in the same time period is around fastisel
> using a lot of per-instruction stats, and it causing some slow-down.
>
> However, I can't find any actual data/testcases in this discussion at
all,
> and disabling stats globally is actually fairly annoying for performance
> work.
> :(
>
> Does anyone have any testcases where it is actually slow that i can look
> at?
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170531/1e29647c/attachment.html>

I think adding an option to cmake is fine, but it should definitely be off
by default in release builds. If we make STATISTIC useful, then people will
eventually put it on some hot path, and that will kill multi-threaded
performance for users like ThinLTO, because multiple threads doing
read-modify-write on the same memory is just slow.

On Wed, May 31, 2017 at 7:39 AM, Daniel Berlin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Right now, the LLVM Statistic class does not increment values if we are in
> non-debug mode, unless you define LLVM_ENABLE_STATS (which, sadly, is also
> not available from cmake).
>
> Before i go and add it to cmake, i ran numbers.
>
> I can't find a single testcase, large or small, where enabling
statistics
> all the time isn't completely noise.
>
> It looks like it was disabled in march  2013.
> https://reviews.llvm.org/D486
>
> The only discussion i can find in the same time period is around fastisel
> using a lot of per-instruction stats, and it causing some slow-down.
>
> However, I can't find any actual data/testcases in this discussion at
all,
> and disabling stats globally is actually fairly annoying for performance
> work.
> :(
>
> Does anyone have any testcases where it is actually slow that i can look
> at?
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170531/7ad5d9f1/attachment.html>

On Wed, May 31, 2017 at 7:48 AM, Reid Kleckner <rnk at google.com> wrote:
> I think adding an option to cmake is fine, but it should definitely be off
> by default in release builds.
>
Why?

> If we make STATISTIC useful, then people will eventually put it on some
> hot path, and that will kill multi-threaded performance for users like
> ThinLTO, because multiple threads doing read-modify-write on the same
> memory is just slow.
>
With no offense meant: Can we please be driven by data?
This argument is "people will eventually do something silly and it will
make things slow for others". That is something that can be said about just
about anything in LLVM :).  This is why we have code review, etc.
If your concern is the atomic increments, then is there a reason to not
have it do the counting as a runtime option instead of a compile time one?
>
>
> On Wed, May 31, 2017 at 7:39 AM, Daniel Berlin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Right now, the LLVM Statistic class does not increment values if we are
>> in non-debug mode, unless you define LLVM_ENABLE_STATS (which, sadly,
is
>> also not available from cmake).
>>
>> Before i go and add it to cmake, i ran numbers.
>>
>> I can't find a single testcase, large or small, where enabling
statistics
>> all the time isn't completely noise.
>>
>> It looks like it was disabled in march  2013.
>> https://reviews.llvm.org/D486
>>
>> The only discussion i can find in the same time period is around
fastisel
>> using a lot of per-instruction stats, and it causing some slow-down.
>>
>> However, I can't find any actual data/testcases in this discussion
at
>> all, and disabling stats globally is actually fairly annoying for
>> performance work.
>> :(
>>
>> Does anyone have any testcases where it is actually slow that i can
look
>> at?
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170531/0684ce1a/attachment-0001.html>

Last time I measured 0.5%-1% slowdown with statistics enabled in a release
build. Also see my last thread about this:
https://groups.google.com/forum/#!topic/llvm-dev/xZVBNg5bsSk
<https://groups.google.com/forum/#!topic/llvm-dev/xZVBNg5bsSk>
We also had plans to push statistics (probably newly introduced one rather than
retrofitting the existing ones) through the optimization remark system to allow
some context like per-function statistics. But there is nothing concrete yet.

- Matthias
> On May 31, 2017, at 7:39 AM, Daniel Berlin via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Right now, the LLVM Statistic class does not increment values if we are in
non-debug mode, unless you define LLVM_ENABLE_STATS (which, sadly, is also not
available from cmake).
> 
> Before i go and add it to cmake, i ran numbers.
> 
> I can't find a single testcase, large or small, where enabling
statistics all the time isn't completely noise.
> 
> It looks like it was disabled in march  2013.
> https://reviews.llvm.org/D486 <https://reviews.llvm.org/D486>
> 
> The only discussion i can find in the same time period is around fastisel
using a lot of per-instruction stats, and it causing some slow-down.
> 
> However, I can't find any actual data/testcases in this discussion at
all, and disabling stats globally is actually fairly annoying for performance
work.
> :(
> 
> Does anyone have any testcases where it is actually slow that i can look
at?
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170531/4fbd9593/attachment.html>

> On May 31, 2017, at 10:57 AM, Matthias Braun via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Last time I measured 0.5%-1% slowdown with statistics enabled in a release
build. Also see my last thread about this:
https://groups.google.com/forum/#!topic/llvm-dev/xZVBNg5bsSk
<https://groups.google.com/forum/#!topic/llvm-dev/xZVBNg5bsSk>
> We also had plans to push statistics (probably newly introduced one rather
than retrofitting the existing ones) through the optimization remark system to
allow some context like per-function statistics. But there is nothing concrete
yet.
And just to elaborate, this originated form the observation that there are many
cases where we issue a debug message, emit an optimization remark and also
increment a statistic within a few lines of code.  Having a single way of
feeding all these use cases would be nice code-readability and usability
improvement.

Adam
> 
> - Matthias
> 
>> On May 31, 2017, at 7:39 AM, Daniel Berlin via llvm-dev <llvm-dev at
lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>> 
>> Right now, the LLVM Statistic class does not increment values if we are
in non-debug mode, unless you define LLVM_ENABLE_STATS (which, sadly, is also
not available from cmake).
>> 
>> Before i go and add it to cmake, i ran numbers.
>> 
>> I can't find a single testcase, large or small, where enabling
statistics all the time isn't completely noise.
>> 
>> It looks like it was disabled in march  2013.
>> https://reviews.llvm.org/D486 <https://reviews.llvm.org/D486>
>> 
>> The only discussion i can find in the same time period is around
fastisel using a lot of per-instruction stats, and it causing some slow-down.
>> 
>> However, I can't find any actual data/testcases in this discussion
at all, and disabling stats globally is actually fairly annoying for performance
work.
>> :(
>> 
>> Does anyone have any testcases where it is actually slow that i can
look at?
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170531/fdeaceda/attachment.html>