llvm dev - Nov 2015 - [RFC] A new intrinsic, `llvm.blackbox`, to explicitly prevent constprop, die, etc optimizations

On Fri, 06 Nov 2015 09:27:32 -0800, Daniel Berlin via llvm-dev wrote:

<snip>
> 
> Great!
> 
> You should then test that this happens, and additionally write a test
> that can't be optimized away, since the above is apparently not a
useful
> microbenchmark for anything but the compiler ;-)
> 
> Seriously though, there are basically three cases (with a bit of
> handwaving)
> 
> 1. You want to test that the compiler optimizes something in a certain
> way.  The above example, without anything else, you actually want to
> test that the compiler optimizes this away completely.
> This doesn't require anything except using something like FIleCheck and
> producing IR at the end of Rust's optimization pipeline.
> 
> 2. You want to make the above code into a benchmark, and ensure the
> compiler is required to keep the number and relative order of certain
> operations.
> Use volatile for this.
> 
> Volatile is not what you seem to think it is, or may think about it in
> terms of what people use it for in C/C++.
> volatile in llvm has a well defined meaning:
> http://llvm.org/docs/LangRef.html#volatile-memory-accesses
> 
> 3. You want to get the compiler to only do certain optimizations to your
> code.
> 
> Yes, you have to either write a test harness (even if that test harness
> is "your normal compiler, with certain flags passed"), or use
ours, for
> that ;-)
> 
> 
> It seems like you want #2, so you should use volatile.
> 
> But don't conflate #2 and #3.
> 
> As said:
> If you want the compiler to only do certain things to your code, you
> should tell it to only do those things by giving it a pass pipeline that
> only does those things.  Nothing else is going to solve this problem
> well.
> 
> If you want the compiler to do every optimization it knows to your code,
> but want it to maintain the number and relative order of certain
> operations, that's volatile.
I think the fundamental thing you're missing is that benchmarks are an 
exercise in if/then:

*If* a user exercises this API, *then* how well would it perform?

Of course, in the case of a user, the data could come from anywhere, and 
go anywhere - the terminal, a network socket, whatever.

However, in a benchmark, all the data comes from (and goes) to places the 
compiler and see.

Thus, it's necessary to make the compiler _pretend_ the data came from 
and goes to a "black box", in order for the benchmarks to even
*remotely*
resemble what they're meant to test.

This is actually distinct from #1, #2, _and_ #3 above - quite simply, 
what is needed is a way to simulate a "real usage" scenario without 
actually contacting the external world.

>
>
> I think the fundamental thing you're missing is that benchmarks are an
> exercise in if/then:
>
I don't believe i'm actually missing anything.

>
> *If* a user exercises this API, *then* how well would it perform?

In this case, he exercised the API, and the compiler optimized it away.
If he wants to test whether the API, exercised in some other way, will be
optimized, he should test that.

Your argument that this can't be tested is *almost always* false.  It's
true sometimes, but that's *actually pretty rare*.

> Of course, in the case of a user, the data could come from anywhere, and
> go anywhere - the terminal, a network socket, whatever.
>
> However, in a benchmark, all the data comes from (and goes) to places the
> compiler and see.
>
This is not necessarily true, but as I said, the way around this is to use
volatile.

>
> Thus, it's necessary to make the compiler _pretend_ the data came from
> and goes to a "black box", in order for the benchmarks to even
*remotely*
> resemble what they're meant to test.
>
> This is actually distinct from #1, #2, _and_ #3 above - quite simply,
> what is needed is a way to simulate a "real usage" scenario
without
> actually contacting the external world.
>
>
No. It is not distinct in any way, and you haven't shown why it is.
It is exactly  "i want this operation to happen exactly X times despite
whether the compiler thinks it is necessary or can be removed"

That is volatile.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151109/016f3924/attachment.html>

One thing that volatile doesn't do is escape results that have been written
to memory.

The proposed blackbox intrinsic is modeled as reading and writing any
pointed to memory, which is useful.

I also think blackbox will be a lot easier for people to use than empty
volatile inline asm and volatile loads and stores. That alone seems worth
something. :)

On Mon, Nov 9, 2015 at 6:04 PM, Daniel Berlin via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
>> I think the fundamental thing you're missing is that benchmarks are
an
>> exercise in if/then:
>>
>
> I don't believe i'm actually missing anything.
>
>
>>
>> *If* a user exercises this API, *then* how well would it perform?
>
>
> In this case, he exercised the API, and the compiler optimized it away.
> If he wants to test whether the API, exercised in some other way, will be
> optimized, he should test that.
>
> Your argument that this can't be tested is *almost always* false. 
It's
> true sometimes, but that's *actually pretty rare*.
>
>
>> Of course, in the case of a user, the data could come from anywhere,
and
>> go anywhere - the terminal, a network socket, whatever.
>>
>> However, in a benchmark, all the data comes from (and goes) to places
the
>> compiler and see.
>>
>
> This is not necessarily true, but as I said, the way around this is to use
> volatile.
>
>
>>
>> Thus, it's necessary to make the compiler _pretend_ the data came
from
>> and goes to a "black box", in order for the benchmarks to
even *remotely*
>> resemble what they're meant to test.
>>
>
>
>> This is actually distinct from #1, #2, _and_ #3 above - quite simply,
>> what is needed is a way to simulate a "real usage" scenario
without
>> actually contacting the external world.
>>
>>
> No. It is not distinct in any way, and you haven't shown why it is.
> It is exactly  "i want this operation to happen exactly X times
despite
> whether the compiler thinks it is necessary or can be removed"
>
> That is volatile.
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20151109/62e1a0f5/attachment.html>