llvm dev - May 2020 - Sanitizers + New Pass Manager

Just tested it out, that test does indeed fail under the old PM at -O3 and
even at -O2.

If the ASan pass runs after optimizations and is designed to detect
undefined behavior at runtime, I don't see how it can be super reliable at
higher optimization levels.

On Wed, May 13, 2020 at 12:39 PM David Blaikie <dblaikie at gmail.com>
wrote:
> +some sanitizer/new pass manager folks
>
>
>
> On Wed, May 13, 2020 at 12:22 PM Arthur Eubanks via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> I've been trying to burn down the remaining sanitizer failures
under the
>> new pass manager. Right now I'm stuck on some ASan tests.
>>
>> Some ASan tests run under -O1. There are a couple differences between
the
>> old and new pass managers under -O1, e.g. the old PM doesn't inline
whereas
>> the new PM does. The differences seem to cause some lifetime intrinsics
to
>> get stripped out (e.g. via SROA, InstCombine). It might be due to ASan
>> specifically testing undefined behavior, and different optimizations
run
>> means different behavior. For a specific example,
>> use-after-scope-dtor-order.cpp runs under -O1 and fails under the new
PM
>> because SROA strips out the lifetime intrinsics and by the time the
ASan
>> pass runs it doesn't find the lifetime intrinsics to add its own
>> instrumentation.
>>
>
> That, to me, sounds like a real bug in the optimizations/asan
> implementation if this choice fo optimizations makes the diagnosis go away.
> (is ASan able to diagnose the problem at -O3 (where I guess SROA and other
> things run) with the legacy pass manager?)
>
>
>>
>> What's the proper way to resolve this? Run the tests under -O0?
Change
>> the passes pipeline under the new PM when ASan (and maybe other
sanitizers)
>> is detected?
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200513/f65f9eb0/attachment.html>

I believe it's meant to run after /some/ optimizations to make it a bit
more efficient, while not so optimized that it misses opportunities to
detect bugs - but I could be wrong there. I'll leave it up to other folks
to chime in.

On Wed, May 13, 2020 at 1:04 PM Arthur Eubanks <aeubanks at google.com>
wrote:
> Just tested it out, that test does indeed fail under the old PM at -O3 and
> even at -O2.
>
> If the ASan pass runs after optimizations and is designed to detect
> undefined behavior at runtime, I don't see how it can be super reliable
at
> higher optimization levels.
>
> On Wed, May 13, 2020 at 12:39 PM David Blaikie <dblaikie at
gmail.com> wrote:
>
>> +some sanitizer/new pass manager folks
>>
>>
>>
>> On Wed, May 13, 2020 at 12:22 PM Arthur Eubanks via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> I've been trying to burn down the remaining sanitizer failures
under the
>>> new pass manager. Right now I'm stuck on some ASan tests.
>>>
>>> Some ASan tests run under -O1. There are a couple differences
between
>>> the old and new pass managers under -O1, e.g. the old PM
doesn't inline
>>> whereas the new PM does. The differences seem to cause some
lifetime
>>> intrinsics to get stripped out (e.g. via SROA, InstCombine). It
might be
>>> due to ASan specifically testing undefined behavior, and different
>>> optimizations run means different behavior. For a specific example,
>>> use-after-scope-dtor-order.cpp runs under -O1 and fails under the
new PM
>>> because SROA strips out the lifetime intrinsics and by the time the
ASan
>>> pass runs it doesn't find the lifetime intrinsics to add its
own
>>> instrumentation.
>>>
>>
>> That, to me, sounds like a real bug in the optimizations/asan
>> implementation if this choice fo optimizations makes the diagnosis go
away.
>> (is ASan able to diagnose the problem at -O3 (where I guess SROA and
other
>> things run) with the legacy pass manager?)
>>
>>
>>>
>>> What's the proper way to resolve this? Run the tests under -O0?
Change
>>> the passes pipeline under the new PM when ASan (and maybe other
sanitizers)
>>> is detected?
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200513/94a72dc3/attachment-0001.html>

Is it the case that with the legacy PM there is no inlining at either -O2
or -O3 and with newPM there is? Or is there something else going on?

Sounds like this needs pinpointing where the behavior should be different
than what's currently happening.
Inlining in the NPM doesn't seem the issue, it's what it triggers
afterwards. Is it that the lifetime intrinsics shouldn't have been
stripped, or is it that ASan shouldn't rely on these to add instrumentation
because previous optimizations may have removed them (hence it should not
rely on what optimizations are done or not done; as you suggested, maybe
changing the pass pipeline to add ASan earlier may be needed?), or is it
something else?


Alina

On Wed, May 13, 2020 at 1:04 PM Arthur Eubanks <aeubanks at google.com>
wrote:
> Just tested it out, that test does indeed fail under the old PM at -O3 and
> even at -O2.
>
> If the ASan pass runs after optimizations and is designed to detect
> undefined behavior at runtime, I don't see how it can be super reliable
at
> higher optimization levels.
>
> On Wed, May 13, 2020 at 12:39 PM David Blaikie <dblaikie at
gmail.com> wrote:
>
>> +some sanitizer/new pass manager folks
>>
>>
>>
>> On Wed, May 13, 2020 at 12:22 PM Arthur Eubanks via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> I've been trying to burn down the remaining sanitizer failures
under the
>>> new pass manager. Right now I'm stuck on some ASan tests.
>>>
>>> Some ASan tests run under -O1. There are a couple differences
between
>>> the old and new pass managers under -O1, e.g. the old PM
doesn't inline
>>> whereas the new PM does. The differences seem to cause some
lifetime
>>> intrinsics to get stripped out (e.g. via SROA, InstCombine). It
might be
>>> due to ASan specifically testing undefined behavior, and different
>>> optimizations run means different behavior. For a specific example,
>>> use-after-scope-dtor-order.cpp runs under -O1 and fails under the
new PM
>>> because SROA strips out the lifetime intrinsics and by the time the
ASan
>>> pass runs it doesn't find the lifetime intrinsics to add its
own
>>> instrumentation.
>>>
>>
>> That, to me, sounds like a real bug in the optimizations/asan
>> implementation if this choice fo optimizations makes the diagnosis go
away.
>> (is ASan able to diagnose the problem at -O3 (where I guess SROA and
other
>> things run) with the legacy pass manager?)
>>
>>
>>>
>>> What's the proper way to resolve this? Run the tests under -O0?
Change
>>> the passes pipeline under the new PM when ASan (and maybe other
sanitizers)
>>> is detected?
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200513/5fb65b88/attachment.html>

On 5/13/20 3:31 PM, David Blaikie via llvm-dev wrote:
> I believe it's meant to run after /some/ optimizations to make it a bit
> more efficient, while not so optimized that it misses opportunities to
> detect bugs - but I could be wrong there. I'll leave it up to other
folks
> to chime in.
I think that is right. The more transformations you run the more UB you 
can also "loose" as it is defined to something by the transformation.

Lifetime markers are an example. Once removed, which is generally legal 
in IR, you cannot argue accesses after the end are UB.

> On Wed, May 13, 2020 at 1:04 PM Arthur Eubanks <aeubanks at
google.com> wrote:
>
>> Just tested it out, that test does indeed fail under the old PM at -O3
and
>> even at -O2.
>>
>> If the ASan pass runs after optimizations and is designed to detect
>> undefined behavior at runtime, I don't see how it can be super
reliable at
>> higher optimization levels.
>>
>> On Wed, May 13, 2020 at 12:39 PM David Blaikie <dblaikie at
gmail.com> wrote:
>>
>>> +some sanitizer/new pass manager folks
>>>
>>>
>>>
>>> On Wed, May 13, 2020 at 12:22 PM Arthur Eubanks via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> I've been trying to burn down the remaining sanitizer
failures under the
>>>> new pass manager. Right now I'm stuck on some ASan tests.
>>>>
>>>> Some ASan tests run under -O1. There are a couple differences
between
>>>> the old and new pass managers under -O1, e.g. the old PM
doesn't inline
>>>> whereas the new PM does. The differences seem to cause some
lifetime
>>>> intrinsics to get stripped out (e.g. via SROA, InstCombine). It
might be
>>>> due to ASan specifically testing undefined behavior, and
different
>>>> optimizations run means different behavior. For a specific
example,
>>>> use-after-scope-dtor-order.cpp runs under -O1 and fails under
the new PM
>>>> because SROA strips out the lifetime intrinsics and by the time
the ASan
>>>> pass runs it doesn't find the lifetime intrinsics to add
its own
>>>> instrumentation.
>>>>
>>> That, to me, sounds like a real bug in the optimizations/asan
>>> implementation if this choice fo optimizations makes the diagnosis
go away.
>>> (is ASan able to diagnose the problem at -O3 (where I guess SROA
and other
>>> things run) with the legacy pass manager?)
>>>
>>>
>>>> What's the proper way to resolve this? Run the tests under
-O0? Change
>>>> the passes pipeline under the new PM when ASan (and maybe other
sanitizers)
>>>> is detected?
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200513/c0de20e3/attachment.html>