llvm dev - Feb 2017 - Compilation doesn't finish when building with clang 4.0.0 (was: [cfe-dev] [4.0.0 Release] Release Candidate 2 source and binaries available)

I could bisect it down to this commit [1].

Looking at the code I'm compiling, it makes sense. There are lots of scopes
and in each scope an (inlined) function is called, which has an assert. The
assert is implemented by throwing an exception. So lots of unwind information is
generated. The change seems to do "more" with this unwind info than
previously.

I guess the question then becomes, is this an endless loop or just some
exponential increase in compilation time. I've already reduced the file
quite a bit and will try to post a reproducer.

Best regards,
Martin

[1]
commit ecefac9053b75242fffb5f71d0714200a619f50c
Author: Vitaly Buka <vitalybuka at google.com>
Date:   Fri Jul 22 22:04:38 2016 +0000

    Unpoison stack before resume instruction

    Summary:
    Clang inserts cleanup code before resume similar way as before return
instruction.
    This makes asan poison local variables causing false use-after-scope
reports.

    __asan_handle_no_return does not help here as it was executed before
    llvm.lifetime.end inserted into resume block.

    To avoid false report we need to unpoison stack for resume same way as for
return.

    PR27453

    Reviewers: kcc, eugenis

    Differential Revision: https://reviews.llvm.org/D22661

    git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk at 276480
91177308-0d34-0410-b5e6-96231b3b80d8



-----Original Message-----
From: Richtarsky, Martin 
Sent: Mittwoch, 15. Februar 2017 22:11
To: 'Hans Wennborg' <hans at chromium.org>; Nemanja Ivanovic
<nemanja.i.ibm at gmail.com>
Cc: Renato Golin <renato.golin at linaro.org>; llvm-dev <llvm-dev at
lists.llvm.org>; Release-testers <release-testers at lists.llvm.org>
Subject: Compilation doesn't finish when building with clang 4.0.0 (was:
[llvm-dev] [cfe-dev] [4.0.0 Release] Release Candidate 2 source and binaries
available)

I forgot to mention, this is a build with -fsanitize=address. Removing this (or
using -O0) makes it work.

Here are some timings for things that work and the max RSS at the end:
clang 3.9, -O1, -fsanitize=address: 3m:14.53, 922MB
clang 4.0, -O1, -fno-sanitize=address: 3m:17.31, 697 MB
clang 4.0, -O0, -fsanitize=address: 3m:22.78, 17.9 GB

I'm bisecting now and will also try my luck with creduce later.

Best regards,
Martin
>It sounds like PR31890 was introduced with r294186, which is well
>after the 4.0 branch (r291814), so that's not what's happening here.
>
>Martin, how long does your file usually take to build? Would it be
>possible to use creduce to get to a test case that you can upload to
>the bug tracker?
On Wed, Feb 15, 2017 at 9:50 AM, Nemanja Ivanovic via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> This may be a shot in the dark (I don't even know whether your revision
> includes the culprit revision), but the symptoms seem similar to:
> https://bugs.llvm.org//show_bug.cgi?id=31890
>
> On Wed, Feb 15, 2017 at 12:43 PM, Renato Golin via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>
>> On 15 February 2017 at 10:24, Richtarsky, Martin via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>> > I have encountered very long compile times for three large source
files
>> > containing generated/unrolled code at -O1.
>> > We are talking about 10+ hours here without completing, so it
looks very
>> > much like an endless loop.
>> > The processes are using 15, 22 and 27 GB of memory but do not
appear to
>> > grow further.
>> > This worked fine in the past, so appears to be a regression.
>>
>> Ouch!
>>
>>
>> > Are there any new optimization passes I could try switching off?
>> > I could not find any mention in the release notes of new passes.
>>
>> It doesn't seem to be any new passes, but the scheduler. Did this
work
>> in RC1? If so, it'll be a lot easier to identify the cause. If not,
a
>> bisection might help.
>>
>> cheers,
>> --renato
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

On 16 February 2017 at 14:37, Richtarsky, Martin
<martin.richtarsky at sap.com> wrote:
> I could bisect it down to this commit [1].
>
> Looking at the code I'm compiling, it makes sense. There are lots of
scopes and in each scope an (inlined) function is called, which has an assert.
The assert is implemented by throwing an exception. So lots of unwind
information is generated. The change seems to do "more" with this
unwind info than previously.
>
> I guess the question then becomes, is this an endless loop or just some
exponential increase in compilation time. I've already reduced the file
quite a bit and will try to post a reproducer.
Hi Martin,

Thanks, this is great investigative work!

Having a small reproducer creating 22GB objects in memory is a great
way to say it's broken. :)

However, the fact that this hasn't shown in other cases probably means
it's just exponential and your program is huge. So your reduced output
will still very likely be large as well.

To me, that commit doesn't seem like it's doing much, so it probably
have exposed a problem in the actual instrumentation machinery (for
example, not cleaning the vector properly after handling those
instructions).

We should handle this issue in bugzilla, though. When you create the
bug, please make sure you copy Hans, Vitaly and the people copied on
that review (kcc, eugenis).

cheers,
--renato

Hi Martin,

Did you have any progress with creduce?

Thanks,
Hans

On Thu, Feb 16, 2017 at 6:37 AM, Richtarsky, Martin
<martin.richtarsky at sap.com> wrote:
> I could bisect it down to this commit [1].
>
> Looking at the code I'm compiling, it makes sense. There are lots of
scopes and in each scope an (inlined) function is called, which has an assert.
The assert is implemented by throwing an exception. So lots of unwind
information is generated. The change seems to do "more" with this
unwind info than previously.
>
> I guess the question then becomes, is this an endless loop or just some
exponential increase in compilation time. I've already reduced the file
quite a bit and will try to post a reproducer.
>
> Best regards,
> Martin
>
> [1]
> commit ecefac9053b75242fffb5f71d0714200a619f50c
> Author: Vitaly Buka <vitalybuka at google.com>
> Date:   Fri Jul 22 22:04:38 2016 +0000
>
>     Unpoison stack before resume instruction
>
>     Summary:
>     Clang inserts cleanup code before resume similar way as before return
instruction.
>     This makes asan poison local variables causing false use-after-scope
reports.
>
>     __asan_handle_no_return does not help here as it was executed before
>     llvm.lifetime.end inserted into resume block.
>
>     To avoid false report we need to unpoison stack for resume same way as
for return.
>
>     PR27453
>
>     Reviewers: kcc, eugenis
>
>     Differential Revision: https://reviews.llvm.org/D22661
>
>     git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk at 276480
91177308-0d34-0410-b5e6-96231b3b80d8
>
>
>
> -----Original Message-----
> From: Richtarsky, Martin
> Sent: Mittwoch, 15. Februar 2017 22:11
> To: 'Hans Wennborg' <hans at chromium.org>; Nemanja Ivanovic
<nemanja.i.ibm at gmail.com>
> Cc: Renato Golin <renato.golin at linaro.org>; llvm-dev <llvm-dev
at lists.llvm.org>; Release-testers <release-testers at lists.llvm.org>
> Subject: Compilation doesn't finish when building with clang 4.0.0
(was: [llvm-dev] [cfe-dev] [4.0.0 Release] Release Candidate 2 source and
binaries available)
>
> I forgot to mention, this is a build with -fsanitize=address. Removing this
(or using -O0) makes it work.
>
> Here are some timings for things that work and the max RSS at the end:
> clang 3.9, -O1, -fsanitize=address: 3m:14.53, 922MB
> clang 4.0, -O1, -fno-sanitize=address: 3m:17.31, 697 MB
> clang 4.0, -O0, -fsanitize=address: 3m:22.78, 17.9 GB
>
> I'm bisecting now and will also try my luck with creduce later.
>
> Best regards,
> Martin
>
>>It sounds like PR31890 was introduced with r294186, which is well
>>after the 4.0 branch (r291814), so that's not what's happening
here.
>>
>>Martin, how long does your file usually take to build? Would it be
>>possible to use creduce to get to a test case that you can upload to
>>the bug tracker?
>
> On Wed, Feb 15, 2017 at 9:50 AM, Nemanja Ivanovic via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>> This may be a shot in the dark (I don't even know whether your
revision
>> includes the culprit revision), but the symptoms seem similar to:
>> https://bugs.llvm.org//show_bug.cgi?id=31890
>>
>> On Wed, Feb 15, 2017 at 12:43 PM, Renato Golin via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>>
>>> On 15 February 2017 at 10:24, Richtarsky, Martin via llvm-dev
>>> <llvm-dev at lists.llvm.org> wrote:
>>> > I have encountered very long compile times for three large
source files
>>> > containing generated/unrolled code at -O1.
>>> > We are talking about 10+ hours here without completing, so it
looks very
>>> > much like an endless loop.
>>> > The processes are using 15, 22 and 27 GB of memory but do not
appear to
>>> > grow further.
>>> > This worked fine in the past, so appears to be a regression.
>>>
>>> Ouch!
>>>
>>>
>>> > Are there any new optimization passes I could try switching
off?
>>> > I could not find any mention in the release notes of new
passes.
>>>
>>> It doesn't seem to be any new passes, but the scheduler. Did
this work
>>> in RC1? If so, it'll be a lot easier to identify the cause. If
not, a
>>> bisection might help.
>>>
>>> cheers,
>>> --renato
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>