llvm dev - Feb 2015 - [LLVMdev] [cfe-dev] [3.6 Release] RC3 has been tagged

I finally got around to testing this on a Bloomfield processor (Early
2009 MacPro 2x2.66 GHz dual-quad core) and the regressions from
http://llvm.org/bugs/show_bug.cgi?id=22589 are even more severe. For
10 runs of scimark2_1c built with "-O3 -march=native"...

llvm 3.5.1   1204.16+/-2.66 Mflops
3.6 branch   866.49+/-1.26 Mflops

Do you seriously want to ship with a 39% performance regression in a
major benchmark?
              Jack

On Wed, Feb 18, 2015 at 10:33 AM, Renato Golin <renato.golin at
linaro.org> wrote:
> On 18 February 2015 at 14:37, Hal Finkel <hfinkel at anl.gov> wrote:
>> I think that, unfortunately, for the others, there's not sufficient
time to investigate before the release.
>
> This looks like a serious case for 3.6.1, not RC5.
>
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On Wed, Feb 18, 2015 at 10:55 AM, Jack Howarth
<howarth.mailing.lists at gmail.com> wrote:
> I finally got around to testing this on a Bloomfield processor (Early
> 2009 MacPro 2x2.66 GHz dual-quad core) and the regressions from
> http://llvm.org/bugs/show_bug.cgi?id=22589 are even more severe. For
> 10 runs of scimark2_1c built with "-O3 -march=native"...
>
> llvm 3.5.1   1204.16+/-2.66 Mflops
> 3.6 branch   866.49+/-1.26 Mflops
>
The proposed patch mitigates the damage on Bloomfield...

patched 3.6svn   1073.69+/-1.97

so we are only regressing the benchmark 24% rather than 39% there.
         Jack

> Do you seriously want to ship with a 39% performance regression in a
> major benchmark?
>               Jack
>
> On Wed, Feb 18, 2015 at 10:33 AM, Renato Golin <renato.golin at
linaro.org> wrote:
>> On 18 February 2015 at 14:37, Hal Finkel <hfinkel at anl.gov>
wrote:
>>> I think that, unfortunately, for the others, there's not
sufficient time to investigate before the release.
>>
>> This looks like a serious case for 3.6.1, not RC5.
>>
>> cheers,
>> --renato
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On Wed, Feb 18, 2015 at 11:22 AM, Jack Howarth
<howarth.mailing.lists at gmail.com> wrote:
> On Wed, Feb 18, 2015 at 10:55 AM, Jack Howarth
> <howarth.mailing.lists at gmail.com> wrote:
>> I finally got around to testing this on a Bloomfield processor (Early
>> 2009 MacPro 2x2.66 GHz dual-quad core) and the regressions from
>> http://llvm.org/bugs/show_bug.cgi?id=22589 are even more severe. For
>> 10 runs of scimark2_1c built with "-O3 -march=native"...
>>
>> llvm 3.5.1   1204.16+/-2.66 Mflops
>> 3.6 branch   866.49+/-1.26 Mflops
>>
>
> The proposed patch mitigates the damage on Bloomfield...
>
> patched 3.6svn   1073.69+/-1.97
>
> so we are only regressing the benchmark 24% rather than 39% there.
>          Jack
>
This still all begs the question of what exact metrics exist for the
Q/A of llvm releases? IMHO, the bad PR from shoving out compiler
releases with severe performance regressions in the generated code far
outweighs a brief delay to triage these issues as much as possible.
         Jack
>
>> Do you seriously want to ship with a 39% performance regression in a
>> major benchmark?
>>               Jack
>>
>> On Wed, Feb 18, 2015 at 10:33 AM, Renato Golin <renato.golin at
linaro.org> wrote:
>>> On 18 February 2015 at 14:37, Hal Finkel <hfinkel at anl.gov>
wrote:
>>>> I think that, unfortunately, for the others, there's not
sufficient time to investigate before the release.
>>>
>>> This looks like a serious case for 3.6.1, not RC5.
>>>
>>> cheers,
>>> --renato
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev