llvm dev - Apr 2010 - [LLVMdev] [cfe-dev] 2.7 Pre-release1 available for testing

On 03/30/2010 09:15 PM, Tanya Lattner wrote:
> 
> Thanks for testing the release!
> 
>> Tests were run on x86-64, Debian unstable, Linux 2.6.33, gcc 4.4.3,
>> 64-bit. I built srcdir == objdir, I have built llvm and clang myself,
>> and used the binaries for llvm-gcc.
>>
>> 1. llvm-gcc 2.7 vs 2.6
>> compared to my results from Aug 31 2009, ignoring CBE failures:
>>
>> new JIT failures:
>> MultiSource/Applications/spiff/spiff
>> SingleSource/Regression/C/2004-03-15-IndirectGoto
>>
> 
> Yes, I'm seeing the second regression on darwin too. Please file a bug
for the other one if you havent already.
I haven't, will do tomorrow.
> 
>> 2. llvm-gcc 2.7 vs clang 2.7
>> When comparing the 2.7 llvm-gcc and clang results I see these
>> differences (is llvm-gcc considered baseline for clang?):
>> ALL FAIL (pass in llvm-gcc):
>> MultiSource/Benchmarks/PAQ8p/paq8p
>> MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4
>> MultiSource/Benchmarks/Prolangs-C/archie-client/archie
>> MultiSource/Benchmarks/Prolangs-C/cdecl/cdecl
>> SingleSource/Benchmarks/Misc-C++/bigfib
>> SingleSource/Regression/C++/EH/ConditionalExpr
>> SingleSource/Regression/C++/EH/ctor_dtor_count-2
>> SingleSource/Regression/C++/EH/function_try_block
>> SingleSource/Regression/C++/EH/simple_throw
>> SingleSource/UnitTests/2006-12-04-DynAllocAndRestore
>> SingleSource/UnitTests/Vector/SSE/sse.expandfft
>> SingleSource/UnitTests/Vector/SSE/sse.stepfft
>>
>> JIT failures in clang, pass in llvm-gcc:
>> MultiSource/Applications/sqlite3/sqlite3
>> SingleSource/Regression/C++/ofstream_ctor
>>
> 
> This isn't part of our release criteria. So, these are not release
blockers.
Ok, something to keep in mind for LLVM 2.8 then.
> 
>> 3. Some performance regressions GCC/LLC  (2.6 -> 2.7), but keep in
mind
>> that I wasn't using GCC 4.4.3 as comparison for llvm 2.6!
>>
>> MultiSource/Applications/hexxagon/hexxagon  1.22 -> 1.14
>> MultiSource/Applications/lua/lua  0.91 -> 0.84
>> MultiSource/Applications/obsequi/Obsequi  0.93 -> 0.86
>> MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk  1.01 -> 0.91
>> MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow     0.94 ->
0.75
>> MultiSource/Benchmarks/FreeBench/neural/neural   1.0 -> 0.9
>> MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm   1.06 ->
0.9
>> MultiSource/Benchmarks/Olden/treeadd/treeadd  11.44 -> 9.89
>> MultiSource/Benchmarks/Olden/tsp/tsp  1.14 -> 1.02
>> MultiSource/Benchmarks/Ptrdist/anagram/anagram 1.33 -> 1.23
>> SingleSource/Benchmarks/Dhrystone/dry  7.32 -> 5.16
>> SingleSource/Benchmarks/Dhrystone/fldry   8.02 -> 6.65
>> ....
>>
> 
> Unfortunately, we just don't have enough man power to have performance
be a release criteria at this time. We also need a better infrastructure in
place to track this stuff (Daniel is working on it).
Yes, I understand that.
> 
>> I'll have to write a script to compare the results, its boring and
>> inaccurate to do by hand.
>>
>> Will go through the bugzilla tomorrow
I still didn't have time to do this unfortunately.
>> and see if I need to open new bugs
>> for this stuff.
>>
>>>  /To test clang:/ 
>>>
>>> 1) Compile llvm and clang from source. 
>>>
>>> 2) Run make check for llvm. 
>>>
>>> 3) Run make  -C tools/clang-2.6 test VERBOSE=1 (report any failures
or
>>> unexpected passes) 
>> Surely you meant tools/clang-2.7
>>
> 
> Yes.
> 
>> FYI I pulled the following revisions for ClamAV's llvm on top of
2.7:
>> r98349
>> r98410
>> r98447
>> r98508
>> r99143
>> r99146
>> r99147
>> r99160
>> r99400
>>
>> I don't know if any of these qualify as regression fixes for 2.7,
I'll
>> leave it up to you to decide if you want to put them into 2.7 or not.
>>
> 
> I'll have to discuss with Chris about these. Its technically not a
release blocker.
Meanwhile I pulled these too:
99762 (Evan approved)
99883 (leakfix, so I don't think it qualifies for release criteria)

Best regards,
--Edwin

On 03/30/2010 09:21 PM, Török Edwin wrote:
> On 03/30/2010 09:15 PM, Tanya Lattner wrote:
>> Thanks for testing the release!
>>
>>> Tests were run on x86-64, Debian unstable, Linux 2.6.33, gcc 4.4.3,
>>> 64-bit. I built srcdir == objdir, I have built llvm and clang
myself,
>>> and used the binaries for llvm-gcc.
>>>
>>> 1. llvm-gcc 2.7 vs 2.6
>>> compared to my results from Aug 31 2009, ignoring CBE failures:
>>>
>>> new JIT failures:
>>> MultiSource/Applications/spiff/spiff
>>> SingleSource/Regression/C/2004-03-15-IndirectGoto
>>>
>> Yes, I'm seeing the second regression on darwin too. Please file a
bug for the other one if you havent already.
> 
> I haven't, will do tomorrow.
Sorry for taking so long, I opened a PR for spiff, but its not a
regression, see PR6785.
To sum up: all tests (including native) run out of memory, llvm 2.6 had
this problem too, but it didn't miscompare (it was all Out of memory).
Now in 2.7 the JIT crashes when running out of memory.

While that is a bug, I'm not sure if its part of the release criteria.
I tried to make some changes in MallocSlabAllocator to not crash, but
then FoldingSet crashed on OOM.

P.S.: looking at the current 2.7 release blockers I only see PR6696 as
relevant, the others are warnings, experimental code, and a fortran failure.
Is that the case, or are the others release blockers too?


Best regards,
--Edwin

On Apr 5, 2010, at 1:28 PM, Török Edwin wrote:
> On 03/30/2010 09:21 PM, Török Edwin wrote:
>> On 03/30/2010 09:15 PM, Tanya Lattner wrote:
>>> Thanks for testing the release!
>>> 
>>>> Tests were run on x86-64, Debian unstable, Linux 2.6.33, gcc
4.4.3,
>>>> 64-bit. I built srcdir == objdir, I have built llvm and clang
myself,
>>>> and used the binaries for llvm-gcc.
>>>> 
>>>> 1. llvm-gcc 2.7 vs 2.6
>>>> compared to my results from Aug 31 2009, ignoring CBE failures:
>>>> 
>>>> new JIT failures:
>>>> MultiSource/Applications/spiff/spiff
>>>> SingleSource/Regression/C/2004-03-15-IndirectGoto
>>>> 
>>> Yes, I'm seeing the second regression on darwin too. Please
file a bug for the other one if you havent already.
>> 
>> I haven't, will do tomorrow.
> 
> Sorry for taking so long, I opened a PR for spiff, but its not a
> regression, see PR6785.
> To sum up: all tests (including native) run out of memory, llvm 2.6 had
> this problem too, but it didn't miscompare (it was all Out of memory).
> Now in 2.7 the JIT crashes when running out of memory.
> 
If thats the case, then not a release blocker.
> P.S.: looking at the current 2.7 release blockers I only see PR6696 as
> relevant, the others are warnings, experimental code, and a fortran
failure.
> Is that the case, or are the others release blockers too?
> 
- warnings are release blockers. 
- fortran build failure may end up not being a release blocker. This is under
discussion since we had originally added compiling llvm-gcc w/ fortran support
as part of the criteria, but its not being tested at all on darwin (which is a
huge problem and I dont think we should only be testing it right before a
release).

All the bugs have been updated with additional information. I will remove them
if its determined to not be a release blocker (ie. PR6778), but for now.. they
all are.

-Tanya