Török Edwin
2010-Mar-24 21:47 UTC
[LLVMdev] [cfe-dev] 2.7 Pre-release1 available for testing
On 03/17/2010 10:12 PM, Tanya Lattner wrote:> The 2.7 binaries are available for testing: > http://llvm.org/pre-releases/2.7/pre-release1/ > > You will also find the source tarballs there as well. > > We rely on the community to help make our releases great, so please help > test 2.7 if you can. Please follow these instructions to test 2.7: > > /To test llvm-gcc:/ > > 1) Compile llvm from source and untar the llvm-test in the projects > directory (name it llvm-test or test-suite). Choose to use a pre- > compiled llvm-gcc or re-compile it yourself. > > 2) Run make check, report any failures (FAIL or unexpected pass). Note > that you need to reconfigure llvm with llvm-gcc in your path or with -- with-llvmgccdir > > 3) Run "make TEST=nightly report". Compare these results to a 2.6 llvm-test nightly report or send the results to the list. For supported targets, we'll try to examine the results, but its best if you can do the comparison yourself. >Hi Tanya, Attached are the nightly test results when run with llvm-gcc (report.nightly.txt), and when run with clang (clang-report.nightly.txt). Tests were run on x86-64, Debian unstable, Linux 2.6.33, gcc 4.4.3, 64-bit. I built srcdir == objdir, I have built llvm and clang myself, and used the binaries for llvm-gcc. 1. llvm-gcc 2.7 vs 2.6 compared to my results from Aug 31 2009, ignoring CBE failures: new JIT failures: MultiSource/Applications/spiff/spiff SingleSource/Regression/C/2004-03-15-IndirectGoto 2. llvm-gcc 2.7 vs clang 2.7 When comparing the 2.7 llvm-gcc and clang results I see these differences (is llvm-gcc considered baseline for clang?): ALL FAIL (pass in llvm-gcc): MultiSource/Benchmarks/PAQ8p/paq8p MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4 MultiSource/Benchmarks/Prolangs-C/archie-client/archie MultiSource/Benchmarks/Prolangs-C/cdecl/cdecl SingleSource/Benchmarks/Misc-C++/bigfib SingleSource/Regression/C++/EH/ConditionalExpr SingleSource/Regression/C++/EH/ctor_dtor_count-2 SingleSource/Regression/C++/EH/function_try_block SingleSource/Regression/C++/EH/simple_throw SingleSource/UnitTests/2006-12-04-DynAllocAndRestore SingleSource/UnitTests/Vector/SSE/sse.expandfft SingleSource/UnitTests/Vector/SSE/sse.stepfft JIT failures in clang, pass in llvm-gcc: MultiSource/Applications/sqlite3/sqlite3 SingleSource/Regression/C++/ofstream_ctor 3. Some performance regressions GCC/LLC (2.6 -> 2.7), but keep in mind that I wasn't using GCC 4.4.3 as comparison for llvm 2.6! MultiSource/Applications/hexxagon/hexxagon 1.22 -> 1.14 MultiSource/Applications/lua/lua 0.91 -> 0.84 MultiSource/Applications/obsequi/Obsequi 0.93 -> 0.86 MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk 1.01 -> 0.91 MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow 0.94 -> 0.75 MultiSource/Benchmarks/FreeBench/neural/neural 1.0 -> 0.9 MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm 1.06 -> 0.9 MultiSource/Benchmarks/Olden/treeadd/treeadd 11.44 -> 9.89 MultiSource/Benchmarks/Olden/tsp/tsp 1.14 -> 1.02 MultiSource/Benchmarks/Ptrdist/anagram/anagram 1.33 -> 1.23 SingleSource/Benchmarks/Dhrystone/dry 7.32 -> 5.16 SingleSource/Benchmarks/Dhrystone/fldry 8.02 -> 6.65 .... I'll have to write a script to compare the results, its boring and inaccurate to do by hand. Will go through the bugzilla tomorrow and see if I need to open new bugs for this stuff.> > /To test clang:/ > > 1) Compile llvm and clang from source. > > 2) Run make check for llvm. > > 3) Run make -C tools/clang-2.6 test VERBOSE=1 (report any failures or > unexpected passes)Surely you meant tools/clang-2.7 FYI I pulled the following revisions for ClamAV's llvm on top of 2.7: r98349 r98410 r98447 r98508 r99143 r99146 r99147 r99160 r99400 I don't know if any of these qualify as regression fixes for 2.7, I'll leave it up to you to decide if you want to put them into 2.7 or not. Best regards, --Edwin -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: report.nightly.txt URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100324/835efbf0/attachment.txt> -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: clang-report.nightly.txt URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100324/835efbf0/attachment-0001.txt>
Török Edwin
2010-Mar-30 18:21 UTC
[LLVMdev] [cfe-dev] 2.7 Pre-release1 available for testing
On 03/30/2010 09:15 PM, Tanya Lattner wrote:> > Thanks for testing the release! > >> Tests were run on x86-64, Debian unstable, Linux 2.6.33, gcc 4.4.3, >> 64-bit. I built srcdir == objdir, I have built llvm and clang myself, >> and used the binaries for llvm-gcc. >> >> 1. llvm-gcc 2.7 vs 2.6 >> compared to my results from Aug 31 2009, ignoring CBE failures: >> >> new JIT failures: >> MultiSource/Applications/spiff/spiff >> SingleSource/Regression/C/2004-03-15-IndirectGoto >> > > Yes, I'm seeing the second regression on darwin too. Please file a bug for the other one if you havent already.I haven't, will do tomorrow.> >> 2. llvm-gcc 2.7 vs clang 2.7 >> When comparing the 2.7 llvm-gcc and clang results I see these >> differences (is llvm-gcc considered baseline for clang?): >> ALL FAIL (pass in llvm-gcc): >> MultiSource/Benchmarks/PAQ8p/paq8p >> MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4 >> MultiSource/Benchmarks/Prolangs-C/archie-client/archie >> MultiSource/Benchmarks/Prolangs-C/cdecl/cdecl >> SingleSource/Benchmarks/Misc-C++/bigfib >> SingleSource/Regression/C++/EH/ConditionalExpr >> SingleSource/Regression/C++/EH/ctor_dtor_count-2 >> SingleSource/Regression/C++/EH/function_try_block >> SingleSource/Regression/C++/EH/simple_throw >> SingleSource/UnitTests/2006-12-04-DynAllocAndRestore >> SingleSource/UnitTests/Vector/SSE/sse.expandfft >> SingleSource/UnitTests/Vector/SSE/sse.stepfft >> >> JIT failures in clang, pass in llvm-gcc: >> MultiSource/Applications/sqlite3/sqlite3 >> SingleSource/Regression/C++/ofstream_ctor >> > > This isn't part of our release criteria. So, these are not release blockers.Ok, something to keep in mind for LLVM 2.8 then.> >> 3. Some performance regressions GCC/LLC (2.6 -> 2.7), but keep in mind >> that I wasn't using GCC 4.4.3 as comparison for llvm 2.6! >> >> MultiSource/Applications/hexxagon/hexxagon 1.22 -> 1.14 >> MultiSource/Applications/lua/lua 0.91 -> 0.84 >> MultiSource/Applications/obsequi/Obsequi 0.93 -> 0.86 >> MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk 1.01 -> 0.91 >> MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow 0.94 -> 0.75 >> MultiSource/Benchmarks/FreeBench/neural/neural 1.0 -> 0.9 >> MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm 1.06 -> 0.9 >> MultiSource/Benchmarks/Olden/treeadd/treeadd 11.44 -> 9.89 >> MultiSource/Benchmarks/Olden/tsp/tsp 1.14 -> 1.02 >> MultiSource/Benchmarks/Ptrdist/anagram/anagram 1.33 -> 1.23 >> SingleSource/Benchmarks/Dhrystone/dry 7.32 -> 5.16 >> SingleSource/Benchmarks/Dhrystone/fldry 8.02 -> 6.65 >> .... >> > > Unfortunately, we just don't have enough man power to have performance be a release criteria at this time. We also need a better infrastructure in place to track this stuff (Daniel is working on it).Yes, I understand that.> >> I'll have to write a script to compare the results, its boring and >> inaccurate to do by hand. >> >> Will go through the bugzilla tomorrowI still didn't have time to do this unfortunately.>> and see if I need to open new bugs >> for this stuff. >> >>> /To test clang:/ >>> >>> 1) Compile llvm and clang from source. >>> >>> 2) Run make check for llvm. >>> >>> 3) Run make -C tools/clang-2.6 test VERBOSE=1 (report any failures or >>> unexpected passes) >> Surely you meant tools/clang-2.7 >> > > Yes. > >> FYI I pulled the following revisions for ClamAV's llvm on top of 2.7: >> r98349 >> r98410 >> r98447 >> r98508 >> r99143 >> r99146 >> r99147 >> r99160 >> r99400 >> >> I don't know if any of these qualify as regression fixes for 2.7, I'll >> leave it up to you to decide if you want to put them into 2.7 or not. >> > > I'll have to discuss with Chris about these. Its technically not a release blocker.Meanwhile I pulled these too: 99762 (Evan approved) 99883 (leakfix, so I don't think it qualifies for release criteria) Best regards, --Edwin
Seemingly Similar Threads
- [LLVMdev] [cfe-dev] 2.7 Pre-release1 available for testing
- [LLVMdev] [cfe-dev] 2.7 Pre-release1 available for testing
- [LLVMdev] [cfe-dev] 2.7 Pre-release1 available for testing
- [LLVMdev] 2.7 Pre-release1 available for testing
- [LLVMdev] Problem While Running Test Suite