thr3ads.net - llvm dev - [LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD_

If this information is useful, please help other people find it:
Share via:

Duncan Sands

2011-Oct-12 08:24 UTC

[LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

Hi Bob, are these performance regressions real?  They look pretty serious.

Ciao, Duncan.

On 10/12/11 09:40, llvm-testresults at cs.uiuc.edu
wrote:>
>   bwilson__llvm-gcc_PROD__i386 nightly tester results
>
> URL	http://llvm.org/perf/db_default/simple/nts/332/
> Nickname	bwilson__llvm-gcc_PROD__i386:4
> Name	curlew.apple.com
>
> Run 	ID 	Order 	Start Time 	End Time
> Current	332	0	2011-10-12 06:59:18	2011-10-12 07:42:03
> Previous	331	0	2011-10-11 06:59:18	2011-10-11 07:41:45
>
>
>       Tests Summary
>
> Status Group	#
> Performance Regressions	17
> Performance Improvements	6
> Added Tests	2
> Unchanged Tests	938
> *Total Tests*	*963*
>
>
>       Changes Detail
>
> Performance Regressions 	Δ	Previous	Current 	σ
> MultiSource/Applications/aha/aha.exec	12.07%	3.5424	3.9698	0.0002
> MultiSource/Benchmarks/VersaBench/bmm/bmm.exec	11.38%	2.2970	2.5585	0.0007
> External/SPEC/CFP2000/179_art/179_art.exec	10.95%	1.4190	1.5744	0.0013
> MultiSource/Applications/JM/lencod/lencod.exec	6.81%	11.8149	12.6195	0.0267
> External/SPEC/CINT2006/458_sjeng/458_sjeng.exec	5.72%	6.0263	6.3711	0.0076
> MultiSource/Benchmarks/mafft/pairlocalalign.exec	3.98%	29.8022	30.9869
0.0810
> MultiSource/Benchmarks/Fhourstones-3_1/fhourstones3_1.exec	3.88%	2.0729
2.1534
> 0.0010
> External/SPEC/CINT2000/256_bzip2/256_bzip2.exec	2.68%	10.3792	10.6571
0.0138
> External/SPEC/CINT2000/164_gzip/164_gzip.exec	2.64%	13.4866	13.8424	0.0737
> External/SPEC/CINT2000/175_vpr/175_vpr.exec	2.61%	4.5220	4.6399	0.0007
> External/SPEC/CINT2000/197_parser/197_parser.compile	2.21%	3.1692	3.2392
0.0075
> MultiSource/Benchmarks/sim/sim.exec	1.97%	6.1997	6.3219	0.0004
> MultiSource/Applications/sqlite3/sqlite3.exec	1.30%	4.6635	4.7241	0.0028
> MultiSource/Benchmarks/Bullet/bullet.compile	1.26%	45.2907	45.8608	0.0695
> External/SPEC/CINT2006/483_xalancbmk/483_xalancbmk.compile	1.25%	223.7703
> 226.5641	0.1639
> External/SPEC/CFP2006/450_soplex/450_soplex.compile	1.04%	30.5935	30.9124
0.0283
> SingleSource/Benchmarks/CoyoteBench/almabench.exec	1.00%	12.0546	12.1754
0.0049
>
> Performance Improvements 	Δ	Previous	Current 	σ
> SingleSource/Benchmarks/Misc/fbench.exec	-8.84%	2.0834	1.8992	0.0050
> External/SPEC/CINT2006/400_perlbench/400_perlbench.exec	-5.17%	9.5375
9.0445	0.0241
> External/SPEC/CINT2006/456_hmmer/456_hmmer.exec	-2.78%	5.3663	5.2169	0.0008
> MultiSource/Applications/lua/lua.exec	-2.50%	28.8779	28.1550	0.0036
> External/Povray/povray.exec	-1.73%	4.3798	4.3040	0.0271
> MultiSource/Applications/lambda-0_1_3/lambda.exec	-1.56%	6.6249	6.5214
0.0019
>
> Added Tests
> SingleSource/Benchmarks/Stanford/FloatMM.compile
> SingleSource/Benchmarks/Stanford/FloatMM.exec
>
>
>
> _______________________________________________
> llvm-testresults mailing list
> llvm-testresults at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-testresults

Bob Wilson

2011-Oct-12 16:03 UTC

head link

[LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

Yes, they are real.  I re-ran the two tests with the biggest execution time
regressions, and the results were completely reproducible.

On Oct 12, 2011, at 1:24 AM, Duncan Sands wrote:
> Hi Bob, are these performance regressions real?  They look pretty serious.
> 
> Ciao, Duncan.
> 
> On 10/12/11 09:40, llvm-testresults at cs.uiuc.edu wrote:
>> 
>>  bwilson__llvm-gcc_PROD__i386 nightly tester results
>> 
>> URL	http://llvm.org/perf/db_default/simple/nts/332/
>> Nickname	bwilson__llvm-gcc_PROD__i386:4
>> Name	curlew.apple.com
>> 
>> Run 	ID 	Order 	Start Time 	End Time
>> Current	332	0	2011-10-12 06:59:18	2011-10-12 07:42:03
>> Previous	331	0	2011-10-11 06:59:18	2011-10-11 07:41:45
>> 
>> 
>>      Tests Summary
>> 
>> Status Group	#
>> Performance Regressions	17
>> Performance Improvements	6
>> Added Tests	2
>> Unchanged Tests	938
>> *Total Tests*	*963*
>> 
>> 
>>      Changes Detail
>> 
>> Performance Regressions 	Δ	Previous	Current 	σ
>> MultiSource/Applications/aha/aha.exec	12.07%	3.5424	3.9698	0.0002
>> MultiSource/Benchmarks/VersaBench/bmm/bmm.exec	11.38%	2.2970	2.5585
0.0007
>> External/SPEC/CFP2000/179_art/179_art.exec	10.95%	1.4190	1.5744	0.0013
>> MultiSource/Applications/JM/lencod/lencod.exec	6.81%	11.8149	12.6195
0.0267
>> External/SPEC/CINT2006/458_sjeng/458_sjeng.exec	5.72%	6.0263	6.3711
0.0076
>> MultiSource/Benchmarks/mafft/pairlocalalign.exec	3.98%	29.8022	30.9869
0.0810
>> MultiSource/Benchmarks/Fhourstones-3_1/fhourstones3_1.exec	3.88%	2.0729
2.1534
>> 0.0010
>> External/SPEC/CINT2000/256_bzip2/256_bzip2.exec	2.68%	10.3792	10.6571
0.0138
>> External/SPEC/CINT2000/164_gzip/164_gzip.exec	2.64%	13.4866	13.8424
0.0737
>> External/SPEC/CINT2000/175_vpr/175_vpr.exec	2.61%	4.5220	4.6399	0.0007
>> External/SPEC/CINT2000/197_parser/197_parser.compile	2.21%	3.1692
3.2392	0.0075
>> MultiSource/Benchmarks/sim/sim.exec	1.97%	6.1997	6.3219	0.0004
>> MultiSource/Applications/sqlite3/sqlite3.exec	1.30%	4.6635	4.7241
0.0028
>> MultiSource/Benchmarks/Bullet/bullet.compile	1.26%	45.2907	45.8608
0.0695
>> External/SPEC/CINT2006/483_xalancbmk/483_xalancbmk.compile	1.25%
223.7703
>> 226.5641	0.1639
>> External/SPEC/CFP2006/450_soplex/450_soplex.compile	1.04%	30.5935
30.9124	0.0283
>> SingleSource/Benchmarks/CoyoteBench/almabench.exec	1.00%	12.0546
12.1754	0.0049
>> 
>> Performance Improvements 	Δ	Previous	Current 	σ
>> SingleSource/Benchmarks/Misc/fbench.exec	-8.84%	2.0834	1.8992	0.0050
>> External/SPEC/CINT2006/400_perlbench/400_perlbench.exec	-5.17%	9.5375
9.0445	0.0241
>> External/SPEC/CINT2006/456_hmmer/456_hmmer.exec	-2.78%	5.3663	5.2169
0.0008
>> MultiSource/Applications/lua/lua.exec	-2.50%	28.8779	28.1550	0.0036
>> External/Povray/povray.exec	-1.73%	4.3798	4.3040	0.0271
>> MultiSource/Applications/lambda-0_1_3/lambda.exec	-1.56%	6.6249	6.5214
0.0019
>> 
>> Added Tests
>> SingleSource/Benchmarks/Stanford/FloatMM.compile
>> SingleSource/Benchmarks/Stanford/FloatMM.exec
>> 
>> 
>> 
>> _______________________________________________
>> llvm-testresults mailing list
>> llvm-testresults at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-testresults
>

Evan Cheng

2011-Oct-13 18:48 UTC

head link

[LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

If I were to guess, I would say it's due to machine LICM changes. Although I
can't understand why the regressions are not showing up on Clang testers.

Evan

On Oct 12, 2011, at 9:03 AM, Bob Wilson wrote:
> Yes, they are real.  I re-ran the two tests with the biggest execution time
regressions, and the results were completely reproducible.
> 
> On Oct 12, 2011, at 1:24 AM, Duncan Sands wrote:
> 
>> Hi Bob, are these performance regressions real?  They look pretty
serious.
>> 
>> Ciao, Duncan.
>> 
>> On 10/12/11 09:40, llvm-testresults at cs.uiuc.edu wrote:
>>> 
>>> bwilson__llvm-gcc_PROD__i386 nightly tester results
>>> 
>>> URL	http://llvm.org/perf/db_default/simple/nts/332/
>>> Nickname	bwilson__llvm-gcc_PROD__i386:4
>>> Name	curlew.apple.com
>>> 
>>> Run 	ID 	Order 	Start Time 	End Time
>>> Current	332	0	2011-10-12 06:59:18	2011-10-12 07:42:03
>>> Previous	331	0	2011-10-11 06:59:18	2011-10-11 07:41:45
>>> 
>>> 
>>>     Tests Summary
>>> 
>>> Status Group	#
>>> Performance Regressions	17
>>> Performance Improvements	6
>>> Added Tests	2
>>> Unchanged Tests	938
>>> *Total Tests*	*963*
>>> 
>>> 
>>>     Changes Detail
>>> 
>>> Performance Regressions 	Δ	Previous	Current 	σ
>>> MultiSource/Applications/aha/aha.exec	12.07%	3.5424	3.9698	0.0002
>>> MultiSource/Benchmarks/VersaBench/bmm/bmm.exec	11.38%	2.2970	2.5585
0.0007
>>> External/SPEC/CFP2000/179_art/179_art.exec	10.95%	1.4190	1.5744
0.0013
>>> MultiSource/Applications/JM/lencod/lencod.exec	6.81%	11.8149
12.6195	0.0267
>>> External/SPEC/CINT2006/458_sjeng/458_sjeng.exec	5.72%	6.0263	6.3711
0.0076
>>> MultiSource/Benchmarks/mafft/pairlocalalign.exec	3.98%	29.8022
30.9869	0.0810
>>> MultiSource/Benchmarks/Fhourstones-3_1/fhourstones3_1.exec	3.88%
2.0729	2.1534
>>> 0.0010
>>> External/SPEC/CINT2000/256_bzip2/256_bzip2.exec	2.68%	10.3792
10.6571	0.0138
>>> External/SPEC/CINT2000/164_gzip/164_gzip.exec	2.64%	13.4866	13.8424
0.0737
>>> External/SPEC/CINT2000/175_vpr/175_vpr.exec	2.61%	4.5220	4.6399
0.0007
>>> External/SPEC/CINT2000/197_parser/197_parser.compile	2.21%	3.1692
3.2392	0.0075
>>> MultiSource/Benchmarks/sim/sim.exec	1.97%	6.1997	6.3219	0.0004
>>> MultiSource/Applications/sqlite3/sqlite3.exec	1.30%	4.6635	4.7241
0.0028
>>> MultiSource/Benchmarks/Bullet/bullet.compile	1.26%	45.2907	45.8608
0.0695
>>> External/SPEC/CINT2006/483_xalancbmk/483_xalancbmk.compile	1.25%
223.7703
>>> 226.5641	0.1639
>>> External/SPEC/CFP2006/450_soplex/450_soplex.compile	1.04%	30.5935
30.9124	0.0283
>>> SingleSource/Benchmarks/CoyoteBench/almabench.exec	1.00%	12.0546
12.1754	0.0049
>>> 
>>> Performance Improvements 	Δ	Previous	Current 	σ
>>> SingleSource/Benchmarks/Misc/fbench.exec	-8.84%	2.0834	1.8992
0.0050
>>> External/SPEC/CINT2006/400_perlbench/400_perlbench.exec	-5.17%
9.5375	9.0445	0.0241
>>> External/SPEC/CINT2006/456_hmmer/456_hmmer.exec	-2.78%	5.3663
5.2169	0.0008
>>> MultiSource/Applications/lua/lua.exec	-2.50%	28.8779	28.1550	0.0036
>>> External/Povray/povray.exec	-1.73%	4.3798	4.3040	0.0271
>>> MultiSource/Applications/lambda-0_1_3/lambda.exec	-1.56%	6.6249
6.5214	0.0019
>>> 
>>> Added Tests
>>> SingleSource/Benchmarks/Stanford/FloatMM.compile
>>> SingleSource/Benchmarks/Stanford/FloatMM.exec
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> llvm-testresults mailing list
>>> llvm-testresults at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-testresults
>> 
> 
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Seemingly Similar Threads

Search for more apparently analagous threads

llvm dev - Oct 2011 - [LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

[LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

[LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

[LLVMdev] [llvm-testresults] bwilson__llvm-gcc_PROD__i386 nightly tester results

Seemingly Similar Threads