thr3ads.net - llvm dev - [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks [Sep 2009]

If this information is useful, please help other people find it:
Share via:

Keir Mierle

2009-Sep-14 20:02 UTC

[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

On Mon, Sep 14, 2009 at 12:23 PM, Renato Golin <rengolin at
systemcall.org>wrote:
> 2009/9/14 Edward O'Callaghan <eocallaghan at auroraux.org>:
> > screw that site, its useless info run by a linux gnu zealot.
>
> Well, being a Linux GNU zealot doesn't invalidate numbers. What does
> invalidate is that he has no numbers!
>
> He just fired a few applications and counted the seconds, this is
> utterly pointless. No reference to how many times he ran, standard
> deviations, possible effects of OS cache, compilation parameters, etc,
> etc. There simply isn't anything in there. It even lack text in
> between the charts, a conclusion, and... well... everything.
>
> "Ignore that site", would also be more polite... ;)
>
Nevertheless, the site has a large audience. From a publicity perspective,
it may be worthwhile to work with the author to get more representative
numbers.

Keir


>
> cheers,
> --renato
>
> Reclaim your digital rights, eliminate DRM, learn more at
> http://www.defectivebydesign.org/what_is_drm
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20090914/b7adc9af/attachment.html>

Olivier Meurant

2009-Sep-15 13:16 UTC

head link

[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

Hi everyone,
I have replayed the "unix benchmark v3.6 dhrystone 2" test.

You can find the archive of the test here :
http://www.phoronix-test-suite.com/benchmark-files/byte-benchmark-1.tar.gz
To play the test on gcc :
make
./Run dhry2
To play the test on llvm-gcc :
Replace in Makefile : CC=gcc by CC=llvm-gcc
in Run : CC=gcc by CC=llvm-gcc
make
./Run dhry2

Some information on the test platform :

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION="Ubuntu 9.04"

uname -a
Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC 2009
i686 GNU/Linux

/proc/cpuinfo
dual core Intel(R) Core(TM)2 CPU X6800  @ 2.93GHz (cache size 4096)

llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../llvm-gcc4.2-2.5.source/
configure
--prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/../install
--program-prefix=llvm-
--enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/
--disable-bootstrap --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5)
(Binary from official x86 2.5 release)

gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
4.3.3-5ubuntu4' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --enable-nls
--with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr
--enable-targets=all --with-tune=generic --enable-checking=release
--build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4)

The test displays its results in form of a number with unit seems to be
"loops per seconds". So it's logical to say that the bigger is
better.
I run the test 10 times for each compiler (for an approximate duration of 20
minutes each compiler).

Results:
LLVM average : 10696687.14
GCC average : 9563046.99
LLVM is on average 10.58% faster than GCC on this test.

If you need more informations or if you think my test was not correctly
made... ping me. :)

Here a quick report on the test. (shows individual result of each test) :


LLVM GCC Difference Difference %




 Run 1 10720897.7 9875431 845466.7 7.89  Run 2 10411474.6 9666138.5 745336.1
7.16  Run 3 10580705.2 9265904.6 1314800.6 12.43  Run 4 10746408 9539499.9
1206908.1 11.23  Run 5 10818845.5 9766210.7 1052634.8 9.73  Run 6 10904404.3
10034620.8 869783.5 7.98  Run 7 10861482.8 9028718.6 1832764.2 16.87  Run 8
10480568.5 9568849.4 911719.1 8.7  Run 9 10857968.2 9306331.6 1551636.6
14.29  Run 10 10584116.6 9578764.8 1005351.8 9.5




 Average 10696687.14 9563046.99 1133640.15 10.58  Standard deviation
172714.92 300445.48 345992.26 3.13
Cheers,
Olivier.


On Mon, Sep 14, 2009 at 10:02 PM, Keir Mierle <mierle at gmail.com>
wrote:>
> On Mon, Sep 14, 2009 at 12:23 PM, Renato Golin <rengolin at
systemcall.org>
wrote:>>
>> 2009/9/14 Edward O'Callaghan <eocallaghan at auroraux.org>:
>> > screw that site, its useless info run by a linux gnu zealot.
>>
>> Well, being a Linux GNU zealot doesn't invalidate numbers. What
does
>> invalidate is that he has no numbers!
>>
>> He just fired a few applications and counted the seconds, this is
>> utterly pointless. No reference to how many times he ran, standard
>> deviations, possible effects of OS cache, compilation parameters, etc,
>> etc. There simply isn't anything in there. It even lack text in
>> between the charts, a conclusion, and... well... everything.
>>
>> "Ignore that site", would also be more polite... ;)
>
> Nevertheless, the site has a large audience. From a publicity perspective,it may be worthwhile to work with the author to get more representative
numbers.> Keir
>
>>
>> cheers,
>> --renato
>>
>> Reclaim your digital rights, eliminate DRM, learn more at
>> http://www.defectivebydesign.org/what_is_drm
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20090915/edc5ef63/attachment.html>

David Greene

2009-Sep-15 15:35 UTC

head link

[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

On Tuesday 15 September 2009 08:16, Olivier Meurant wrote:
> llvm-gcc -v
> Using built-in specs.
> Target: i686-pc-linux-gnu
>
> gcc -v
> Using built-in specs.
> Target: i486-linux-gnu
Different targets.  It's no surprise 486 code would perform much worse than 
i686 code.  These compilers need to target the same architecture.  Ideally the 
Makefile would include appropriate -march/-mtune/whatever the gcc architecture 
switch of the day is.

All test reports should also include switches passed to the compiler for each 
run.

Right now this isn't a fair test.  It's practically meaningless.

Remember, the goal here isn't to show how great LLVM is.  It's to get an
honest assessment of where we are at.  Phoronix did us a big favor.  Getting
more details about his tests would help us even more.

For one thing, we (Cray) have a number of LLVM enhancements geared toward 
Opteron that make a big difference in performance.  I'm planning to
contribute
much of this to our open source community.  So there is definitely room for 
improvement to LLVM.  The biggest mistake we can make is to think we're the 
best.

                                 -Dave

Seemingly Similar Threads

Search for more possibly parallel threads

llvm dev - Sep 2009 - [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

Seemingly Similar Threads