On Tuesday 15 September 2009 08:16, Olivier Meurant wrote:> llvm-gcc -v > Using built-in specs. > Target: i686-pc-linux-gnu> > gcc -v > Using built-in specs. > Target: i486-linux-gnuDifferent targets. It's no surprise 486 code would perform much worse than i686 code. These compilers need to target the same architecture. Ideally the Makefile would include appropriate -march/-mtune/whatever the gcc architecture switch of the day is. All test reports should also include switches passed to the compiler for each run. Right now this isn't a fair test. It's practically meaningless. Remember, the goal here isn't to show how great LLVM is. It's to get an honest assessment of where we are at. Phoronix did us a big favor. Getting more details about his tests would help us even more. For one thing, we (Cray) have a number of LLVM enhancements geared toward Opteron that make a big difference in performance. I'm planning to contribute much of this to our open source community. So there is definitely room for improvement to LLVM. The biggest mistake we can make is to think we're the best. -Dave
罗勇刚(Yonggang Luo)
2009-Sep-15 16:28 UTC
[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
Wow, that's beautiful: and thanks 2009/9/15, David Greene <dag at cray.com>:> On Tuesday 15 September 2009 08:16, Olivier Meurant wrote: > >> llvm-gcc -v >> Using built-in specs. >> Target: i686-pc-linux-gnu > >> >> gcc -v >> Using built-in specs. >> Target: i486-linux-gnu > > Different targets. It's no surprise 486 code would perform much worse than > i686 code. These compilers need to target the same architecture. Ideally > the > Makefile would include appropriate -march/-mtune/whatever the gcc > architecture > switch of the day is. > > All test reports should also include switches passed to the compiler for > each > run. > > Right now this isn't a fair test. It's practically meaningless. > > Remember, the goal here isn't to show how great LLVM is. It's to get an > honest assessment of where we are at. Phoronix did us a big favor. Getting > more details about his tests would help us even more. > > For one thing, we (Cray) have a number of LLVM enhancements geared toward > Opteron that make a big difference in performance. I'm planning to > contribute > much of this to our open source community. So there is definitely room for > improvement to LLVM. The biggest mistake we can make is to think we're the > best. > > -Dave > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-- 从我的移动设备发送 此致 礼 罗勇刚 Yours sincerely, Yonggang Luo
On Tue, Sep 15, 2009 at 5:35 PM, David Greene <dag at cray.com> wrote:> > Remember, the goal here isn't to show how great LLVM is. It's to get an > honest assessment of where we are at. Phoronix did us a big favor. > Getting > more details about his tests would help us even more. >It's exactly what I'm trying to do. I'm not even a LLVM contributor, so I don't feel responsible for a good or bad performances of LLVM. :) Anyway, thanks for pointing this target differences. After some hours at recompiling gcc, here we are : Some information on the test platform : cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=9.04 DISTRIB_CODENAME=jaunty DISTRIB_DESCRIPTION="Ubuntu 9.04" uname -a Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC 2009 i686 GNU/Linux /proc/cpuinfo dual core Intel(R) Core(TM)2 CPU X6800 @ 2.93GHz (cache size 4096) llvm-gcc -v Using built-in specs. Target: i686-pc-linux-gnu Configured with: ../llvm-gcc4.2-2.5.source/configure --prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/../install --program-prefix=llvm- --enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/ --disable-bootstrap --enable-languages=c,c++,fortran Thread model: posix gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5) (Binary from official x86 2.5 release) gcc -v Using built-in specs. Target: i686-pc-linux-gnu Configured with: ./configure --prefix=/usr/local Thread model: posix gcc version 4.4.1 (GCC) The compilation line are : gcc -o bin src.c -O3 llvm-gcc -o bin src.c -O3 Results : LLVM average : 13553161.95 gcc average : 14624441.45 Average : gcc is better with 8% The full report : "LLVM" "GCC" "Difference" "Difference %" "Run 1" 13842616 14693115.8 -850499.8 -6.14 "Run 2" 13773202.9 14785734.9 -1012532 -7.35 "Run 3" 13671309.3 14985114 -1313804.7 -9.61 "Run 4" 12916939 14640392.9 -1723453.9 -13.34 "Run 5" 13572133.3 14425575 -853441.7 -6.29 "Run 6" 13810096.5 14694240.6 -884144.1 -6.4 "Run 7" 13427435.8 14695094.6 -1267658.8 -9.44 "Run 8" 14056659.9 14468968.4 -412308.5 -2.93 "Run 9" 13699125.3 14373360.9 -674235.6 -4.92 "Run 10 12762101.5 14482817.4 -1720715.9 -13.48 "Average" 13553161.95 14624441.45 -1071279.5 -7.99 "Standard deviation" 412810.77 187898.68 430895.31 3.46 Is there again something wrong on this test ? Olivier. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090916/17b14394/attachment.html>
Anton Korobeynikov
2009-Sep-16 09:14 UTC
[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
Hello, Oliver> The compilation line are : > gcc -o bin src.c -O3 > llvm-gcc -o bin src.c -O3I'm not quite sure which target processor the compiler will optimize code for. I'd suggest adding explicit -march=core2 here. Also, could you please do a separate run with O2, not O3. Thanks! -- With best regards, Anton Korobeynikov Faculty of Mathematics and Mechanics, Saint Petersburg State University
On 2009-09-16 11:42, Olivier Meurant wrote:> On Tue, Sep 15, 2009 at 5:35 PM, David Greene <dag at cray.com > <mailto:dag at cray.com>> wrote: > > > Remember, the goal here isn't to show how great LLVM is. It's to > get an > honest assessment of where we are at. Phoronix did us a big > favor. Getting > more details about his tests would help us even more. > > > It's exactly what I'm trying to do. I'm not even a LLVM contributor, > so I don't feel responsible for a good or bad performances of LLVM. :) > Anyway, thanks for pointing this target differences. After some hours > at recompiling gcc, here we are : > > Some information on the test platform : > > cat /etc/lsb-release > DISTRIB_ID=Ubuntu > DISTRIB_RELEASE=9.04 > DISTRIB_CODENAME=jaunty > DISTRIB_DESCRIPTION="Ubuntu 9.04" > > uname -a > Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC > 2009 i686 GNU/Linux > > /proc/cpuinfo > dual core Intel(R) Core(TM)2 CPU X6800 @ 2.93GHz (cache size 4096) > > llvm-gcc -v > Using built-in specs. > Target: i686-pc-linux-gnu > Configured with: ../llvm-gcc4.2-2.5.source/configure > --prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/../install > --program-prefix=llvm- > --enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/ > --disable-bootstrap --enable-languages=c,c++,fortran > Thread model: posix > gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5) > (Binary from official x86 2.5 release) > > gcc -v > Using built-in specs. > Target: i686-pc-linux-gnu > Configured with: ./configure --prefix=/usr/local > Thread model: posix > gcc version 4.4.1 (GCC)Phoronix tested with gcc 4.2.1 (Apple's).> > The compilation line are : > gcc -o bin src.c -O3 > llvm-gcc -o bin src.c -O3The Makefile uses -O by default, perhaps that is what Phoronix used too.> > > Results : > LLVM average : 13553161.95 > gcc average : 14624441.45 > Average : gcc is better with 8% > > The full report : > "LLVM" "GCC" "Difference" "Difference %" > > "Run 1" 13842616 14693115.8 -850499.8 -6.14 > "Run 2" 13773202.9 14785734.9 -1012532 -7.35 > "Run 3" 13671309.3 14985114 -1313804.7 -9.61 > "Run 4" 12916939 14640392.9 -1723453.9 -13.34 > "Run 5" 13572133.3 14425575 -853441.7 -6.29 > "Run 6" 13810096.5 14694240.6 -884144.1 -6.4 > "Run 7" 13427435.8 14695094.6 -1267658.8 -9.44 > "Run 8" 14056659.9 14468968.4 -412308.5 -2.93 > "Run 9" 13699125.3 14373360.9 -674235.6 -4.92 > "Run 10 12762101.5 14482817.4 -1720715.9 -13.48 > > "Average" 13553161.95 14624441.45 -1071279.5 -7.99 > "Standard deviation" 412810.77 187898.68 > 430895.31 3.46 > > Is there again something wrong on this test ?No, it seems fine. I get different results on x86-64 though (with -O), the difference is very small (4% with 3% stddev, or 3% with 1.5% stddev). Maybe you should try to reproduce one of the other benchmarks that show a higher difference, according to phoronix JOhn the Ripper shown 40% performance drop, there was also a huge drop for blowfish. GCC 4.4.1 GCC 4.2.4 LLVM-GCC 2.5 LLVM-GCC 2.6 GCC-4.4/LLVM2.5 GCC-4.4/LLVM2.6 GCC-4.4/GCC-4.2 144663277 139072037 154747143 151467440 0.935 0.955 1.040 144413249 139042455 150949846 150322055 0.957 0.961 1.039 144915166 138131283 150630261 150650540 0.962 0.962 1.049 145602711 138587982 154993781 147281539 0.939 0.989 1.051 145547868 139224734 150650813 147998859 0.966 0.983 1.045 145719970 139224734 144055081 152474829 1.012 0.956 1.047 average 145143706.83 138880537.5 151004487.5 150032543.67 0.962 0.968 1.045 stdev 551855.54 435155.79 3961926.31 2009642.82 0.027 0.015 0.005 Using built-in specs. Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Debian 4.4.1-4' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --program-suffix=-4.4 --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-mpfr --enable-objc-gc --with-arch-32=i486 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 4.4.1 (Debian 4.4.1-4) llvm2.5: Using built-in specs. Target: x86_64-unknown-linux-gnu Configured with: ../llvm-gcc-4.2/configure --program-prefix=llvm- --enable-llvm=/home/edwin/llvm2.5/obj --enable-languages=c,c++,fortran Thread model: posix gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build) llvm2.6pre1: Using built-in specs. Target: x86_64-unknown-linux-gnu Configured with: ../llvm-gcc4.2-2.6.source/configure --program-prefix=llvm- --enable-languages=c,c++,fortran --enable-llvm=/home/edwin/llvm2.6/obj/ Thread model: posix gcc version 4.2.1 (Based on Apple Inc. build 5649) (LLVM build) Using built-in specs. Target: x86_64-linux-gnu Configured with: ../src/configure -v --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 4.2.4 (Debian 4.2.4-6) Best regards, --Edwin
Possibly Parallel Threads
- [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
- [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
- [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
- [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
- [LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks