Xinliang David Li
2010-Nov-13 20:56 UTC
[LLVMdev] tot clang/llvm and tot gcc performance comparision
Hi, I have looked at the LLVM code generation quality using small test cases
and in general it is better than I thought and in some cases better than
gcc. However, there are still some gap in SPEC performance. I have not
looked at the root cause of those gaps. Anyone who cares about LLVM
performance need to take this seriously.
For fair comparison, I used -fno-strict-aliasing in gcc to turn off type
based aliasing. I also used -fomit-frame-pointer for clang/llvm because this
is not the default. The base option is -O2. Measured on 2.4GHz core-2 box.
64bit:
llvm gcc
gcc vs LLVM
164.gzip 1268 1320 4.15%
175.vpr 1605 1534 -4.42%
176.gcc 2203 2315 5.08%
181.mcf 1625 1737 6.85%
186.crafty 2411 2307 -4.30%
197.parser 1173 1166 -0.57%
252.eon 2245 2464 9.72%
253.perlbmk 2214 2444 10.37%
254.gap 1987 1978 -0.47%
255.vortex 2497 2422 -3.00%
256.bzip2 1585 1740 9.80%
300.twolf 2294 2281 -0.58%
As you can see, LLVM beats gcc in vpr, crafty and vortex by about 3%, but
falls behind gcc in gzip, gcc, mcf, eon, perfbmk, and bzip2 by a large
margin.
32bit:
llvm gcc
gcc vs llvm
164.gzip 1210 1239 2.44%
175.vpr 1662 1621 -2.42%
181.mcf 2733 3109 13.75%
186.crafty 1812 1721 -5.00%
197.parser 1328 1289 -2.92%
253.perlbmk 2086 2580 23.67%
254.gap 1968 1912 -2.86%
255.vortex 1842 1965 6.66%
256.bzip2 1440 1553 7.82%
300.twolf 2284 2213 -3.08%
It beats gcc in vpr, crafty, parser, gap, and twolf by ~3%, but falls behind
gcc in gzip, mcf, perlbmk, vortex, and bzip2. Some of them have very large
performance gap. eon and gcc mis-compile.
Thanks,
David
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20101113/2254f37b/attachment.html>
Evan Cheng
2010-Nov-14 19:52 UTC
[LLVMdev] tot clang/llvm and tot gcc performance comparision
Thanks David. Unfortunately many of us cannot use GPL v3 gcc so it's hard for us to investigate this. One question, can you tell if gcc is inlining significantly more than llvm? We have reports that this is one of the issue plaguing eon performance. There are also some relatively well known spec optimizations that we haven't implemented. e.g. http://gcc.gnu.org/wiki/summit2010?action=AttachFile&do=get&target=meissner2.pdf Are there more? Evan On Nov 13, 2010, at 12:56 PM, Xinliang David Li wrote:> > Hi, I have looked at the LLVM code generation quality using small test cases and in general it is better than I thought and in some cases better than gcc. However, there are still some gap in SPEC performance. I have not looked at the root cause of those gaps. Anyone who cares about LLVM performance need to take this seriously. > > For fair comparison, I used -fno-strict-aliasing in gcc to turn off type based aliasing. I also used -fomit-frame-pointer for clang/llvm because this is not the default. The base option is -O2. Measured on 2.4GHz core-2 box. > > 64bit: > > llvm gcc gcc vs LLVM > 164.gzip 1268 1320 4.15% > 175.vpr 1605 1534 -4.42% > 176.gcc 2203 2315 5.08% > 181.mcf 1625 1737 6.85% > 186.crafty 2411 2307 -4.30% > 197.parser 1173 1166 -0.57% > 252.eon 2245 2464 9.72% > 253.perlbmk 2214 2444 10.37% > 254.gap 1987 1978 -0.47% > 255.vortex 2497 2422 -3.00% > 256.bzip2 1585 1740 9.80% > 300.twolf 2294 2281 -0.58% > > As you can see, LLVM beats gcc in vpr, crafty and vortex by about 3%, but falls behind gcc in gzip, gcc, mcf, eon, perfbmk, and bzip2 by a large margin. > > > 32bit: > llvm gcc gcc vs llvm > 164.gzip 1210 1239 2.44% > 175.vpr 1662 1621 -2.42% > 181.mcf 2733 3109 13.75% > 186.crafty 1812 1721 -5.00% > 197.parser 1328 1289 -2.92% > 253.perlbmk 2086 2580 23.67% > 254.gap 1968 1912 -2.86% > 255.vortex 1842 1965 6.66% > 256.bzip2 1440 1553 7.82% > 300.twolf 2284 2213 -3.08% > > It beats gcc in vpr, crafty, parser, gap, and twolf by ~3%, but falls behind gcc in gzip, mcf, perlbmk, vortex, and bzip2. Some of them have very large performance gap. eon and gcc mis-compile. > > Thanks, > > David > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101114/050058a9/attachment.html>
Xinliang David Li
2010-Nov-15 06:36 UTC
[LLVMdev] tot clang/llvm and tot gcc performance comparision
On Sun, Nov 14, 2010 at 11:52 AM, Evan Cheng <evan.cheng at apple.com> wrote:> Thanks David. Unfortunately many of us cannot use GPL v3 gcc so it's hard > for us to investigate this. One question, can you tell if gcc is inlining > significantly more than llvm? >Evan, I have not looked at the details, so I am not sure.> We have reports that this is one of the issue plaguing eon performance. > >For C++ program, inliner can be a big differentiator. For C programs, without profile feedback, the inliner probably won't play as big a role. I looked at some small cases, llvm seems quite aggressive in inlining.> There are also some relatively well known spec optimizations that we > haven't implemented. e.g. > http://gcc.gnu.org/wiki/summit2010?action=AttachFile&do=get&target=meissner2.pdf Are > there more? >The paper talks about SPEC06 programs. I hope neither gcc nor llvm resorts too much on benchmark specific tunings .. Thanks, David> > Evan > > On Nov 13, 2010, at 12:56 PM, Xinliang David Li wrote: > > > Hi, I have looked at the LLVM code generation quality using small test > cases and in general it is better than I thought and in some cases better > than gcc. However, there are still some gap in SPEC performance. I have not > looked at the root cause of those gaps. Anyone who cares about LLVM > performance need to take this seriously. > > For fair comparison, I used -fno-strict-aliasing in gcc to turn off type > based aliasing. I also used -fomit-frame-pointer for clang/llvm because this > is not the default. The base option is -O2. Measured on 2.4GHz core-2 box. > > 64bit: > > llvm gcc > gcc vs LLVM > 164.gzip 1268 1320 4.15% > 175.vpr 1605 1534 -4.42% > 176.gcc 2203 2315 5.08% > 181.mcf 1625 1737 6.85% > 186.crafty 2411 2307 -4.30% > 197.parser 1173 1166 -0.57% > 252.eon 2245 2464 9.72% > 253.perlbmk 2214 2444 10.37% > 254.gap 1987 1978 -0.47% > 255.vortex 2497 2422 -3.00% > 256.bzip2 1585 1740 9.80% > 300.twolf 2294 2281 -0.58% > > As you can see, LLVM beats gcc in vpr, crafty and vortex by about 3%, but > falls behind gcc in gzip, gcc, mcf, eon, perfbmk, and bzip2 by a large > margin. > > > 32bit: > llvm gcc > gcc vs llvm > 164.gzip 1210 1239 2.44% > 175.vpr 1662 1621 -2.42% > 181.mcf 2733 3109 13.75% > 186.crafty 1812 1721 -5.00% > 197.parser 1328 1289 -2.92% > 253.perlbmk 2086 2580 23.67% > 254.gap 1968 1912 -2.86% > 255.vortex 1842 1965 6.66% > 256.bzip2 1440 1553 7.82% > 300.twolf 2284 2213 -3.08% > > It beats gcc in vpr, crafty, parser, gap, and twolf by ~3%, but falls > behind gcc in gzip, mcf, perlbmk, vortex, and bzip2. Some of them have very > large performance gap. eon and gcc mis-compile. > > Thanks, > > David > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101114/6c3dd075/attachment.html>
Duncan Sands
2010-Nov-18 07:14 UTC
[LLVMdev] tot clang/llvm and tot gcc performance comparision
Hi Evan,> Thanks David. Unfortunately many of us cannot use GPL v3 gcc so it's hard for us > to investigate this. One question, can you tell if gcc is inlining significantly > more than llvm? We have reports that this is one of the issue plaguing eon > performance.are you allowed to look at assembler output by GPL v3 gcc? If so, maybe someone else can do the compiling for you, and just send you results and assembler? Ciao, Duncan.