Hi folks, As Tim pointed out, we recently had the opportunity to collect 64-bit benchmark performance data for GCC 4.9, AArch64 and ARM64 compilers on a real hardware. It is a cortex-a53 device. Due to proprietary reasons we cannot share the full hardware configuration. The preliminary results were shared at the hackers lab at EuroLLVM yesterday. For those who could not make it, below is the summarized performance data. A positive number means the ARM64 run is better by the number %. A negative number means the baseline (GCC 4.9 or AArch64) is better by the number %. Tuning of AArch64 backend on this processor has not been completely done yet (some initial work has started on modeling cortex-a53). But we quickly investigated the bad vectorized code in some of the tests (Linpack for example) and identified straightforward fixes that improved AArch64 performance (similar patches are present in ARM64, e.g. loop unroll default limit, unaligned memory accesses, etc.). These patches are going to the AArch64 commits list for review. This experiment indicates that from the point of view of correctness and performance either ARM64 or AArch64 could be the base compiler of choice if the known correctness issues (in ARM64) and lack of performance tuning (in AArch64) are addressed. However much more work has to be done to catch up with GCC 4.9 middle-end and backend optimizations. Benchmark ARM64 vs GCC 4.9 % ARM64 vs AArch64 % ARM64 vs AArch64 patched % EEMBC (no consumer) geomean -17 1 -2 EEMBC (consumer only) geomean -21 -2 -5 Linpack Double -29 45 -1 Linpack Single -51 40 1 SPEC2000 geomean -6 0 1 Thanks, Ana. -----Original Message----- From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Tim Northover Sent: Tuesday, April 08, 2014 12:04 AM To: LLVM Developers Mailing List Subject: Re: [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM Hi again, In my original message I was attempting to summarise the key arguments as I saw them. Other points came up in the discussion, which Ana kindly recorded and I'll summarise here: First, extra arguments brought up in favour of each backend (I'll mention duplicates too so that the list is as complete as possible): + Register class usage in ARM64 is cleaner. + FastISel is on ARM64, but not AArch64. Some TableGen work will be needed to enable it because of how patterns are written there. + There is no macro support in AArch64. + Both NEON syntax variants (general & iOS) are supported by ARM64 now. + ARM64 assumes neon enabled by default, and indeed has no notion that a CPU might not have NEON. Instructions will need to be predicated to check NEON is present and probably some corresponding .cpp changes where it was also assumed. + Inline asm is possibly better in ARM64. + Anecdotal evidence suggests it's easier to debug MC layer issues on ARM64 than on AArch64. Other important points that we discussed: + We need to setup a buildbot for performance using some real hardware (volunteers with hardware?) so patches can be validated in the supported targets. And also for correctness using qemu. + Google is working on a framework to build and run benchmarks – to be available soon? And should enable the buildbot setup from item above. + We need to sort out differences between cortex-a53 and Cyclone model descriptions (both use the new approach for MI scheduler, but one requires annotating instructions and the other does not). We should pin down Andy and get him to describe the perfect machine model. Cheers. Tim _______________________________________________ LLVM Developers mailing list <mailto:LLVMdev at cs.uiuc.edu> LLVMdev at cs.uiuc.edu <http://llvm.cs.uiuc.edu> http://llvm.cs.uiuc.edu <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140408/b262b628/attachment.html>
Gerolf Hoflehner
2014-Apr-08 23:46 UTC
[LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM
Hi Ana, could you share the SPEC2000 data per suite and per benchmark? Thanks Gerolf On Apr 8, 2014, at 1:33 AM, Ana Pazos <apazos at codeaurora.org> wrote:> Hi folks, > > As Tim pointed out, we recently had the opportunity to collect 64-bit benchmark performance data for GCC 4.9, AArch64 and ARM64 compilers on a real hardware. It is a cortex-a53 device. Due to proprietary reasons we cannot share the full hardware configuration. > > The preliminary results were shared at the hackers lab at EuroLLVM yesterday. For those who could not make it, below is the summarized performance data. > > A positive number means the ARM64 run is better by the number %. A negative number means the baseline (GCC 4.9 or AArch64) is better by the number %. > > Tuning of AArch64 backend on this processor has not been completely done yet (some initial work has started on modeling cortex-a53). But we quickly investigated the bad vectorized code in some of the tests (Linpack for example) and identified straightforward fixes that improved AArch64 performance (similar patches are present in ARM64, e.g. loop unroll default limit, unaligned memory accesses, etc.). These patches are going to the AArch64 commits list for review. > > This experiment indicates that from the point of view of correctness and performance either ARM64 or AArch64 could be the base compiler of choice if the known correctness issues (in ARM64) and lack of performance tuning (in AArch64) are addressed. > > However much more work has to be done to catch up with GCC 4.9 middle-end and backend optimizations. > > Benchmark > ARM64 vs GCC 4.9 % > ARM64 vs AArch64 % > ARM64 vs AArch64 patched % > EEMBC (no consumer) geomean > -17 > 1 > -2 > EEMBC (consumer only) geomean > -21 > -2 > -5 > Linpack Double > -29 > 45 > -1 > Linpack Single > -51 > 40 > 1 > SPEC2000 geomean > -6 > 0 > 1 > > Thanks, > Ana. > > > > -----Original Message----- > From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Tim Northover > Sent: Tuesday, April 08, 2014 12:04 AM > To: LLVM Developers Mailing List > Subject: Re: [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM > > Hi again, > > In my original message I was attempting to summarise the key arguments as I saw them. Other points came up in the discussion, which Ana kindly recorded and I'll summarise here: > > First, extra arguments brought up in favour of each backend (I'll mention duplicates too so that the list is as complete as possible): > > + Register class usage in ARM64 is cleaner. > + FastISel is on ARM64, but not AArch64. Some TableGen work will be > needed to enable it because of how patterns are written there. > + There is no macro support in AArch64. > + Both NEON syntax variants (general & iOS) are supported by ARM64 now. > + ARM64 assumes neon enabled by default, and indeed has no notion that > a CPU might not have NEON. Instructions will need to be predicated to check NEON is present and probably some corresponding .cpp changes where it was also assumed. > + Inline asm is possibly better in ARM64. > + Anecdotal evidence suggests it's easier to debug MC layer issues on > ARM64 than on AArch64. > > Other important points that we discussed: > > + We need to setup a buildbot for performance using some real hardware > (volunteers with hardware?) so patches can be validated in the supported targets. And also for correctness using qemu. > > + Google is working on a framework to build and run benchmarks – to be > available soon? And should enable the buildbot setup from item above. > > + We need to sort out differences between cortex-a53 and Cyclone model > descriptions (both use the new approach for MI scheduler, but one requires annotating instructions and the other does not). We should pin down Andy and get him to describe the perfect machine model. > > Cheers. > > Tim > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140408/ba8631ec/attachment.html>
Hi Gerolf, Sorry for the delayed response. I had to get permission to share more details. I am allowed to share relative numbers but not absolute numbers. Any missing test is due to runtime failures (e.g., gcc failure due to the fused multiply pattern bug which Tim fixed later on). Thanks, Ana. Benchmarks ARM64 vs GCC 4.9 % ARM64 vs AArch64 % ARM64 vs AArch64 patched % SPEC 2000 art -10 -5 -1 bzip2 -3 5 5 crafty -5 1 3 gap -8 1 2 gzip 0 4 3 mcf -2 -1 -1 mesa -15 -3 -1 parser -10 -2 4 perlbmk 5 7 5 vortex -3 -6 -4 vpr -15 -1 0 From: Gerolf Hoflehner [mailto:ghoflehner at apple.com] Sent: Tuesday, April 08, 2014 4:46 PM To: Ana Pazos Cc: Tim Northover; LLVM Developers Mailing List Subject: Re: [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM Hi Ana, could you share the SPEC2000 data per suite and per benchmark? Thanks Gerolf On Apr 8, 2014, at 1:33 AM, Ana Pazos <apazos at codeaurora.org> wrote: Hi folks, As Tim pointed out, we recently had the opportunity to collect 64-bit benchmark performance data for GCC 4.9, AArch64 and ARM64 compilers on a real hardware. It is a cortex-a53 device. Due to proprietary reasons we cannot share the full hardware configuration. The preliminary results were shared at the hackers lab at EuroLLVM yesterday. For those who could not make it, below is the summarized performance data. A positive number means the ARM64 run is better by the number %. A negative number means the baseline (GCC 4.9 or AArch64) is better by the number %. Tuning of AArch64 backend on this processor has not been completely done yet (some initial work has started on modeling cortex-a53). But we quickly investigated the bad vectorized code in some of the tests (Linpack for example) and identified straightforward fixes that improved AArch64 performance (similar patches are present in ARM64, e.g. loop unroll default limit, unaligned memory accesses, etc.). These patches are going to the AArch64 commits list for review. This experiment indicates that from the point of view of correctness and performance either ARM64 or AArch64 could be the base compiler of choice if the known correctness issues (in ARM64) and lack of performance tuning (in AArch64) are addressed. However much more work has to be done to catch up with GCC 4.9 middle-end and backend optimizations. Benchmark ARM64 vs GCC 4.9 % ARM64 vs AArch64 % ARM64 vs AArch64 patched % EEMBC (no consumer) geomean -17 1 -2 EEMBC (consumer only) geomean -21 -2 -5 Linpack Double -29 45 -1 Linpack Single -51 40 1 SPEC2000 geomean -6 0 1 Thanks, Ana. -----Original Message----- From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Tim Northover Sent: Tuesday, April 08, 2014 12:04 AM To: LLVM Developers Mailing List Subject: Re: [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM Hi again, In my original message I was attempting to summarise the key arguments as I saw them. Other points came up in the discussion, which Ana kindly recorded and I'll summarise here: First, extra arguments brought up in favour of each backend (I'll mention duplicates too so that the list is as complete as possible): + Register class usage in ARM64 is cleaner. + FastISel is on ARM64, but not AArch64. Some TableGen work will be needed to enable it because of how patterns are written there. + There is no macro support in AArch64. + Both NEON syntax variants (general & iOS) are supported by ARM64 now. + ARM64 assumes neon enabled by default, and indeed has no notion that a CPU might not have NEON. Instructions will need to be predicated to check NEON is present and probably some corresponding .cpp changes where it was also assumed. + Inline asm is possibly better in ARM64. + Anecdotal evidence suggests it's easier to debug MC layer issues on ARM64 than on AArch64. Other important points that we discussed: + We need to setup a buildbot for performance using some real hardware (volunteers with hardware?) so patches can be validated in the supported targets. And also for correctness using qemu. + Google is working on a framework to build and run benchmarks - to be available soon? And should enable the buildbot setup from item above. + We need to sort out differences between cortex-a53 and Cyclone model descriptions (both use the new approach for MI scheduler, but one requires annotating instructions and the other does not). We should pin down Andy and get him to describe the perfect machine model. Cheers. Tim _______________________________________________ LLVM Developers mailing list <mailto:LLVMdev at cs.uiuc.edu> LLVMdev at cs.uiuc.edu <http://llvm.cs.uiuc.edu/> http://llvm.cs.uiuc.edu <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev _______________________________________________ LLVM Developers mailing list <mailto:LLVMdev at cs.uiuc.edu> LLVMdev at cs.uiuc.edu <http://llvm.cs.uiuc.edu/> http://llvm.cs.uiuc.edu <http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140423/c18032c0/attachment.html>
Apparently Analagous Threads
- [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM
- [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM
- [LLVMdev] Proposal: AArch64/ARM64 merge from EuroLLVM
- [LLVMdev] question about enabling cfl-aa and collecting a57 numbers
- [LLVMdev] question about enabling cfl-aa and collecting a57 numbers