On 19 February 2013 15:16, Arnold Schwaighofer <aschwaighofer at apple.com>wrote:> Do you have a base run with vectorization turned off? So we could see > where we are degrading things? >I wanted to, but after a few failed attempts, I couldn't pass the option to clang to disable vectorization. I don't want to make Galina reconfig the master every time, so I set up a master on my own laptop and will fiddle. But the fastest way I can test, for now, is to run the LNT tests manually with and without vectorization and compare. I'm not expecting many issues with vectorization, to be honest, but you never know... ;) When you say good results, I take it you mean successfully completing the> test, not execution time of the resulting binary? Or did you do an analysis > of performance, too? >Good results because this is the first public test-suite for ARM and we only had 19 errors out of 1104!! And 8 of them are "expected", so it's about 1% or failures. The non-EH problems should be either mechanical changes on tests, or simple fixes in LLVM, so I'm not expecting a lot of work to get the LNT on the same state on ARM than x86. I'm not checking performance yet, but the data is being collected here http://llvm.org/perf/db_default/v4/nts/machine/10 and should give us some idea on how to proceed from now on on performance measurements. For now, I'm interested in correctness, so I won't worry too much with those numbers (I've heard I should disable some Turbo mode to make more predictable results, though I only saw one test running at a time, so maybe it was off). Once we have an acceptable state (mostly green, except EH), I'll start worrying about performance. cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130219/ae655e57/attachment.html>
On Tue, Feb 19, 2013 at 7:36 AM, Renato Golin <renato.golin at linaro.org>wrote:> On 19 February 2013 15:16, Arnold Schwaighofer <aschwaighofer at apple.com>wrote: > >> Do you have a base run with vectorization turned off? So we could see >> where we are degrading things? >> > > I wanted to, but after a few failed attempts, I couldn't pass the option > to clang to disable vectorization. I don't want to make Galina reconfig the > master every time, so I set up a master on my own laptop and will fiddle. > But the fastest way I can test, for now, is to run the LNT tests manually > with and without vectorization and compare. > > I'm not expecting many issues with vectorization, to be honest, but you > never know... ;) > > > > When you say good results, I take it you mean successfully completing the >> test, not execution time of the resulting binary? Or did you do an analysis >> of performance, too? >> > > Good results because this is the first public test-suite for ARM and we > only had 19 errors out of 1104!! And 8 of them are "expected", so it's > about 1% or failures. > > The non-EH problems should be either mechanical changes on tests, or > simple fixes in LLVM, so I'm not expecting a lot of work to get the LNT on > the same state on ARM than x86. > > I'm not checking performance yet, but the data is being collected here > http://llvm.org/perf/db_default/v4/nts/machine/10 and should give us some > idea on how to proceed from now on on performance measurements. > > For now, I'm interested in correctness, so I won't worry too much with > those numbers (I've heard I should disable some Turbo mode to make more > predictable results, though I only saw one test running at a time, so maybe > it was off). >Turbo is a CPU option, not a test suite execution option. Turning it off stops the system from varying the CPU clock based on load (when this feature is enabled it can be a power saving, but it results in varying performance - bad for perf analysis). Anotehr thing to consider disabling is Address Space Layout Randomization so that you get consistent hashing & other behavior run-over-run.> Once we have an acceptable state (mostly green, except EH), I'll start > worrying about performance. >Sounds reasonable. - David -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130219/d0bf5b28/attachment.html>
Hi Renato, I'm playing with A15 bots too (running Ubuntu). This is probably what you want to have predictable performance: Disable auto-resetting the CPU scaling governor to ondemand: sudo update-rc.d -f ondemand remove Then add this to /etc/rc.local: # Disable power management. for cpu in `find /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor`; do echo performance > $cpu done On Tue, Feb 19, 2013 at 9:24 AM, David Blaikie <dblaikie at gmail.com> wrote:> > > > On Tue, Feb 19, 2013 at 7:36 AM, Renato Golin <renato.golin at linaro.org>wrote: > >> On 19 February 2013 15:16, Arnold Schwaighofer <aschwaighofer at apple.com>wrote: >> >>> Do you have a base run with vectorization turned off? So we could see >>> where we are degrading things? >>> >> >> I wanted to, but after a few failed attempts, I couldn't pass the option >> to clang to disable vectorization. I don't want to make Galina reconfig the >> master every time, so I set up a master on my own laptop and will fiddle. >> But the fastest way I can test, for now, is to run the LNT tests manually >> with and without vectorization and compare. >> >> I'm not expecting many issues with vectorization, to be honest, but you >> never know... ;) >> >> >> >> When you say good results, I take it you mean successfully completing the >>> test, not execution time of the resulting binary? Or did you do an analysis >>> of performance, too? >>> >> >> Good results because this is the first public test-suite for ARM and we >> only had 19 errors out of 1104!! And 8 of them are "expected", so it's >> about 1% or failures. >> >> The non-EH problems should be either mechanical changes on tests, or >> simple fixes in LLVM, so I'm not expecting a lot of work to get the LNT on >> the same state on ARM than x86. >> >> I'm not checking performance yet, but the data is being collected here >> http://llvm.org/perf/db_default/v4/nts/machine/10 and should give us >> some idea on how to proceed from now on on performance measurements. >> >> For now, I'm interested in correctness, so I won't worry too much with >> those numbers (I've heard I should disable some Turbo mode to make more >> predictable results, though I only saw one test running at a time, so maybe >> it was off). >> > > Turbo is a CPU option, not a test suite execution option. Turning it off > stops the system from varying the CPU clock based on load (when this > feature is enabled it can be a power saving, but it results in varying > performance - bad for perf analysis). > > Anotehr thing to consider disabling is Address Space Layout Randomization > so that you get consistent hashing & other behavior run-over-run. > > >> Once we have an acceptable state (mostly green, except EH), I'll start >> worrying about performance. >> > > Sounds reasonable. > > - David > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130219/3945078e/attachment.html>
On 19 February 2013 17:24, David Blaikie <dblaikie at gmail.com> wrote:> Anotehr thing to consider disabling is Address Space Layout Randomization > so that you get consistent hashing & other behavior run-over-run. >Good idea, done, thanks! --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130219/c04e6e68/attachment.html>