llvm dev - Jun 2016 - FireFox-46.0.1 build with interprocedural register allocation enabled

On Mon, Jun 20, 2016 at 4:14 AM, Sanjoy Das <sanjoy at
playingwithpointers.com>
wrote:
> Hi Vivek,
>
> [Dropping firefox-dev, since I don't want to spam them]
>
> vivek pandya via llvm-dev wrote:
> > Measure W/O IPRAWITH IPRA
> > =======================> > Total Build Time76 mins82.3 mins8%
increment
> > Octane v2.0 JS Benchmark Score (higher is better)18675.69 19665.165%
> improvement
> > Kraken JS Benchmark time (lower is better)1416.2 ms 1421.3 ms 0.35%
> regression
> > JetStream JS Benchmark Score (higer is better)110.10112.882.52%
> improvement
>
> This is great!
>
> Do you have a sense of how much noise these benchmarks have?  For
> instance, if you run the benchmark 10 times, what is standard
> deviation in the numbers?
>
Hello Sanjoy,

For Octane and Kraken I have run them 4 times and above result is geometric
mean. For Octane standard deviation (SD) is 918.54 (NO_IPRA) and 597.82
(With_IPRA). For Kraken unfortunately I don't have readings any more but
there was very minor change in each run. JetStream it self runs the test
for 3 times reports the result. From next time onwards I will run test at
least for 10 times
>
> It'd also be great to have a precise, non-measurement oriented view of
> the benchmarks.
I don't understand this point actually all these benchmarks are suggested
by firefox-devs and they wanted the results back.
> Is it possible to collect some statistics on how much
> you've improved the register allocation in these workloads?
Actually while testing single source test case  with debug build I used to
compare results of -stats with regalloc keyword but while building such a
huge software I prefer release build of llvm. And also I don't know if
there is any way in llvm to generate stats for the whole build.
> Perhaps
> you could count the instances where you preserved a register over a
> call site that wouldn't have been possible without IPRA?
>
Yes this seems a good idea and I think this is implementable, after
calculating new regmask when inserting data into immutable pass two
regmasks can be compared to calculate improvements. I will work on this and
let you know the progress.
>
> Thanks!
> -- Sanjoy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160620/9872cb64/attachment.html>

Hi Vivek,

vivek pandya wrote:

 > For Octane and Kraken I have run them 4 times and above result is
 > geometric mean. For Octane standard deviation (SD) is
 > 918.54 (NO_IPRA) and 597.82 (With_IPRA). For Kraken unfortunately I
 > don't have readings any more but there was very
 > minor change in each run. JetStream it self runs the test for 3
 > times reports the result. From next time onwards I will
 > run test at least for 10 times

Oh, no, I used "10 times" as an anecdotal number.  Geomean of 4 times
is enough.  Of course, if it does not take extra effort, running them
for more iterations is better, but don't bother if it will e.g. take
more than a trivial amount of manual work.

 >     It'd also be great to have a precise, non-measurement oriented
view of
 >     the benchmarks.
 >
 > I don't understand this point actually all these benchmarks are
 > suggested by firefox-devs and they wanted the results back.

I was talking about the `-stats` bit there ^.

 >     Is it possible to collect some statistics on how much
 >     you've improved the register allocation in these workloads?
 >
 > Actually while testing single source test case with debug build I
 > used to compare results of -stats with regalloc
 > keyword but while building such a huge software I prefer release
 > build of llvm. And also I don't know if there is any
 > way in llvm to generate stats for the whole build.

Can you do something quick and dirty, like just have some local
changes that dumps out some information on outs() ?

 >     Perhaps
 >     you could count the instances where you preserved a register over a
 >     call site that wouldn't have been possible without IPRA?
 >
 > Yes this seems a good idea and I think this is implementable, after
 > calculating new regmask when inserting data into
 > immutable pass two regmasks can be compared to calculate
 > improvements. I will work on this and let you know the progress.

Just to be clear: you don't *have* to do this (specifically, check
with your mentor before sinking too much time into this).  But if we
can come up with easy to "confirm your kill" (i.e. ensure what you
think should happen is what is actually happening) then I think we
should do it.

Thanks, and best of luck!
-- Sanjoy

On Mon, Jun 20, 2016 at 10:05 PM, Sanjoy Das <sanjoy at
playingwithpointers.com
> wrote:
> Hi Vivek,
>
> vivek pandya wrote:
>
> > For Octane and Kraken I have run them 4 times and above result is
> > geometric mean. For Octane standard deviation (SD) is
> > 918.54 (NO_IPRA) and 597.82 (With_IPRA). For Kraken unfortunately I
> > don't have readings any more but there was very
> > minor change in each run. JetStream it self runs the test for 3
> > times reports the result. From next time onwards I will
> > run test at least for 10 times
>
> Oh, no, I used "10 times" as an anecdotal number.  Geomean of 4
times
> is enough.  Of course, if it does not take extra effort, running them
> for more iterations is better, but don't bother if it will e.g. take
> more than a trivial amount of manual work.
>
> >     It'd also be great to have a precise, non-measurement oriented
view
> of
> >     the benchmarks.
> >
> > I don't understand this point actually all these benchmarks are
> > suggested by firefox-devs and they wanted the results back.
>
> I was talking about the `-stats` bit there ^.
>
> >     Is it possible to collect some statistics on how much
> >     you've improved the register allocation in these workloads?
> >
> > Actually while testing single source test case with debug build I
> > used to compare results of -stats with regalloc
> > keyword but while building such a huge software I prefer release
> > build of llvm. And also I don't know if there is any
> > way in llvm to generate stats for the whole build.
>
> Can you do something quick and dirty, like just have some local
> changes that dumps out some information on outs() ?
Yes that should not take too much of time.
>
>
> >     Perhaps
> >     you could count the instances where you preserved a register over
a
> >     call site that wouldn't have been possible without IPRA?
> >
> > Yes this seems a good idea and I think this is implementable, after
> > calculating new regmask when inserting data into
> > immutable pass two regmasks can be compared to calculate
> > improvements. I will work on this and let you know the progress.
>
> Just to be clear: you don't *have* to do this (specifically, check
> with your mentor before sinking too much time into this).  But if we
> can come up with easy to "confirm your kill" (i.e. ensure what
you
> think should happen is what is actually happening) then I think we
> should do it.
>
I will try if this can be done in a simple way. I am sure mentors will
welcome it.
-Vivek
>
> Thanks, and best of luck!
> -- Sanjoy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160621/8445b5b5/attachment.html>