llvm dev - Sep 2016 - Benchmarks for LLVM-generated Binaries

> On 2 Sep 2016, at 11:19, Renato Golin <renato.golin at linaro.org>
wrote:
> 
> On 2 September 2016 at 02:13, Dean Michael Berris <dean.berris at
gmail.com> wrote:
>> I think it should be possible to have a snapshot of it included. I
don't know what the licensing implications are (I'm not a lawyer, but I
know someone who is -- paging Danny Berlin).
> 
> The test-suite has a very large number of licenses (compared to LLVM),
> so licensing should be less of a problem there. Though Dan can help
> more than I can. :)
> 
Cool, let's wait for what Danny thinks on the patch I'll be preparing.
:)
> 
>> I'm not as concerned about falling behind on versions there though
mostly because it should be trivial to update it if we need it. Though like you,
I agree this isn't the best way of doing it. :)
> 
> If we start using it more (maybe we should, at least for the
> benchmarks, I've been long wanting to do something decent there), then
> we'd need to add a proper update procedure.
> 
> I'm fine with some checkout if it's a stable release, not trunk, as
it
> would make things a lot easier to update later (patch releases, new
> releases, etc).
> 
SGTM.
> 
> 
>> Thanks -- this doesn't tell me how to run the test though... I
could certainly do it by hand (i.e. build the executables and run it) and I
suspect I'm not alone in wanting to be able to do this easily through the
CMake+Ninja (or other generator) workflow.
> 
> Ah, no, that helped you adding your test. :)
> 
> 
>> Do you know if someone is working on that aspect?
> 
> http://llvm.org/docs/lnt/quickstart.html
> 
> This is *exactly* what Perf (the monitoring website) does, so you're
> sure to get the same result on both sides if you run it locally like
> that. I do.
> 
Ah, cool. That works for me. :)
> You can choose to run down to a specific test/benchmark, so it's quick
> and easy to use while developing, too.
> 
Awesome stuff, thanks Renato!

-- Dean

I'm working on this now, and I had a few more questions below for Renato and
the list in general. Please see inline below.
> On 2 Sep 2016, at 11:27, Dean Michael Berris <dean.berris at
gmail.com> wrote:
> 
>> On 2 Sep 2016, at 11:19, Renato Golin <renato.golin at
linaro.org> wrote:
>> 
>> On 2 September 2016 at 02:13, Dean Michael Berris <dean.berris at
gmail.com> wrote:
>>> I think it should be possible to have a snapshot of it included. I
don't know what the licensing implications are (I'm not a lawyer, but I
know someone who is -- paging Danny Berlin).
>> 
>> The test-suite has a very large number of licenses (compared to LLVM),
>> so licensing should be less of a problem there. Though Dan can help
>> more than I can. :)
>> 
> 
> Cool, let's wait for what Danny thinks on the patch I'll be
preparing. :)
> 
>> 
>>> I'm not as concerned about falling behind on versions there
though mostly because it should be trivial to update it if we need it. Though
like you, I agree this isn't the best way of doing it. :)
>> 
>> If we start using it more (maybe we should, at least for the
>> benchmarks, I've been long wanting to do something decent there),
then
>> we'd need to add a proper update procedure.
>> 
>> I'm fine with some checkout if it's a stable release, not
trunk, as it
>> would make things a lot easier to update later (patch releases, new
>> releases, etc).
>> 
> 
> SGTM.
> 
Is there a preference on where to place the library? I had a look at
{SingleSource/MultiSource}/Benchmarks/ and I didn't find a common location
for libraries used. I'm tempted to create a top-level "libs"
directory that will host common libraries but I'm also fine with just having
the benchmark library living alongside the XRay benchmarks.

So two options here:

1) libs/googlebenchmark/
2) MultiSource/Benchmarks/XRay/googlebench/

Thoughts?

Cheers

-- Dean

> On Sep 14, 2016, at 12:50 AM, Dean Michael Berris via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
> 
> I'm working on this now, and I had a few more questions below for
Renato and the list in general. Please see inline below.
> 
>> On 2 Sep 2016, at 11:27, Dean Michael Berris <dean.berris at
gmail.com> wrote:
>> 
>>> On 2 Sep 2016, at 11:19, Renato Golin <renato.golin at
linaro.org> wrote:
>>> 
>>> On 2 September 2016 at 02:13, Dean Michael Berris <dean.berris
at gmail.com> wrote:
>>>> I think it should be possible to have a snapshot of it
included. I don't know what the licensing implications are (I'm not a
lawyer, but I know someone who is -- paging Danny Berlin).
>>> 
>>> The test-suite has a very large number of licenses (compared to
LLVM),
>>> so licensing should be less of a problem there. Though Dan can help
>>> more than I can. :)
>>> 
>> 
>> Cool, let's wait for what Danny thinks on the patch I'll be
preparing. :)
>> 
>>> 
>>>> I'm not as concerned about falling behind on versions there
though mostly because it should be trivial to update it if we need it. Though
like you, I agree this isn't the best way of doing it. :)
>>> 
>>> If we start using it more (maybe we should, at least for the
>>> benchmarks, I've been long wanting to do something decent
there), then
>>> we'd need to add a proper update procedure.
>>> 
>>> I'm fine with some checkout if it's a stable release, not
trunk, as it
>>> would make things a lot easier to update later (patch releases, new
>>> releases, etc).
>>> 
>> 
>> SGTM.
>> 
> 
> Is there a preference on where to place the library? I had a look at
{SingleSource/MultiSource}/Benchmarks/ and I didn't find a common location
for libraries used. I'm tempted to create a top-level "libs"
directory that will host common libraries but I'm also fine with just having
the benchmark library living alongside the XRay benchmarks.
> 
> So two options here:
> 
> 1) libs/googlebenchmark/
> 2) MultiSource/Benchmarks/XRay/googlebench/
This is something that may be used (or is intended) to be used by others in the
future, the first option makes it easier (or encouraging at least).

—
Mehdi

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160914/02cd0908/attachment.html>