llvm dev - Nov 2012 - [LLVMdev] New benchmark in test-suite

Hi Daniel,

I'm trying to add LivermoreLoops test to the benchmark suite (tar ball
attached), but I'm getting the error below:

--- Tested: 2 tests --
FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.compile_time (1 of 2)
FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.execution_time (2 of 2)

When I use the option to only run this test:

--only-test SingleSource/Benchmarks/LivermoreLoops

I can see the binary and if I execute, it runs (still missing bits and
bobs, but ignore that for now). I've seen some files with extension
"reference_output", does that have anything to do with it?

Since it "fails" on "compile_time", I'm guessing I'm
not quite there
yet, though.

If you want to reproduce it, just add "LivermoreLoops" to the
Benchmarks' Makefile and untar the attached to it.

--
cheers,
--renato

http://systemcall.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: LivermoreLoops.tar.gz
Type: application/x-gzip
Size: 41951 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121105/f7e6c40b/attachment.bin>

Hey Renato,

You are right, the failure on compile_time indicates that the test isn't
even building. As provided, the tests don't actually define the cpuida() or
calculateMHz() functions so that seems expected to me.

The compile failures end up getting buried in the logs, but they will
either be in the test.log file in the top-level sandbox directory, or
inside the <filename>.llvm.o.compile files in the Output directory for
that
test (i.e. SingleSource/Benchmarks/LivermoreLoops/Output/...).

 - Daniel


On Mon, Nov 5, 2012 at 3:51 PM, Renato Golin <rengolin at
systemcall.org>wrote:
> Hi Daniel,
>
> I'm trying to add LivermoreLoops test to the benchmark suite (tar ball
> attached), but I'm getting the error below:
>
> --- Tested: 2 tests --
> FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.compile_time (1 of 2)
> FAIL: SingleSource/Benchmarks/LivermoreLoops/lloops.execution_time (2 of 2)
>
> When I use the option to only run this test:
>
> --only-test SingleSource/Benchmarks/LivermoreLoops
>
> I can see the binary and if I execute, it runs (still missing bits and
> bobs, but ignore that for now). I've seen some files with extension
> "reference_output", does that have anything to do with it?
>
> Since it "fails" on "compile_time", I'm guessing
I'm not quite there
> yet, though.
>
> If you want to reproduce it, just add "LivermoreLoops" to the
> Benchmarks' Makefile and untar the attached to it.
>
> --
> cheers,
> --renato
>
> http://systemcall.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20121106/7ed56972/attachment.html>

On 6 November 2012 22:34, Daniel Dunbar <daniel at zuster.org>
wrote:
> You are right, the failure on compile_time indicates that the test
isn't
> even building. As provided, the tests don't actually define the
cpuida() or
> calculateMHz() functions so that seems expected to me.
I defined both functions as NOPs.

I got what it was. The original makefile had a "-o lloops" on the
compile command and I wrongfully copied it to the LDFLAGS, so it was
generating the binary on the current dir, not on the Output dir, where
it should belong. :)

> The compile failures end up getting buried in the logs, but they will
either
> be in the test.log file in the top-level sandbox directory, or inside the
> <filename>.llvm.o.compile files in the Output directory for that test
(i.e.
> SingleSource/Benchmarks/LivermoreLoops/Output/...).
Yes, I saw those logs and they were fine (3 warnings, 0 errors), so I
got confused. Is there a place where you log the verbose Makefile
output? That would have given me a better clue as to what was
happening.

cheers,
--renato