On 12 March 2013 14:24, Tim Northover <t.p.northover at gmail.com> wrote:> Could be worth a try. But if that thing really is generating random > numbers I'm not sure replacing one genuine cast-iron random number > with another is the best solution long-term. >The test is initializing srand(1), so in theory, it shouldn't be different between compilers, since Clang is using the same libraries. Also, if the "native" result is generated by GCC, than all problems go away, since the result will be target dependent (or rather, library dependent). Is there a way to turn on the dynamic generation of the native file instead of copying it from the reference_output? cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130312/c9fb4761/attachment.html>
Hi Renato,> The test is initializing srand(1), so in theory, it shouldn't be different > between compilers, since Clang is using the same libraries.If Clang and GCC disagree on the same source, same machine and with the same libraries, that certainly is odd. But it doesn't make checking against the output of a particular libc's RNG any better an idea in general. Cheers. Tim.
On 12 March 2013 14:36, Tim Northover <t.p.northover at gmail.com> wrote:> If Clang and GCC disagree on the same source, same machine and with > the same libraries, that certainly is odd.They don't. That's the odd bit. GCC and Clang agree on the output on both ARM and x86_64, and neither agree with the reference_output file. What could be happening is that the version of the libraries on that buildbot is old, and both ARM and x86_64 have been updated. I'm not suggesting we should keep replacing the "golden" file for the new value, but that we should disable checking the reference_output at all, and rely on a GCC vs. Clang comparison. I agree that the comparison is no better than a reference file (since it, too, could be wrong), but comparing both outputs eliminate any library mismatch and it's less likely that both GCC and Clang will be wrong about exactly the same thing at the same time. Is there a way to turn off the check against the reference_output and make it check against a GCC executable output? cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130312/33e6bab3/attachment.html>
On Mar 12, 2013, at 7:36 AM, Tim Northover <t.p.northover at gmail.com> wrote:> Hi Renato, > >> The test is initializing srand(1), so in theory, it shouldn't be different >> between compilers, since Clang is using the same libraries. > > If Clang and GCC disagree on the same source, same machine and with > the same libraries, that certainly is odd. But it doesn't make > checking against the output of a particular libc's RNG any better an > idea in general.I agree; I'm pretty sure that the only guarantee is that for a given implementation of stand, if you initialize it with the same seed, you get the same sequence. There is no "correct" sequence. -- Marshall Marshall Clow Idio Software <mailto:mclow.lists at gmail.com> A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- Yu Suzuki
Hi Renato, On 12/03/13 15:33, Renato Golin wrote:> On 12 March 2013 14:24, Tim Northover <t.p.northover at gmail.com > <mailto:t.p.northover at gmail.com>> wrote: > > Could be worth a try. But if that thing really is generating random > numbers I'm not sure replacing one genuine cast-iron random number > with another is the best solution long-term. > > > The test is initializing srand(1), so in theory, it shouldn't be different > between compilers, since Clang is using the same libraries. > > Also, if the "native" result is generated by GCC, than all problems go away, > since the result will be target dependent (or rather, library dependent). Is > there a way to turn on the dynamic generation of the native file instead of > copying it from the reference_output?IIRC the reference output is not used by default. You have to put USE_REFERENCE_OUTPUT := 1 in the Makefile in order to make use of the reference output. As BenchmarkGame doesn't have this, are you sure the reference output is causing the problem? Ciao, Duncan.
On 12 March 2013 16:21, Duncan Sands <baldrick at free.fr> wrote:> IIRC the reference output is not used by default. You have to put > USE_REFERENCE_OUTPUT := 1 > in the Makefile in order to make use of the reference output. As > BenchmarkGame doesn't have this, are you sure the reference output > is causing the problem? >That was my initial assumption, too. But if I just run that test, the Makefile doesn't use GCC at all and only copies the reference_output to the out-nat file. I then copied a "good" output to the reference_output, and the test passed. I'm intrigued... ;) Attached is a test.log of a local run. The buildbots' logs are pretty similar on BenchmarkGame. cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130312/74a331bf/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: test.log Type: application/octet-stream Size: 42749 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130312/74a331bf/attachment.obj>