On 7 January 2013 14:37, Duncan Sands <baldrick at free.fr> wrote:> sorry, what change do you plan to make? Did you work out what the bug is? > My > basic worry is that it sounds like you are trying to hide the underlying > issue > rather than fixing it, please correct me if I'm wrong.Hi Duncan, There are two issues: 1. The LTO bug we found by running Livermore Loops on test-suite. I'm still trying to isolate this and will report as soon as I get a smaller test case. (I sent a tarball earlier to the list on how to reproduce it). Bugs like these will not be caught any more with the standard LNT tests, true, but there's also point 2 below... 2. Buildbots with multiple test builders, confusing and generating too much noise. LNT is not testing LTO at the moment, but David said there someone working on it right now. So, the way to go would be to have LNT on all buildbots (in the long run) testing with and without LTO (and possibly other variations), so we can have a coherent story and an easy way to reproduce errors locally. To test you need to set up your own build master. It's not that hard.>That's a good point. I'll do that. cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130107/de498b31/attachment.html>
We are testing LTO internally and have not run into this issue IIRC. But on the other hand, we are doing a straight compilation (I.e. not doing it in parts as you said you were). Even so, you are right, we should have public lnt LTO testers. On Jan 7, 2013, at 7:27 AM, Renato Golin <renato.golin at linaro.org> wrote:> On 7 January 2013 14:37, Duncan Sands <baldrick at free.fr> wrote: >> sorry, what change do you plan to make? Did you work out what the bug is? My >> basic worry is that it sounds like you are trying to hide the underlying issue >> rather than fixing it, please correct me if I'm wrong. > > Hi Duncan, > > There are two issues: > > 1. The LTO bug we found by running Livermore Loops on test-suite. I'm still trying to isolate this and will report as soon as I get a smaller test case. (I sent a tarball earlier to the list on how to reproduce it). Bugs like these will not be caught any more with the standard LNT tests, true, but there's also point 2 below...ping > > 2. Buildbots with multiple test builders, confusing and generating too much noise. LNT is not testing LTO at the moment, but David said there someone working on it right now. So, the way to go would be to have LNT on all buildbots (in the long run) testing with and without LTO (and possibly other variations), so we can have. a coherent story and an easy way to reproduce errors locally. > > >> To test you need to set up your own build master. It's not that hard. > > That's a good point. I'll do that. > > cheers, > --renato-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130107/b63d452f/attachment.html>
On 7 January 2013 15:32, Michael Gottesman <mgottesman at apple.com> wrote:> We are testing LTO internally and have not run into this issue IIRC. But > on the other hand, we are doing a straight compilation (I.e. not doing it > in parts as you said you were). Even so, you are right, we should have > public lnt LTO testers. >My opinion is that tests should be intentional. If you spot a difference between two calls, you either start testing both explicitly or ignore one of them. Relying on side-effects for testing, in the majority of cases, increase the perceived importance of small matters and takes away time to fix real bugs. If LTO is important (I think it is), then we should have explicit LTO tests. If testing the order of passes is important, we should consistently test it on all important configurations we have (ex. using bugpoint). Keeping an old testing style *just* to have the side-effect of testing LTO leads to confusion and noise which is worse in the end. Random tests are one way of achieving a huge vector space in a fair way. The hard bits is to know what to ignore (ie. it'll never happen in real world) against the real bugs, that need fixing, or the real bugs that have very little importance, etc. But all that should only be pursued when all the other proper tests are set up and giving meaningful results. cheers, --renato -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130107/8778aba1/attachment.html>