llvm dev - Apr 2014 - [LLVMdev] Proposal: Move host CPU auto-detection out of the TargetMachine

On Thu, Apr 10, 2014 at 7:41 PM, Reid Kleckner <rnk at google.com>
wrote:
> On Thu, Apr 10, 2014 at 6:32 PM, Jim Grosbach <grosbach at apple.com>
wrote:
>>
>> It's very important that a run of "llc" on one machine
produce the same
>> output on two heterogenous machines given the same input and command
lines*.
>> That's not true right now, leading to lots of bot failures that
patch
>> originators can't reproduce because they're getting different
code locally
>> due to the auto-detection. The recent "Test failures with
3.4.1" thread for
>> examples.
>
>
> I think we should do this, but only to make llc's behavior more
> deterministic and predictable. I don't think we should start checking
in
> tests that rely on the default subtarget features without explicitly
> requesting the relevant features.
So this is largely my concern here. I hate breaking the Atom bots all
the time. That said, breaking the Atom bots all the time keeps me
aware that I want to make my tests resilient against various different
cpu types or that I should be picking explicit options to test a
particular backend feature.

-eric
>
> Consider somebody who works on an ARM or x86 variant like atom.  Probably
> what they'll want to do is set up a bot that runs the LLVM test suite
in
> such a way that their subtarget features are on by default.  Our test suite
> currently "supports" that if the host CPU features happen to be
the ones you
> want.  Instead, we should probably move to a world where bots can set
> different defaults by configuring the tools appropriately.  For example,
> they could rewrite 'llc' to 'llc -mcpu=blah' in lit if no
mcpu flags exist.
>
> This would be similar to what we do in Clang for the default C++ ABI. If
the
> test actually needs the MSVC or Itanium C++ ABI, they ask for it
explicitly.
> Otherwise they test one or the other depending on the default target
triple.
> I don't think we should double the number of RUN lines to keep that
test
> coverage, and I don't think the cost of Windows-bot-only test failures
is
> too high.
>
> 2c

On Apr 11, 2014, at 3:51 PM, Eric Christopher <echristo at gmail.com>
wrote:
> On Thu, Apr 10, 2014 at 7:41 PM, Reid Kleckner <rnk at google.com>
wrote:
>> On Thu, Apr 10, 2014 at 6:32 PM, Jim Grosbach <grosbach at
apple.com> wrote:
>>> 
>>> It's very important that a run of "llc" on one
machine produce the same
>>> output on two heterogenous machines given the same input and
command lines*.
>>> That's not true right now, leading to lots of bot failures that
patch
>>> originators can't reproduce because they're getting
different code locally
>>> due to the auto-detection. The recent "Test failures with
3.4.1" thread for
>>> examples.
>> 
>> 
>> I think we should do this, but only to make llc's behavior more
>> deterministic and predictable. I don't think we should start
checking in
>> tests that rely on the default subtarget features without explicitly
>> requesting the relevant features.
> 
> So this is largely my concern here. I hate breaking the Atom bots all
> the time. That said, breaking the Atom bots all the time keeps me
> aware that I want to make my tests resilient against various different
> cpu types or that I should be picking explicit options to test a
> particular backend feature.
> 
Won’t this make that even easier? Now those same tests will break locally for
you and you won’t have to wait for the bots to yell at you. And even when
there’s no bot at all that would have ever noticed.

Now, one additional thing we could do (outside the scope of this proposal) is
also implement a fuzzer of some kind to look for target tests that don’t have an
explicit arch/feature/whatever setting on the RUN line and run it with all
available -mcpu= values and see if it still passes (which is what the lack of
such things on the RUN line implies).

>> 
>> Consider somebody who works on an ARM or x86 variant like atom. 
Probably
>> what they'll want to do is set up a bot that runs the LLVM test
suite in
>> such a way that their subtarget features are on by default.  Our test
suite
>> currently "supports" that if the host CPU features happen to
be the ones you
>> want.  Instead, we should probably move to a world where bots can set
>> different defaults by configuring the tools appropriately.  For
example,
>> they could rewrite 'llc' to 'llc -mcpu=blah' in lit if
no mcpu flags exist.
>> 
>> This would be similar to what we do in Clang for the default C++ ABI.
If the
>> test actually needs the MSVC or Itanium C++ ABI, they ask for it
explicitly.
>> Otherwise they test one or the other depending on the default target
triple.
>> I don't think we should double the number of RUN lines to keep that
test
>> coverage, and I don't think the cost of Windows-bot-only test
failures is
>> too high.
>> 
>> 2c

On Fri, Apr 11, 2014 at 3:58 PM, Jim Grosbach <grosbach at apple.com>
wrote:
>
> Now, one additional thing we could do (outside the scope of this proposal)
> is also implement a fuzzer of some kind to look for target tests that don’t
> have an explicit arch/feature/whatever setting on the RUN line and run it
> with all available -mcpu= values and see if it still passes (which is what
> the lack of such things on the RUN line implies).

We should totally do this.  We'll could have 'check' and
'check-alltargets'
targets.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140411/989ea7a8/attachment.html>