llvm dev - Apr 2014 - [LLVMdev] Proposal: Move host CPU auto-detection out of the TargetMachine

All,

Currently the X86 backend does CPU auto-detection and subtarget feature
detection when the TargetMachine is created if no explicit CPU was specified.
It's counterintuitive for low level tools like ‘llc’ to do this, as it means
the same .ll file compiled on heterogenous machines generates different results
from the same ‘llc’ command line. It is still useful to be able to opt-in to
such behavior to, for example, replicate clang’s behavior when -mcpu=native is
supplied to clang. My thought is to do something similar here and teach ‘llc’ to
recognize -mcpu=native and probe the host CPU if that is given. The subtarget
features will then be filled in according to the feature string for that CPU.
This (a) changes the auto-detection from opt-out to opt-in and (b) moves the
logic out of the core target backend and into the tools drivers.

Attached are draft patches that do this for X86. Similar but smaller cleanups
can also be done for SystemZ and PowerPC if it’s agreed this is a good idea.

-Jim


-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-X86-Remove-TargetMachine-CPU-auto-detection.patch
Type: application/octet-stream
Size: 12006 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140407/9f62b25c/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-llc-Add-support-for-mcpu-native.patch
Type: application/octet-stream
Size: 1136 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140407/9f62b25c/attachment-0001.obj>

I support the general direction of this change.  I haven't reviewed the 
actual code changes.

Philip

On 04/07/2014 05:59 PM, Jim Grosbach wrote:
> All,
>
> Currently the X86 backend does CPU auto-detection and subtarget feature
detection when the TargetMachine is created if no explicit CPU was specified.
It's counterintuitive for low level tools like 'llc' to do this, as
it means the same .ll file compiled on heterogenous machines generates different
results from the same 'llc' command line. It is still useful to be able
to opt-in to such behavior to, for example, replicate clang's behavior when
-mcpu=native is supplied to clang. My thought is to do something similar here
and teach 'llc' to recognize -mcpu=native and probe the host CPU if that
is given. The subtarget features will then be filled in according to the feature
string for that CPU. This (a) changes the auto-detection from opt-out to opt-in
and (b) moves the logic out of the core target backend and into the tools
drivers.
>
> Attached are draft patches that do this for X86. Similar but smaller
cleanups can also be done for SystemZ and PowerPC if it's agreed this is a
good idea.
>
> -Jim
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140409/cd425d7e/attachment.html>

I'm not a huge fan of this because then you get to decide on a default
for all the ports, but I can understand if people want to move this
way to reduce uncertainty.

Alternately have a way for the backend to pretty print out the results
of the auto-detection when asked?

*shrug*

-eric

On Tue, Apr 8, 2014 at 1:59 AM, Jim Grosbach <grosbach at apple.com>
wrote:
> All,
>
> Currently the X86 backend does CPU auto-detection and subtarget feature
detection when the TargetMachine is created if no explicit CPU was specified.
It's counterintuitive for low level tools like 'llc' to do this, as
it means the same .ll file compiled on heterogenous machines generates different
results from the same 'llc' command line. It is still useful to be able
to opt-in to such behavior to, for example, replicate clang's behavior when
-mcpu=native is supplied to clang. My thought is to do something similar here
and teach 'llc' to recognize -mcpu=native and probe the host CPU if that
is given. The subtarget features will then be filled in according to the feature
string for that CPU. This (a) changes the auto-detection from opt-out to opt-in
and (b) moves the logic out of the core target backend and into the tools
drivers.
>
> Attached are draft patches that do this for X86. Similar but smaller
cleanups can also be done for SystemZ and PowerPC if it's agreed this is a
good idea.
>
> -Jim
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

Eric Christopher <echristo at gmail.com> writes:
> I'm not a huge fan of this because then you get to decide on a default
> for all the ports, but I can understand if people want to move this
> way to reduce uncertainty.
FWIW, since it's one of the three targets Jim mentioned, -march=z10 is
the obvious default for SystemZ.

On the other hand, I think it's good to run the tests on both z10 and z196,
since there were significant additions to the architecture in z196.
Until now we've been getting that by implicitly using -mcpu=z10 for
cross testing on the usual x86_64 buildbots and implicitly using
-mcpu=native (-mcpu=z196) on the z buildbot.

We could do it instead by having two llcs and FileChecks per
CodeGen/SystemZ test, which would give better cross-toolchain
coverage, but would almost double the test time.  Would that
still be OK?

Thanks,
Richard

We already decide on a default. For these three targets is “whatever the user is
running on.” For all the other targets it’s something the backend selects. I’m
proposing we, as you say, remove the uncertainty and have the default always be
the same from one run to the next even (especially) when those runs are on
different machines.

For X86, there’s two options, the “generic” target that backend already supports
or “core2” which is what clang typically defaults to for x86_64h. I lean towards
the former for simplicity. We also don’t have to change all three of the ports.
I’m personally motivated primarily by x86, and was intending to leave it to the
maintainers of the other two targets to decide what they want to do here,
perhaps with some strong encouragement.

It’s very important that a run of “llc” on one machine produce the same output
on two heterogenous machines given the same input and command lines*. That’s not
true right now, leading to lots of bot failures that patch originators can’t
reproduce because they’re getting different code locally due to the
auto-detection. The recent “Test failures with 3.4.1” thread for examples.

-Jim

* With the caveat of platform differences like elf vs. macho. A similar argument
can be made there, but unlike CPU, there’s no sensible way to define a
least-common-denominator default, so it’s harder. I’m explicitly avoiding
opening that can of worms.

On Apr 9, 2014, at 10:19 PM, Eric Christopher <echristo at gmail.com>
wrote:
> I'm not a huge fan of this because then you get to decide on a default
> for all the ports, but I can understand if people want to move this
> way to reduce uncertainty.
> 
> Alternately have a way for the backend to pretty print out the results
> of the auto-detection when asked?
> 
> *shrug*
> 
> -eric
> 
> On Tue, Apr 8, 2014 at 1:59 AM, Jim Grosbach <grosbach at apple.com>
wrote:
>> All,
>> 
>> Currently the X86 backend does CPU auto-detection and subtarget feature
detection when the TargetMachine is created if no explicit CPU was specified.
It's counterintuitive for low level tools like 'llc' to do this, as
it means the same .ll file compiled on heterogenous machines generates different
results from the same 'llc' command line. It is still useful to be able
to opt-in to such behavior to, for example, replicate clang's behavior when
-mcpu=native is supplied to clang. My thought is to do something similar here
and teach 'llc' to recognize -mcpu=native and probe the host CPU if that
is given. The subtarget features will then be filled in according to the feature
string for that CPU. This (a) changes the auto-detection from opt-out to opt-in
and (b) moves the logic out of the core target backend and into the tools
drivers.
>> 
>> Attached are draft patches that do this for X86. Similar but smaller
cleanups can also be done for SystemZ and PowerPC if it's agreed this is a
good idea.
>> 
>> -Jim
>> 
>> 
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>