thr3ads.net - llvm dev - [LLVMdev] Set up ExecutionEngine according to actual machine capabilities [May 2015]

If this information is useful, please help other people find it:
Share via:

Mueller-Roemer, Johannes Sebastian

2015-May-11 08:59 UTC

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

I am currently setting up my Module with

module->setTargetTriple(llvm::sys::getProcessTriple()
#ifdef _WIN32
                               + "-elf"
#endif
);

And my ExecutionEngine with

llvm::EngineBuilder(std::move(module))
                                               .setErrorStr(&err)
                                              
.setMCPU(llvm::sys::getHostCPUName())
                                               .create()

This works fine on most machines, however on some virtualized machines this
fails because the host CPU name implies AVX, however AVX is in fact disabled,
leading to an illegal instruction when running JIT-compiled functions.

Is there a better way to set up the executionengine so that such failure do not
occur (but all available features are used)?

--
Johannes S. Mueller-Roemer, MSc
Wiss. Mitarbeiter - Interactive Engineering Technologies (IET)

Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
Tel +49 6151 155-606  |  Fax +49 6151 155-139
johannes.mueller-roemer at igd.fraunhofer.de  |  www.igd.fraunhofer.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150511/00b4db0b/attachment.html>

Yaron Keren

2015-May-11 09:17 UTC

head link

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

That's odd. getHostCPUName uses cpuid to get the actual capabilities of the
CPU before selecting the name.
Does cpuid reports incorrect information in these virtual machines?

In any case you can further tune CPU capabilities with
EngineBuilder::setMAttrs.


2015-05-11 11:59 GMT+03:00 Mueller-Roemer, Johannes Sebastian <
Johannes.Sebastian.Mueller-Roemer at igd.fraunhofer.de>:
>  I am currently setting up my Module with
>
>
>
> module->setTargetTriple(llvm::sys::getProcessTriple()
>
> #ifdef _WIN32
>
>                                + "-elf"
>
> #endif
>
> );
>
>
>
> And my ExecutionEngine with
>
>
>
> llvm::EngineBuilder(std::move(module))
>
>                                                .setErrorStr(&err)
>
>
> .setMCPU(llvm::sys::getHostCPUName())
>
>                                                .create()
>
>
>
> This works fine on most machines, however on some virtualized machines
> this fails because the host CPU name implies AVX, however AVX is in fact
> disabled, leading to an illegal instruction when running JIT-compiled
> functions.
>
>
>
> Is there a better way to set up the executionengine so that such failure
> do not occur (but all available features are used)?
>
>
>
> --
>
> Johannes S. Mueller-Roemer, MSc
>
> Wiss. Mitarbeiter - Interactive Engineering Technologies (IET)
>
>
>
> Fraunhofer-Institut für Graphische Datenverarbeitung IGD
>
> Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
>
> Tel +49 6151 155-606  |  Fax +49 6151 155-139
>
> johannes.mueller-roemer at igd.fraunhofer.de  |  www.igd.fraunhofer.de
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150511/3d7a8729/attachment.html>

Mueller-Roemer, Johannes Sebastian

2015-May-11 09:21 UTC

head link

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

It has no AVX according to CPUz (which I assume uses cpuid?), but it reports as
an E5-2680, which normally does have AVX.

--
Johannes S. Mueller-Roemer, MSc
Wiss. Mitarbeiter - Interactive Engineering Technologies (IET)

Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
Tel +49 6151 155-606  |  Fax +49 6151 155-139
johannes.mueller-roemer at igd.fraunhofer.de  |  www.igd.fraunhofer.de

From: Yaron Keren [mailto:yaron.keren at gmail.com]
Sent: Monday, May 11, 2015 11:18
To: Mueller-Roemer, Johannes Sebastian
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] Set up ExecutionEngine according to actual machine
capabilities

That's odd. getHostCPUName uses cpuid to get the actual capabilities of the
CPU before selecting the name.
Does cpuid reports incorrect information in these virtual machines?

In any case you can further tune CPU capabilities with EngineBuilder::setMAttrs.


2015-05-11 11:59 GMT+03:00 Mueller-Roemer, Johannes Sebastian
<Johannes.Sebastian.Mueller-Roemer at
igd.fraunhofer.de<mailto:Johannes.Sebastian.Mueller-Roemer at
igd.fraunhofer.de>>:
I am currently setting up my Module with

module->setTargetTriple(llvm::sys::getProcessTriple()
#ifdef _WIN32
                               + "-elf"
#endif
);

And my ExecutionEngine with

llvm::EngineBuilder(std::move(module))
                                               .setErrorStr(&err)
                                              
.setMCPU(llvm::sys::getHostCPUName())
                                               .create()

This works fine on most machines, however on some virtualized machines this
fails because the host CPU name implies AVX, however AVX is in fact disabled,
leading to an illegal instruction when running JIT-compiled functions.

Is there a better way to set up the executionengine so that such failure do not
occur (but all available features are used)?

--
Johannes S. Mueller-Roemer, MSc
Wiss. Mitarbeiter - Interactive Engineering Technologies (IET)

Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
Tel +49 6151 155-606  |  Fax +49 6151 155-139
johannes.mueller-roemer at igd.fraunhofer.de<mailto:johannes.mueller-roemer
at igd.fraunhofer.de>  | 
www.igd.fraunhofer.de<http://www.igd.fraunhofer.de>


_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu<mailto:LLVMdev at cs.uiuc.edu>        
http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150511/3faecd98/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: e5-2680-no-avx.png
Type: image/png
Size: 20716 bytes
Desc: e5-2680-no-avx.png
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150511/3faecd98/attachment.png>

Benjamin Kramer

2015-May-11 09:23 UTC

head link

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

On Mon, May 11, 2015 at 10:59 AM, Mueller-Roemer, Johannes Sebastian
<Johannes.Sebastian.Mueller-Roemer at igd.fraunhofer.de>
wrote:> I am currently setting up my Module with
>
>
>
> module->setTargetTriple(llvm::sys::getProcessTriple()
>
> #ifdef _WIN32
>
>                                + "-elf"
>
> #endif
>
> );
>
>
>
> And my ExecutionEngine with
>
>
>
> llvm::EngineBuilder(std::move(module))
>
>                                                .setErrorStr(&err)
>
>
> .setMCPU(llvm::sys::getHostCPUName())
>
>                                                .create()
>
>
>
> This works fine on most machines, however on some virtualized machines this
> fails because the host CPU name implies AVX, however AVX is in fact
> disabled, leading to an illegal instruction when running JIT-compiled
> functions.
>
>
>
> Is there a better way to set up the executionengine so that such failure do
> not occur (but all available features are used)?
The proper way to do this in LLVM trunk is to use
sys::getHostCPUFeatures instead of the name and set the individual
feature bits via setMAttrs. getHostCPUName used to be aware of
AVX-disabled environments but that was a bit of a hack and was removed
recently.

- Ben

Mueller-Roemer, Johannes Sebastian

2015-May-11 11:38 UTC

head link

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

Ok I now have something like the following, which appears to work (at least it
doesn't crash)

auto host_features = llvm::StringMap<bool>{};
if(!llvm::sys::getHostCPUFeatures(host_features))
	throw std::runtime_error("could not retrieve host CPU features");
auto host_attrs = llvm::SmallVector<llvm::StringRef, 16>{};
for(auto const & pair : host_features)
	if(pair.second)
		host_attrs.emplace_back(pair.first());
auto ee = std::unique_ptr<llvm::ExecutionEngine>{
	llvm::EngineBuilder(std::move(module))
		.setErrorStr(&err)
		.setMAttrs(host_attrs)
		.create()
};

Too bad setMattrs doesn't take a StringMap directly

--
Johannes S. Mueller-Roemer, MSc
Wiss. Mitarbeiter - Interactive Engineering Technologies (IET)

Fraunhofer-Institut für Graphische Datenverarbeitung IGD
Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
Tel +49 6151 155-606  |  Fax +49 6151 155-139
johannes.mueller-roemer at igd.fraunhofer.de  |  www.igd.fraunhofer.de


-----Original Message-----
From: Benjamin Kramer [mailto:benny.kra at gmail.com] 
Sent: Monday, May 11, 2015 11:24
To: Mueller-Roemer, Johannes Sebastian
Cc: llvmdev at cs.uiuc.edu
Subject: Re: [LLVMdev] Set up ExecutionEngine according to actual machine
capabilities

On Mon, May 11, 2015 at 10:59 AM, Mueller-Roemer, Johannes Sebastian
<Johannes.Sebastian.Mueller-Roemer at igd.fraunhofer.de>
wrote:> I am currently setting up my Module with
>
>
>
> module->setTargetTriple(llvm::sys::getProcessTriple()
>
> #ifdef _WIN32
>
>                                + "-elf"
>
> #endif
>
> );
>
>
>
> And my ExecutionEngine with
>
>
>
> llvm::EngineBuilder(std::move(module))
>
>                                                .setErrorStr(&err)
>
>
> .setMCPU(llvm::sys::getHostCPUName())
>
>                                                .create()
>
>
>
> This works fine on most machines, however on some virtualized machines 
> this fails because the host CPU name implies AVX, however AVX is in 
> fact disabled, leading to an illegal instruction when running 
> JIT-compiled functions.
>
>
>
> Is there a better way to set up the executionengine so that such 
> failure do not occur (but all available features are used)?
The proper way to do this in LLVM trunk is to use sys::getHostCPUFeatures
instead of the name and set the individual feature bits via setMAttrs.
getHostCPUName used to be aware of AVX-disabled environments but that was a bit
of a hack and was removed recently.

- Ben

llvm dev - May 2015 - [LLVMdev] Set up ExecutionEngine according to actual machine capabilities

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities

[LLVMdev] Set up ExecutionEngine according to actual machine capabilities