What is the preferred method for compiler (frontend) developers to optimize and generate target machine code from IR? At one point I found a tutorial that recommended simply dumping the IR to a file and spawning llc to do the job. Up until now I have "manually" created a TargetMachine, PassManager, etc. to generate my object code. The initial version of my code was cribbed from llc for LLVM 3.2, and has since been updated for 3.5.1. However, with every new release of LLVM, the API to the backend optimization and code generation passes changes. The changes from 3.5.1 to 3.6 are quite significant. If I'm lucky, the impact of a change is that my C++ code refuses to compile, and I have to fix. If I'm unlucky, the impact may be that my code compiles and runs, but LLVM works suboptimally - perhaps some optimizations don't happen. (This can happen if a newer API expects me to do some step which was not required in earlier releases, and there is no assert to catch it.) As an alternative, I am seriously considering "simply dumping the IR to a file and spawning llc" to perform my backend work. The API to create IR is much more stable than the API to do useful things with it. Furthermore, it's a lot easier to manually debug IR that has been dumped to a file. Finally, I can spawn multiple, independent, concurrent invocations of llc on a multi-core machine. I needn't worry about concurrency, as the standard Linux fork/waitpid type calls will suffice. Given that 90% of my runtime is spent inside LLVM, I get 90% of the benefit of a fully concurrent design with almost zero work. But now it seems that this usage model is frowned upon. What is the recommended usage model? On Tue, Mar 17, 2015 at 2:32 PM, Owen Anderson <resistor at mac.com> wrote:> > On Mar 17, 2015, at 11:13 AM, Dario Domizioli <dario.domizioli at gmail.com> > wrote: > > What does the community think? > Discuss. :-) > > > Devil’s advocate: opt, llc, lli, etc. are development/debugging tools for > LLVM developers, not for end users, and the project optimizes their > functionality for that use case. > > —Owen > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150317/f69caa0b/attachment.html>
Thanks for all the replies!
Whoa, it looks like there's pretty much a massive consensus on "use
clang
and never use opt/llc in that kind of scenario".
I appreciate that opt and llc are mainly debugging/testing tools.
The problem is mainly that these programs "are there" in the open
source
build of LLVM. Users know that they're getting a product based on open
source LLVM, so:
- If opt and llc were not shipped, then it would be reasonable for users to
ask: "why can't we have them?"
- If they were shipped, then it would be reasonable for users to ask: "can
we have them just work?"
Also, intuitively it feels that llc could be more lightweight / performing
compared to clang (which has to spawn a second process), although I admit
that I don't have performance numbers for this comparison (yet).
Anyway, now we definitely have an answer for the two questions above. :-)
Cheers,
Dario Domizioli
SN Systems - Sony Computer Entertainment Group
On 17 March 2015 at 20:27, David Jones <djones at xtreme-eda.com> wrote:
> What is the preferred method for compiler (frontend) developers to
> optimize and generate target machine code from IR?
>
> At one point I found a tutorial that recommended simply dumping the IR to
> a file and spawning llc to do the job.
>
> Up until now I have "manually" created a TargetMachine,
PassManager, etc.
> to generate my object code. The initial version of my code was cribbed from
> llc for LLVM 3.2, and has since been updated for 3.5.1.
>
> However, with every new release of LLVM, the API to the backend
> optimization and code generation passes changes. The changes from 3.5.1 to
> 3.6 are quite significant. If I'm lucky, the impact of a change is that
my
> C++ code refuses to compile, and I have to fix. If I'm unlucky, the
impact
> may be that my code compiles and runs, but LLVM works suboptimally -
> perhaps some optimizations don't happen. (This can happen if a newer
API
> expects me to do some step which was not required in earlier releases, and
> there is no assert to catch it.)
>
> As an alternative, I am seriously considering "simply dumping the IR
to a
> file and spawning llc" to perform my backend work. The API to create
IR is
> much more stable than the API to do useful things with it. Furthermore,
> it's a lot easier to manually debug IR that has been dumped to a file.
> Finally, I can spawn multiple, independent, concurrent invocations of llc
> on a multi-core machine. I needn't worry about concurrency, as the
standard
> Linux fork/waitpid type calls will suffice. Given that 90% of my runtime is
> spent inside LLVM, I get 90% of the benefit of a fully concurrent design
> with almost zero work.
>
> But now it seems that this usage model is frowned upon.
>
> What is the recommended usage model?
>
>
> On Tue, Mar 17, 2015 at 2:32 PM, Owen Anderson <resistor at mac.com>
wrote:
>
>>
>> On Mar 17, 2015, at 11:13 AM, Dario Domizioli <dario.domizioli at
gmail.com>
>> wrote:
>>
>> What does the community think?
>> Discuss. :-)
>>
>>
>> Devil’s advocate: opt, llc, lli, etc. are development/debugging tools
for
>> LLVM developers, not for end users, and the project optimizes their
>> functionality for that use case.
>>
>> —Owen
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150318/8aa834cd/attachment.html>
Sorry to be a bit controversial, but as a follow-up from this thread,
should we then explicitly document that opt, llc and lli are debugging
tools in the LLVM documentation?
The LLVM User Guide main page says that "The documentation here is intended
for users who have a need to work with the intermediate LLVM
representation."
So yes, it is for advanced users, but this wording appears to include users
having their own IR generators (they are technically "working with the
intermediate representation").
The official man pages for llc, opt and lli certainly make them look like
standalone and usable tools:
http://llvm.org/docs/CommandGuide/llc.html
http://llvm.org/docs/CommandGuide/opt.html
http://llvm.org/docs/CommandGuide/lli.html
I can't find anything explicit about them only being debugging/testing
tools, nor anything explicitly discouraging their use in production.
Even the "Getting Started with the LLVM System" documentation uses llc
in
the example!
http://llvm.org/docs/GettingStarted.html#an-example-using-the-llvm-tool-chain
If llc is just a debugging/testing tool, why does the documentation tell
beginners (the obvious target of a "getting started" document) about
its
existence?!
It could be argued that advanced users "should know better", but the
documentation could be clearer too.
Cheers,
Dario Domizioli
SN Systems - Sony Computer Entertainment Group
On 18 March 2015 at 00:17, Dario Domizioli <dario.domizioli at gmail.com>
wrote:
> Thanks for all the replies!
>
> Whoa, it looks like there's pretty much a massive consensus on
"use clang
> and never use opt/llc in that kind of scenario".
>
> I appreciate that opt and llc are mainly debugging/testing tools.
> The problem is mainly that these programs "are there" in the open
source
> build of LLVM. Users know that they're getting a product based on open
> source LLVM, so:
> - If opt and llc were not shipped, then it would be reasonable for users
> to ask: "why can't we have them?"
> - If they were shipped, then it would be reasonable for users to ask:
"can
> we have them just work?"
>
> Also, intuitively it feels that llc could be more lightweight / performing
> compared to clang (which has to spawn a second process), although I admit
> that I don't have performance numbers for this comparison (yet).
>
> Anyway, now we definitely have an answer for the two questions above. :-)
>
> Cheers,
> Dario Domizioli
> SN Systems - Sony Computer Entertainment Group
>
>
>
>
> On 17 March 2015 at 20:27, David Jones <djones at xtreme-eda.com>
wrote:
>
>> What is the preferred method for compiler (frontend) developers to
>> optimize and generate target machine code from IR?
>>
>> At one point I found a tutorial that recommended simply dumping the IR
to
>> a file and spawning llc to do the job.
>>
>> Up until now I have "manually" created a TargetMachine,
PassManager, etc.
>> to generate my object code. The initial version of my code was cribbed
from
>> llc for LLVM 3.2, and has since been updated for 3.5.1.
>>
>> However, with every new release of LLVM, the API to the backend
>> optimization and code generation passes changes. The changes from
3.5.1 to
>> 3.6 are quite significant. If I'm lucky, the impact of a change is
that my
>> C++ code refuses to compile, and I have to fix. If I'm unlucky, the
impact
>> may be that my code compiles and runs, but LLVM works suboptimally -
>> perhaps some optimizations don't happen. (This can happen if a
newer API
>> expects me to do some step which was not required in earlier releases,
and
>> there is no assert to catch it.)
>>
>> As an alternative, I am seriously considering "simply dumping the
IR to a
>> file and spawning llc" to perform my backend work. The API to
create IR is
>> much more stable than the API to do useful things with it. Furthermore,
>> it's a lot easier to manually debug IR that has been dumped to a
file.
>> Finally, I can spawn multiple, independent, concurrent invocations of
llc
>> on a multi-core machine. I needn't worry about concurrency, as the
standard
>> Linux fork/waitpid type calls will suffice. Given that 90% of my
runtime is
>> spent inside LLVM, I get 90% of the benefit of a fully concurrent
design
>> with almost zero work.
>>
>> But now it seems that this usage model is frowned upon.
>>
>> What is the recommended usage model?
>>
>>
>> On Tue, Mar 17, 2015 at 2:32 PM, Owen Anderson <resistor at
mac.com> wrote:
>>
>>>
>>> On Mar 17, 2015, at 11:13 AM, Dario Domizioli <dario.domizioli
at gmail.com>
>>> wrote:
>>>
>>> What does the community think?
>>> Discuss. :-)
>>>
>>>
>>> Devil’s advocate: opt, llc, lli, etc. are development/debugging
tools
>>> for LLVM developers, not for end users, and the project optimizes
their
>>> functionality for that use case.
>>>
>>> —Owen
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150318/72f539ff/attachment.html>