llvm dev - Feb 2020 - Multi-Threading Compilers

Greetings All,

For the last few months since the GCC Cauldron I've been 
researching/helping out
plan for multi-threading GCC. I've posted my GCC ideas here:
https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIpC9gUMjOxk/edit

as I've heard there is interest on the LLVM side as well. I've talked to
some of the IBM folks from
the Toronto area about it face to face due to them having more middle 
end and back end knowledge
then me.

Not sure if anyone is interested in my ideas or wants me to extend my 
help on the LLVM side,

Nick

Hi Nicholas,

You might want to check out MLIR: its pass manager is already automatically and
implicitly multithreaded.

-Chris
> On Feb 27, 2020, at 7:05 PM, Nicholas Krause via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Greetings All,
> 
> For the last few months since the GCC Cauldron I've been
researching/helping out
> plan for multi-threading GCC. I've posted my GCC ideas here:
>
https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIpC9gUMjOxk/edit
> 
> as I've heard there is interest on the LLVM side as well. I've
talked to some of the IBM folks from
> the Toronto area about it face to face due to them having more middle end
and back end knowledge
> then me.
> 
> Not sure if anyone is interested in my ideas or wants me to extend my help
on the LLVM side,
> 
> Nick
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On 2/28/20 12:19 AM, Chris Lattner wrote:
> Hi Nicholas,
>
> You might want to check out MLIR: its pass manager is already automatically
and implicitly multithreaded.
>
> -Chris
Chris,

I was aware that LLVM was moving to MLIR at some point due to this. I've 
curious as
to how MLIR deals with IPO as that's the problem I was running into. 
Even if you have
pipelines what guarantees do we have about IPO or are there any. For 
example if an
analysis pass runs after a loop pass for heuristics to get good data,  
does MLIR ensure
that?   The problem isn't getting a pipeline as much as IPO issues in 
terms of rescheduling
in a pipeline or incorrectly splitting up into pipelines. I yet to find 
a good solution on the
GCC side as well and it seems that there will be issues with this no 
matter what, not
sure of what trade offs the LLVM side is willing to make.

The other issues is that graph data structures to my knowledge do not 
allow insertion
of multiple nodes at the same time or how to scale the graphs for 
callers or SSA. Not
sure if you guys have encapsulated the operators and data structures in 
a series of
classes as that would be the first part. The other part is figuring out 
how to link and
interact with build systems to launch threads from make -j or other 
similar things.


Thanks for the notice about MLIR through maybe my IPO is not really there
but after reading parts of it seems to be a issue through a little 
smaller still
and thanks for the  prompt response,

Nick
>> On Feb 27, 2020, at 7:05 PM, Nicholas Krause via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>
>> Greetings All,
>>
>> For the last few months since the GCC Cauldron I've been
researching/helping out
>> plan for multi-threading GCC. I've posted my GCC ideas here:
>>
https://docs.google.com/document/d/1po_RRgSCtRyYgMHjV0itW8iOzJXpTdHYIpC9gUMjOxk/edit
>>
>> as I've heard there is interest on the LLVM side as well. I've
talked to some of the IBM folks from
>> the Toronto area about it face to face due to them having more middle
end and back end knowledge
>> then me.
>>
>> Not sure if anyone is interested in my ideas or wants me to extend my
help on the LLVM side,
>>
>> Nick
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev