preejackie via llvm-dev
2019-Mar-13 22:40 UTC
[llvm-dev] [RFC] Improving Speculative Compilation in Concurrent ORC JIT
Dear All, The new concurrent orc jit support speculative compilation with a naive implementation and laziness. Compiling everything up-front can consume more CPU and memory than needed and also impose significant effect on resource constraint systems due to context switching of JIT threads (coarse level parallelism). The main idea here is to develop heuristics to find which functions are worth compiling into executable bits, speculatively using jit threads. Heuristics can be extracted by analyzing higher level program representations such as ASTs, CFGs and from LLVM IR. Profile guided optimization can also used to determine heuristics with greater confidence level based on previous execution steps. Goals: 1) Create & Improve APIs to support heuristic guided speculative compilation in concurrent orc jit. 2) Finding key heuristics to decide what to compile next. 3) Bench marking jit with eagerly speculating full module vs heuristic based approach for different jit threads. Possible caveats: 1) More time spent in analyzing what to compile next than actually compiling. These can addressed once we have basic working prototype, so that we can find out the bottlenecks and remove it. Or may be parallelizing analysis process. Currently, I'm trying to find heuristics that can be used for this project and familiarizing with concurrent jit design and APIs. Also, please note that this idea has been proposed by Lang Hames and I would like to propose this idea & implementation for GSoC 2019 :) and I'm improvising this idea. I would like to know the community responses! This can serve as a good documentation for me to write a proposal and to keep things in track. Any thoughts on finding useful heuristics are highly appreciated :) I have also started a thread in Numba JIT project to see what heuristics they are using. Also anyone interested in mentoring this project please reply! References: LLVM Next Gen JIT API <https://www.youtube.com/watch?v=hILdR8XRvdQ> Updating Orc for concurrency <https://www.youtube.com/watch?v=MOQG5vkh9J8> Thinlto summaries in JIT <https://www.youtube.com/watch?v=ZCnHxRhQmvs> Have a great day! PreeJackie -- Have a great day! PreeJackie -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190314/e07fcc09/attachment.html>
preejackie via llvm-dev
2019-Mar-15 20:46 UTC
[llvm-dev] [RFC] Improving Speculative Compilation in Concurrent ORC JIT
Hi all, I have come up with some idea for the speculation task: 1) Construct the local(per-module) function ordering list based on the sequence of their call in control flow graphs and create stub for each symbol from other module that is referenced by the current function, put the module in waiting list. 2) Replace the stub with actual native code address and notify the modules in the waiting list. The creation of function ordering list involves analysing the control flow graphs and branch probability for conditional function calls. I'm also trying to figure out whether the function atttributes will fit in this picture + more like using module summaries from thinlto builds. These speculation actions should be buried into internals of concurrent compiler and should not be visible to clients right? How I can proceed with plan, I'm open to comments! Anyone interested in mentoring this project for GSoC 2019. Please reply! On 14/03/19 4:10 AM, preejackie wrote:> > Dear All, > > The new concurrent orc jit support speculative compilation with a > naive implementation and laziness. Compiling everything up-front can > consume more CPU and memory than needed and also impose significant > effect on resource constraint systems due to context switching of JIT > threads (coarse level parallelism). The main idea here is to develop > heuristics to find which functions are worth compiling into executable > bits, speculatively using jit threads. > > Heuristics can be extracted by analyzing higher level program > representations such as ASTs, CFGs and from LLVM IR. Profile guided > optimization can also used to determine heuristics with greater > confidence level based on previous execution steps. > > Goals: > > 1) Create & Improve APIs to support heuristic guided speculative > compilation in concurrent orc jit. > > 2) Finding key heuristics to decide what to compile next. > > 3) Bench marking jit with eagerly speculating full module vs heuristic > based approach for different jit threads. > > Possible caveats: > > 1) More time spent in analyzing what to compile next than actually > compiling. These can addressed once we have basic working prototype, > so that we can find out the bottlenecks and remove it. Or may be > parallelizing analysis process. > > > Currently, I'm trying to find heuristics that can be used for this > project and familiarizing with concurrent jit design and APIs. Also, > please note that this idea has been proposed by Lang Hames and I would > like to propose this idea & implementation for GSoC 2019 :) and I'm > improvising this idea. I would like to know the community responses! > > This can serve as a good documentation for me to write a proposal and > to keep things in track. > > Any thoughts on finding useful heuristics are highly appreciated :) I > have also started a thread in Numba JIT project to see what heuristics > they are using. > > Also anyone interested in mentoring this project please reply! > > References: > > LLVM Next Gen JIT API <https://www.youtube.com/watch?v=hILdR8XRvdQ> > > Updating Orc for concurrency <https://www.youtube.com/watch?v=MOQG5vkh9J8> > > Thinlto summaries in JIT <https://www.youtube.com/watch?v=ZCnHxRhQmvs> > > Have a great day! > PreeJackie > -- > Have a great day! > PreeJackie-- Have a great day! PreeJackie -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190316/aae14fab/attachment.html>