Christian Schafmeister via llvm-dev
2017-Nov-06 20:28 UTC
[llvm-dev] ORC JIT and multithreading
<html><head></head><body dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><meta http-equiv="Content-Type" content="text/html charset=utf-8"><div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><div class="">How would I go about enabling the ORC JIT to compile code in multiple threads?</div><div class=""><br class=""></div><div class="">Some background:</div><div class="">I’ve switched Clasp (an implementation of Common Lisp that uses llvm as the backend - <a href="http://github.com/drmeister/clasp" class="">github.com/drmeister/clasp</a>) over to using ORC as its JIT. I did this by following the Kaleidoscope tutorial. I have a ClaspJIT_O class that copies the KaleidoscopeJIT class.</div><div class=""><a href="https://github.com/drmeister/clasp/blob/analyze/include/clasp/llvmo/llvmoExpose.h#L4370" class="">https://github.com/drmeister/clasp/blob/analyze/include/clasp/llvmo/llvmoExpose.h#L4370</a></div><div class=""><br class=""></div><div class="">Clasp is multithreaded and it needs to compile code in multiple threads because it uses the JIT to generate dispatch functions for multiple dispatch/generic functions. To make this possible, every thread gets its own LLVMContext and every type and llvm::Module that is linked into JITted code in each thread is initialized lazily and thread-locally. Despite this - I experience frequent, random crashes when I try to use the ORC JIT from multiple threads. </div><div class=""><br class=""></div><div class="">Here’s what I’ve tried:</div><div class=""><br class=""></div><div class="">(1) This works: wrap a lock/mutex around access to one ClaspJIT_O object, the calls to ClaspJIT_O::addModule and a call to jitFinalizeReplFunction are protected by the lock:</div><div class=""><a href="https://github.com/drmeister/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3999" class="">https://github.com/drmeister/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3999</a></div><div class=""><br class=""></div><div class="">The Common Lisp code that does the lock and calls these functions:</div><div class=""><a href="https://github.com/drmeister/clasp/blob/analyze/src/lisp/kernel/cmp/jit-setup.lsp#L598" class="">https://github.com/drmeister/clasp/blob/analyze/src/lisp/kernel/cmp/jit-setup.lsp#L598</a></div><div class=""><br class=""></div><div class="">This throttles the system and limits one thread at a time to add modules to the JIT and lookup symbols in the JIT. It’s not bad - I can live with it.</div><div class=""><br class=""></div><div class="">(2) This fails: Keep a thread local copy of a ClaspJIT_O object that is lazily initialized as soon as any compilation happens in a thread.</div><div class=""><br class=""></div><div class="">(3) This fails: Keep a thread local copy of a ClaspJIT_O object that is initialized as in #2 AND wrap a lock/mutex around ClaspJIT_O::addModule and a call to jitFinalizeReplFunction. What I thought I was testing here was if there was some global resource that ORC uses and despite having multiple thread-local ClaspJIT_O objects the different threads were trampling that common global resource.</div><div class=""><br class=""></div><div class="">I can provide many more details on request.</div><div class=""><br class=""></div><div class="">Christian Schafmeister</div><div class=""><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;">Professor, Chemistry</div><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;">Temple University</div> </div> <br class=""></div></body></html>
Hi Christian, ORC doesn't have any locks internally at the moment. Approach (1) is the recommended solution. I'm working on a refactor that should be out in a few weeks that will improve threading support, and I expect to put more effort into multi-threaded performance in the next few months. Out of interest, are you saying that approach (1) was a regression compared to MCJIT's behavior? Cheers, Lang. On Mon, Nov 6, 2017 at 12:28 PM, Christian Schafmeister via llvm-dev < llvm-dev at lists.llvm.org> wrote:> > How would I go about enabling the ORC JIT to compile code in multiple > threads? > > Some background: > I’ve switched Clasp (an implementation of Common Lisp that uses llvm as > the backend - github.com/drmeister/clasp) over to using ORC as its JIT. > I did this by following the Kaleidoscope tutorial. I have a ClaspJIT_O > class that copies the KaleidoscopeJIT class. > https://github.com/drmeister/clasp/blob/analyze/include/ > clasp/llvmo/llvmoExpose.h#L4370 > > Clasp is multithreaded and it needs to compile code in multiple threads > because it uses the JIT to generate dispatch functions for multiple > dispatch/generic functions. To make this possible, every thread gets its > own LLVMContext and every type and llvm::Module that is linked into JITted > code in each thread is initialized lazily and thread-locally. Despite > this - I experience frequent, random crashes when I try to use the ORC JIT > from multiple threads. > > Here’s what I’ve tried: > > (1) This works: wrap a lock/mutex around access to one ClaspJIT_O > object, the calls to ClaspJIT_O::addModule and a call to > jitFinalizeReplFunction are protected by the lock: > https://github.com/drmeister/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3999 > > The Common Lisp code that does the lock and calls these functions: > https://github.com/drmeister/clasp/blob/analyze/src/lisp/ > kernel/cmp/jit-setup.lsp#L598 > > This throttles the system and limits one thread at a time to add modules > to the JIT and lookup symbols in the JIT. It’s not bad - I can live with > it. > > (2) This fails: Keep a thread local copy of a ClaspJIT_O object that is > lazily initialized as soon as any compilation happens in a thread. > > (3) This fails: Keep a thread local copy of a ClaspJIT_O object that is > initialized as in #2 AND wrap a lock/mutex around ClaspJIT_O::addModule and > a call to jitFinalizeReplFunction. What I thought I was testing here was > if there was some global resource that ORC uses and despite having multiple > thread-local ClaspJIT_O objects the different threads were trampling that > common global resource. > > I can provide many more details on request. > > Christian Schafmeister > Professor, Chemistry > Temple University > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171114/764bd6db/attachment.html>
Christian Schafmeister via llvm-dev
2017-Nov-15 03:23 UTC
[llvm-dev] ORC JIT and multithreading
Hi Lang, I’m excited to hear about an ORC refactor with improved threading support - I really like ORC. I added multithreading to Clasp after I made the switch to ORC - so I don’t have any experience using MCJIT in a multithreaded environment. Thank you for responding. Best, .Chris.> On Nov 14, 2017, at 7:31 PM, Lang Hames <lhames at gmail.com> wrote: > > Hi Christian, > > ORC doesn't have any locks internally at the moment. Approach (1) is the recommended solution. I'm working on a refactor that should be out in a few weeks that will improve threading support, and I expect to put more effort into multi-threaded performance in the next few months. > > Out of interest, are you saying that approach (1) was a regression compared to MCJIT's behavior? > > Cheers, > Lang. > > > On Mon, Nov 6, 2017 at 12:28 PM, Christian Schafmeister via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > > How would I go about enabling the ORC JIT to compile code in multiple threads? > > Some background: > I’ve switched Clasp (an implementation of Common Lisp that uses llvm as the backend - github.com/drmeister/clasp <http://github.com/drmeister/clasp>) over to using ORC as its JIT. I did this by following the Kaleidoscope tutorial. I have a ClaspJIT_O class that copies the KaleidoscopeJIT class. > https://github.com/drmeister/clasp/blob/analyze/include/clasp/llvmo/llvmoExpose.h#L4370 <https://github.com/drmeister/clasp/blob/analyze/include/clasp/llvmo/llvmoExpose.h#L4370> > > Clasp is multithreaded and it needs to compile code in multiple threads because it uses the JIT to generate dispatch functions for multiple dispatch/generic functions. To make this possible, every thread gets its own LLVMContext and every type and llvm::Module that is linked into JITted code in each thread is initialized lazily and thread-locally. Despite this - I experience frequent, random crashes when I try to use the ORC JIT from multiple threads. > > Here’s what I’ve tried: > > (1) This works: wrap a lock/mutex around access to one ClaspJIT_O object, the calls to ClaspJIT_O::addModule and a call to jitFinalizeReplFunction are protected by the lock: > https://github.com/drmeister/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3999 <https://github.com/drmeister/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3999> > > The Common Lisp code that does the lock and calls these functions: > https://github.com/drmeister/clasp/blob/analyze/src/lisp/kernel/cmp/jit-setup.lsp#L598 <https://github.com/drmeister/clasp/blob/analyze/src/lisp/kernel/cmp/jit-setup.lsp#L598> > > This throttles the system and limits one thread at a time to add modules to the JIT and lookup symbols in the JIT. It’s not bad - I can live with it. > > (2) This fails: Keep a thread local copy of a ClaspJIT_O object that is lazily initialized as soon as any compilation happens in a thread. > > (3) This fails: Keep a thread local copy of a ClaspJIT_O object that is initialized as in #2 AND wrap a lock/mutex around ClaspJIT_O::addModule and a call to jitFinalizeReplFunction. What I thought I was testing here was if there was some global resource that ORC uses and despite having multiple thread-local ClaspJIT_O objects the different threads were trampling that common global resource. > > I can provide many more details on request. > > Christian Schafmeister > Professor, Chemistry > Temple University > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171114/f74d03f6/attachment.html>
On Tue, Nov 14, 2017 at 4:31 PM Lang Hames via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi Christian, > > ORC doesn't have any locks internally at the moment. Approach (1) is the > recommended solution. I'm working on a refactor that should be out in a few > weeks that will improve threading support, and I expect to put more effort > into multi-threaded performance in the next few months. >What's the shared state that necessitates (1)? Having a separate LLVMContext+separate ORC JIT stack (I think that's what's being suggested - in (2) for example), etc should suffice? If a whole process can only have one ORC JIT stack that seems really problematic, and I'd be surprised if that were the case so I guess I'm missing something.> > Out of interest, are you saying that approach (1) was a regression > compared to MCJIT's behavior? > > Cheers, > Lang. > > > On Mon, Nov 6, 2017 at 12:28 PM, Christian Schafmeister via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> >> How would I go about enabling the ORC JIT to compile code in multiple >> threads? >> >> Some background: >> I’ve switched Clasp (an implementation of Common Lisp that uses llvm as >> the backend - github.com/drmeister/clasp) over to using ORC as its JIT. >> I did this by following the Kaleidoscope tutorial. I have a ClaspJIT_O >> class that copies the KaleidoscopeJIT class. >> >> https://github.com/drmeister/clasp/blob/analyze/include/clasp/llvmo/llvmoExpose.h#L4370 >> >> Clasp is multithreaded and it needs to compile code in multiple threads >> because it uses the JIT to generate dispatch functions for multiple >> dispatch/generic functions. To make this possible, every thread gets its >> own LLVMContext and every type and llvm::Module that is linked into JITted >> code in each thread is initialized lazily and thread-locally. Despite >> this - I experience frequent, random crashes when I try to use the ORC JIT >> from multiple threads. >> >> Here’s what I’ve tried: >> >> (1) This works: wrap a lock/mutex around access to one ClaspJIT_O >> object, the calls to ClaspJIT_O::addModule and a call to >> jitFinalizeReplFunction are protected by the lock: >> https://github.com/drmeister/clasp/blob/dev/src/llvmo/llvmoExpose.cc#L3999 >> >> The Common Lisp code that does the lock and calls these functions: >> >> https://github.com/drmeister/clasp/blob/analyze/src/lisp/kernel/cmp/jit-setup.lsp#L598 >> >> This throttles the system and limits one thread at a time to add modules >> to the JIT and lookup symbols in the JIT. It’s not bad - I can live with >> it. >> >> (2) This fails: Keep a thread local copy of a ClaspJIT_O object that is >> lazily initialized as soon as any compilation happens in a thread. >> >> (3) This fails: Keep a thread local copy of a ClaspJIT_O object that is >> initialized as in #2 AND wrap a lock/mutex around ClaspJIT_O::addModule and >> a call to jitFinalizeReplFunction. What I thought I was testing here was >> if there was some global resource that ORC uses and despite having multiple >> thread-local ClaspJIT_O objects the different threads were trampling that >> common global resource. >> >> I can provide many more details on request. >> >> Christian Schafmeister >> Professor, Chemistry >> Temple University >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171115/8d8ea566/attachment.html>
Maybe Matching Threads
- [LLVMdev] How to run LLVM3.6.1 on OS X (Yosemite, Xcode6.4) OR how to link bitcode generated by OS X clang with LLVM3.6.1
- Inlining with the ORC JIT
- [LLVMdev] What is on the LLVM horizon for truly relocatable JITted code?
- ORC JIT api, object files and stackmaps
- [LLVMdev] Assertion: replaceAllUses of value with new value of different type! being thrown all of a sudden