George Brewster
2010-Oct-07 16:41 UTC
[LLVMdev] Using multiple ExecutionEngines for better parallelism?
Hi, I'm working on an application that compiles using LLVM from multiple threads. Currently, I'm doing this using one ExecutionEngine, but I've observed that this does not seem to scale well past a couple cores. I'd like to be able to get as much parallelism as possible in the compilation, and was wondering about using one ExecutionEngine per thread. The only information I've found says this is not allowed, but the post is 2.5 years old, and it seems like threading in LLVM has come a long way since then. Has this changed, or is it still only possible to have one ExecutionEngine? Are there any other ways to increase the parallelism of LLVM compilation? Thanks, -George
Olivier Meurant
2010-Oct-07 17:17 UTC
[LLVMdev] Using multiple ExecutionEngines for better parallelism?
Is is possible since LLVM 2.7 to create multiple ExecutionEngine in the same process. Olivier. On Thu, Oct 7, 2010 at 6:41 PM, George Brewster <g at brewster.org> wrote:> Hi, > I'm working on an application that compiles using LLVM from multiple > threads. Currently, I'm doing this using one ExecutionEngine, but I've > observed that this does not seem to scale well past a couple cores. I'd > like to be able to get as much parallelism as possible in the > compilation, and was wondering about using one ExecutionEngine per thread. > The only information I've found says this is not allowed, but the post > is 2.5 years old, and it seems like threading in LLVM has come a long > way since then. Has this changed, or is it still only possible to have > one ExecutionEngine? Are there any other ways to increase the > parallelism of LLVM compilation? > > Thanks, > -George > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101007/1b3aa35b/attachment.html>
I set up a test case to try using multiple ExecutionEngines, and it ends up coring, where a similar test case which uses only one ExecutionEngine works. Looking at the failure, I found two threading issues that were causing crashes for me -- I've attached a patch for each one (against RELEASE_28). These patches are intended simply to demonstrate the problems, not to be the actual fix, as I know nothing of LLVM coding conventions or the larger plan for handling multi-threading, and there are certainly more clever ways to make these things thread safe. The first, in Attributes.cpp, is a race condition relating to the static AttributesList. Access to the AttributesList is locked, but my theory for what was going on is that at line 125-126, when dropping a reference, we decrement the count, then check to see if it's zero, then delete "this" (if the refcount was zero). After we've checked that the count is zero, but before deleting "this", "this" is still in the AttributesLists set. So in the mean time, another thread comes along, finds "this" in AttributesList, and references it. But the first thread has already determined that the count *was* 0, and then comes back and deletes "this", even though it's now in use by the second thread. The second, in DynamicLibrary.cpp, simply protects OpenedHandles with a mutex. Note that I couldn't use ManagedStatic with the mutex, as that would have introduced a circular library dependency (between Support and System). I've also attached the test case that I was using, in case it will be useful in verifying different solutions to these problems. -George Olivier Meurant wrote:> Is is possible since LLVM 2.7 to create multiple ExecutionEngine in > the same process. > > Olivier. > > > On Thu, Oct 7, 2010 at 6:41 PM, George Brewster <g at brewster.org > <mailto:g at brewster.org>> wrote: > > Hi, > I'm working on an application that compiles using LLVM from multiple > threads. Currently, I'm doing this using one ExecutionEngine, but I've > observed that this does not seem to scale well past a couple > cores. I'd > like to be able to get as much parallelism as possible in the > compilation, and was wondering about using one ExecutionEngine per > thread. > The only information I've found says this is not allowed, but the post > is 2.5 years old, and it seems like threading in LLVM has come a long > way since then. Has this changed, or is it still only possible to have > one ExecutionEngine? Are there any other ways to increase the > parallelism of LLVM compilation? > > Thanks, > -George > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu> > http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- A non-text attachment was scrubbed... Name: llvmAttributes.patch Type: text/x-patch Size: 1080 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101008/432edbaf/attachment.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: llvmDynamicLibrary.patch Type: text/x-patch Size: 1255 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101008/432edbaf/attachment-0001.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: multiple_EE_failure.tar.gz Type: application/x-gzip Size: 4930 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101008/432edbaf/attachment-0002.bin>