Stephen Kyle
2010-Nov-08 21:22 UTC
[LLVMdev] Calling PassManager on previously JITed Modules
Hi, Has anyone had any success with running different PassManagers on llvm::Modules they've already JITed and are executing? In detail: 1) getting the IR, in form of an llvm::Module 2) calling PassManager->run() on the module 3) calling getFunction() and getPointerToFunction() to JIT the module 4) executing the JITed code using the function pointer received in step 3 and then what I want to do is: 5) using a different PassManager with different (more time-consuming) Passes, call run() on the module again 6) call getFunction() and getPointerToFunction() again 7) use the new function pointer to execute the JITed code instead. As an added complication, different threads may be performing the initial compilation or reoptimisation, but before we consider this, is there any reason why the above shouldn't work? When I do it, I occasionally get assertion errors such as: <more path...>/RELEASE_28/include/llvm/ADT/APInt.h:819: bool llvm::APInt::operator==(const llvm::APInt&) const: Assertion `BitWidth == RHS.BitWidth && "Comparison requires equal bit widths"' failed. Stack dump: 0. Running pass 'Function Pass Manager' on module 'lib0000'. 1. Running pass 'Combine redundant instructions' on function '<func name>' Aborted or RELEASE_28/lib/VMCore/LeaksContext.h:50: void llvm::LeakDetectorImpl<T>::addGarbage(const T*) [with T llvm::Value]: Assertion `Ts.count(o) == 0 && "Object already in set!"' failed. Stack dump: 0. Running pass 'Function Pass Manager' on module 'lib000d'. 1. Running pass 'Combine redundant instructions' on function '<func name>' Aborted It only happens sometimes, so this would imply some kind of race condition between the compliation and reoptimisation threads, but I'm definitely not trying to run a PassManager on a module which is currently being JITed, so perhaps something about the module is being corrupted by the call to the JIT? Hopefully the above assertion checks indicate something obvious to someone...? Cheers, Stephen
Stephen Kyle
2010-Nov-09 17:57 UTC
[LLVMdev] Calling PassManager on previously JITed Modules
Hi, I found the following wiki page in the Unladen Swallow project: http://code.google.com/p/unladen-swallow/wiki/CodeLifecycle This would appear to answer my question. Could someone confirm for me if it's definitely unsafe to attempt to optimise/JIT any Modules while a different thread is currently executing a JITed function which has been generated from them? Or am I just missing something here? I've tried using llvm::CloneModule() to copy the Module and then work on that instead to JIT a new function, but it seems that doesn't always work, or even the cloning process itself causes an assertion to be raised. Thanks in advance for any replies, Stephen On 8 November 2010 21:22, Stephen Kyle <stephenckyle at googlemail.com> wrote:> Hi, > > Has anyone had any success with running different PassManagers on > llvm::Modules they've already JITed and are executing? > > In detail: > > 1) getting the IR, in form of an llvm::Module > 2) calling PassManager->run() on the module > 3) calling getFunction() and getPointerToFunction() to JIT the module > 4) executing the JITed code using the function pointer received in step 3 > > and then what I want to do is: > > 5) using a different PassManager with different (more time-consuming) > Passes, call run() on the module again > 6) call getFunction() and getPointerToFunction() again > 7) use the new function pointer to execute the JITed code instead. > > As an added complication, different threads may be performing the > initial compilation or reoptimisation, but before we consider this, is > there any reason why the above shouldn't work? > > When I do it, I occasionally get assertion errors such as: > > <more path...>/RELEASE_28/include/llvm/ADT/APInt.h:819: bool > llvm::APInt::operator==(const llvm::APInt&) const: Assertion `BitWidth > == RHS.BitWidth && "Comparison requires equal bit widths"' failed. > Stack dump: > 0. Running pass 'Function Pass Manager' on module 'lib0000'. > 1. Running pass 'Combine redundant instructions' on function '<func name>' > Aborted > > or > > RELEASE_28/lib/VMCore/LeaksContext.h:50: void > llvm::LeakDetectorImpl<T>::addGarbage(const T*) [with T > llvm::Value]: Assertion `Ts.count(o) == 0 && "Object already in set!"' > failed. > Stack dump: > 0. Running pass 'Function Pass Manager' on module 'lib000d'. > 1. Running pass 'Combine redundant instructions' on function '<func name>' > Aborted > > It only happens sometimes, so this would imply some kind of race > condition between the compliation and reoptimisation threads, but I'm > definitely not trying to run a PassManager on a module which is > currently being JITed, so perhaps something about the module is being > corrupted by the call to the JIT? Hopefully the above assertion checks > indicate something obvious to someone...? > > Cheers, > Stephen >
Olivier Meurant
2010-Nov-09 18:56 UTC
[LLVMdev] Calling PassManager on previously JITed Modules
Hi Stephen, I confirm your observation. AFAIK the current JIT keeps informations from the module, for example AssertingHandle on Values. It's part of my plan to make the MCJIT independent from Module stuff to allow reoptimizations, or to have multiple copies of JITed functions for one function in the module, but there is a long road to go. Olivier. On Tue, Nov 9, 2010 at 6:57 PM, Stephen Kyle <stephenckyle at googlemail.com>wrote:> Hi, > > I found the following wiki page in the Unladen Swallow project: > > http://code.google.com/p/unladen-swallow/wiki/CodeLifecycle > > This would appear to answer my question. Could someone confirm for me > if it's definitely unsafe to attempt to optimise/JIT any Modules while > a different thread is currently executing a JITed function which has > been generated from them? Or am I just missing something here? > > I've tried using llvm::CloneModule() to copy the Module and then work > on that instead to JIT a new function, but it seems that doesn't > always work, or even the cloning process itself causes an assertion to > be raised. > > Thanks in advance for any replies, > Stephen > > On 8 November 2010 21:22, Stephen Kyle <stephenckyle at googlemail.com> > wrote: > > Hi, > > > > Has anyone had any success with running different PassManagers on > > llvm::Modules they've already JITed and are executing? > > > > In detail: > > > > 1) getting the IR, in form of an llvm::Module > > 2) calling PassManager->run() on the module > > 3) calling getFunction() and getPointerToFunction() to JIT the module > > 4) executing the JITed code using the function pointer received in step 3 > > > > and then what I want to do is: > > > > 5) using a different PassManager with different (more time-consuming) > > Passes, call run() on the module again > > 6) call getFunction() and getPointerToFunction() again > > 7) use the new function pointer to execute the JITed code instead. > > > > As an added complication, different threads may be performing the > > initial compilation or reoptimisation, but before we consider this, is > > there any reason why the above shouldn't work? > > > > When I do it, I occasionally get assertion errors such as: > > > > <more path...>/RELEASE_28/include/llvm/ADT/APInt.h:819: bool > > llvm::APInt::operator==(const llvm::APInt&) const: Assertion `BitWidth > > == RHS.BitWidth && "Comparison requires equal bit widths"' failed. > > Stack dump: > > 0. Running pass 'Function Pass Manager' on module 'lib0000'. > > 1. Running pass 'Combine redundant instructions' on function '<func > name>' > > Aborted > > > > or > > > > RELEASE_28/lib/VMCore/LeaksContext.h:50: void > > llvm::LeakDetectorImpl<T>::addGarbage(const T*) [with T > > llvm::Value]: Assertion `Ts.count(o) == 0 && "Object already in set!"' > > failed. > > Stack dump: > > 0. Running pass 'Function Pass Manager' on module 'lib000d'. > > 1. Running pass 'Combine redundant instructions' on function '<func > name>' > > Aborted > > > > It only happens sometimes, so this would imply some kind of race > > condition between the compliation and reoptimisation threads, but I'm > > definitely not trying to run a PassManager on a module which is > > currently being JITed, so perhaps something about the module is being > > corrupted by the call to the JIT? Hopefully the above assertion checks > > indicate something obvious to someone...? > > > > Cheers, > > Stephen > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20101109/22de7032/attachment.html>
Keppler, Dave
2010-Nov-09 19:10 UTC
[LLVMdev] Calling PassManager on previously JITed Modules
Your step #2 will only execute Module level passes. JIT::getPointerToFunction can trigger materialization and will invoke the JIT state's FunctionPassManager, both of which can alter the Module. That's a likely the source of the synchronization problems you're encountering. - Dave>-----Original Message----- >From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On >Behalf Of Stephen Kyle >Sent: Tuesday, November 09, 2010 12:57 PM >To: llvmdev at cs.uiuc.edu >Subject: Re: [LLVMdev] Calling PassManager on previously JITed Modules > >Hi, > >I found the following wiki page in the Unladen Swallow project: > >http://code.google.com/p/unladen-swallow/wiki/CodeLifecycle > >This would appear to answer my question. Could someone confirm for me >if it's definitely unsafe to attempt to optimise/JIT any Modules while >a different thread is currently executing a JITed function which has >been generated from them? Or am I just missing something here? > >I've tried using llvm::CloneModule() to copy the Module and then work >on that instead to JIT a new function, but it seems that doesn't >always work, or even the cloning process itself causes an assertion to >be raised. > >Thanks in advance for any replies, >Stephen > >On 8 November 2010 21:22, Stephen Kyle <stephenckyle at googlemail.com> wrote: >> Hi, >> >> Has anyone had any success with running different PassManagers on >> llvm::Modules they've already JITed and are executing? >> >> In detail: >> >> 1) getting the IR, in form of an llvm::Module >> 2) calling PassManager->run() on the module >> 3) calling getFunction() and getPointerToFunction() to JIT the module >> 4) executing the JITed code using the function pointer received in step 3 >> >> and then what I want to do is: >> >> 5) using a different PassManager with different (more time-consuming) >> Passes, call run() on the module again >> 6) call getFunction() and getPointerToFunction() again >> 7) use the new function pointer to execute the JITed code instead. >> >> As an added complication, different threads may be performing the >> initial compilation or reoptimisation, but before we consider this, is >> there any reason why the above shouldn't work? >> >> When I do it, I occasionally get assertion errors such as: >> >> <more path...>/RELEASE_28/include/llvm/ADT/APInt.h:819: bool >> llvm::APInt::operator==(const llvm::APInt&) const: Assertion `BitWidth >> == RHS.BitWidth && "Comparison requires equal bit widths"' failed. >> Stack dump: >> 0. Running pass 'Function Pass Manager' on module 'lib0000'. >> 1. Running pass 'Combine redundant instructions' on function '<func >name>' >> Aborted >> >> or >> >> RELEASE_28/lib/VMCore/LeaksContext.h:50: void >> llvm::LeakDetectorImpl<T>::addGarbage(const T*) [with T >> llvm::Value]: Assertion `Ts.count(o) == 0 && "Object already in set!"' >> failed. >> Stack dump: >> 0. Running pass 'Function Pass Manager' on module 'lib000d'. >> 1. Running pass 'Combine redundant instructions' on function '<func >name>' >> Aborted >> >> It only happens sometimes, so this would imply some kind of race >> condition between the compliation and reoptimisation threads, but I'm >> definitely not trying to run a PassManager on a module which is >> currently being JITed, so perhaps something about the module is being >> corrupted by the call to the JIT? Hopefully the above assertion checks >> indicate something obvious to someone...? >> >> Cheers, >> Stephen >> > >_______________________________________________ >LLVM Developers mailing list >LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
Reasonably Related Threads
- [LLVMdev] Calling PassManager on previously JITed Modules
- [LLVMdev] Calling PassManager on previously JITed Modules
- [LLVMdev] GSOC Adaptive Compilation Framework for LLVM JIT Compiler
- [LLVMdev] Throwing C++ exception through LLVM JITed code
- Linking existing functions from JITed code