Hi Jeffrey, Thanks for pointing me in the right direction ! I'm not using the JIT in lazy mode, but it was fun to understand the lazy-stub code. Attached you will find a patch which follow your 1st option : a map Stub_address -> JITResolver instance, except that the used map is a "std::map" to apply the same upper_bound trick as in the map CallSiteToFunctionMap of the ResolverState. (If it is necessary for call_site -> function, this should be necessary for call_site -> resolver... Event if I'm not sure to master all subtle reasons - code alignment/prologue ?) A mutex on the StubToResolverMap should prevent any possible race conditions. Every object (Emitter, Resolver, ResolverState) have now a private member instance of JIT (JIT *Jit). The bugpoint tool is using a extern 'C' function : "getPointerToNamedFunction" to get native pointer on function. This function used the TheJit static pointer. To keep alive this functionality, I have added a static SmallVector<JIT*> JitPool (with its mutex JitPoolLock), which keep pointers of available JIT instance. A new static method on JIT object "getPointerToNamedFunctionOnJitPool" fills the need (iterate over available JIT to find the function). Attached you will find a test, based on "HowToUseJit" example, which instantiate 2 JIT, in eager/lazy mode and retrieve native pointer via getPointerToNamedFunction function. Where does it belong ? Should I add something to llvm test suite ? To the test directory ? Let me know what you think ! Thanks. Olivier. On Thu, Feb 4, 2010 at 9:03 PM, Jeffrey Yasskin <jyasskin at google.com> wrote:> In eager compilation mode, I don't know of anything that would go > wrong with having multiple JITs in the process. However, in lazy > compilation mode, we need to map stub addresses to the JIT that knows > how to compile them. Right now, that's done by looking up the static > "TheJITResolver" variable and assuming it's the only JIT, but we could > 1) use a static DenseMap<stub_address, JITResolver*> instead, or 2) > include the JITResolver* inside the stub as an argument to the > compilation callback. Nobody's needed this enough to get it working > though. > > I think it'd make some sense to fix it for eager compilation even > before getting lazy compilation working. Would you like to write a > patch? You'll have to remove all uses of TheJIT and TheJITResolver > except for the one in JITResolver::JITCompilerFn, and change the "only > one JIT" check to say something about "only one JIT compiling lazily". > > I don't think this change will require passing an LLVMContext to the > JIT--it should just use the Context of the function it's jitting. > > The code freeze for llvm-2.7 is on Feb 21 (this probably isn't a > "major" change), so if you want it in the 2.7 release, please try to > mail the patch well ahead of that. > > On Thu, Feb 4, 2010 at 5:47 AM, Olivier Meurant > <meurant.olivier at gmail.com> wrote: > > Hi everyone ! > > > > If I call ExecutionEngine::createJIT (or EngineBuilder::create) more than > > one time, the second time fails on a assertion "Multiple JIT resolvers?". > > It seems that the JIT is designed to be a singleton in the process, and I > > was wondering if it was something mandatory. > > How hard will it be to make it a non-singleton object ? Is this a > JIT-only > > problem (work needed on JIT classes only) ? Is it a much wider design > > constraint (on codegen ?) ? > > > > In case, if you ask why I want to create more than one JIT, here is a > reason > > : > > I have a JIT application which works fine, a thread is responsible to > > prepare code execution (work on module, add functions, optimize, call > > JIT...) and a another thread is responsible for executing the code > produced > > in the first thread. > > It works great, but the preparation thread is heavily loaded. I decide to > > look at what happens, if I spawns 2 threads for preparation of the code. > > I have a working prototype, which have one module and one context by > thread, > > and it works. But the JIT is still shared between 2 threads and the time > > needed to emit a function is not negligible. It will be a great boost of > > performances, if I can create one execution engine by thread. > > > > What do you think ? Do you have any comments on it ? > > Thanks for reading ! > > > > Olivier. > > > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100207/9e038722/attachment.html> -------------- next part -------------- A non-text attachment was scrubbed... Name: jit_not_a_singleton.diff Type: text/x-patch Size: 29297 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100207/9e038722/attachment.bin> -------------- next part -------------- A non-text attachment was scrubbed... Name: HowToUseMultipleJIT.cpp Type: text/x-c++src Size: 6062 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100207/9e038722/attachment.cpp>
Hi! Olivier Meurant wrote:> > Attached you will find a patch which follow your 1st option... >Is there any chance this patch (or a similar solution) will be in 2.7? This would be a very useful for my application. I assumed one could have multiple execution engines, however, from search this mailing list I get the sense that a single execution engine will remain the preferred way of JITting multiple modules. My application is a server that uses LLVM to apply shaders to graphics. The shaders are passed to the server along with the graphical material. The shading language uses global values, and I assumed the way to achieve separation (both in terms of concurrency as well as address space) between the concurrent sessions was to instantiate multiple execution engines. The singleton nature of JITResolver prevents this. If this patch makes it in 2.7 (or any future release), I can for now use a patched trunk until it makes it in an official release. However, if the intend is not to support this, it is more productive to use the ExecutionEngine as intended. It would be nice if the documentation provides some information on the subject... Thanks. Maarten -- View this message in context: http://old.nabble.com/Jit-singleton-tp27453092p27499883.html Sent from the LLVM - Dev mailing list archive at Nabble.com.
Thanks for the patch! I'll clean this up, convert your sample to a unit test, and commit it for 2.7. On Sun, Feb 7, 2010 at 6:09 AM, Olivier Meurant <meurant.olivier at gmail.com> wrote:> Hi Jeffrey, > > Thanks for pointing me in the right direction ! > I'm not using the JIT in lazy mode, but it was fun to understand the > lazy-stub code. > > Attached you will find a patch which follow your 1st option : a map > Stub_address -> JITResolver instance, except that the used map is a > "std::map" to apply the same upper_bound trick as in the map > CallSiteToFunctionMap of the ResolverState. (If it is necessary for > call_site -> function, this should be necessary for call_site -> resolver... > Event if I'm not sure to master all subtle reasons - code alignment/prologue > ?) > A mutex on the StubToResolverMap should prevent any possible race > conditions. > > Every object (Emitter, Resolver, ResolverState) have now a private member > instance of JIT (JIT *Jit). > > The bugpoint tool is using a extern 'C' function : > "getPointerToNamedFunction" to get native pointer on function. This function > used the TheJit static pointer. To keep alive this functionality, I have > added a static SmallVector<JIT*> JitPool (with its mutex JitPoolLock), which > keep pointers of available JIT instance. A new static method on JIT object > "getPointerToNamedFunctionOnJitPool" fills the need (iterate over available > JIT to find the function). > > Attached you will find a test, based on "HowToUseJit" example, which > instantiate 2 JIT, in eager/lazy mode and retrieve native pointer via > getPointerToNamedFunction function. Where does it belong ? Should I add > something to llvm test suite ? To the test directory ? > > Let me know what you think ! > Thanks. > > Olivier. > > On Thu, Feb 4, 2010 at 9:03 PM, Jeffrey Yasskin <jyasskin at google.com> wrote: >> >> In eager compilation mode, I don't know of anything that would go >> wrong with having multiple JITs in the process. However, in lazy >> compilation mode, we need to map stub addresses to the JIT that knows >> how to compile them. Right now, that's done by looking up the static >> "TheJITResolver" variable and assuming it's the only JIT, but we could >> 1) use a static DenseMap<stub_address, JITResolver*> instead, or 2) >> include the JITResolver* inside the stub as an argument to the >> compilation callback. Nobody's needed this enough to get it working >> though. >> >> I think it'd make some sense to fix it for eager compilation even >> before getting lazy compilation working. Would you like to write a >> patch? You'll have to remove all uses of TheJIT and TheJITResolver >> except for the one in JITResolver::JITCompilerFn, and change the "only >> one JIT" check to say something about "only one JIT compiling lazily". >> >> I don't think this change will require passing an LLVMContext to the >> JIT--it should just use the Context of the function it's jitting. >> >> The code freeze for llvm-2.7 is on Feb 21 (this probably isn't a >> "major" change), so if you want it in the 2.7 release, please try to >> mail the patch well ahead of that. >> >> On Thu, Feb 4, 2010 at 5:47 AM, Olivier Meurant >> <meurant.olivier at gmail.com> wrote: >> > Hi everyone ! >> > >> > If I call ExecutionEngine::createJIT (or EngineBuilder::create) more >> > than >> > one time, the second time fails on a assertion "Multiple JIT >> > resolvers?". >> > It seems that the JIT is designed to be a singleton in the process, and >> > I >> > was wondering if it was something mandatory. >> > How hard will it be to make it a non-singleton object ? Is this a >> > JIT-only >> > problem (work needed on JIT classes only) ? Is it a much wider design >> > constraint (on codegen ?) ? >> > >> > In case, if you ask why I want to create more than one JIT, here is a >> > reason >> > : >> > I have a JIT application which works fine, a thread is responsible to >> > prepare code execution (work on module, add functions, optimize, call >> > JIT...) and a another thread is responsible for executing the code >> > produced >> > in the first thread. >> > It works great, but the preparation thread is heavily loaded. I decide >> > to >> > look at what happens, if I spawns 2 threads for preparation of the code. >> > I have a working prototype, which have one module and one context by >> > thread, >> > and it works. But the JIT is still shared between 2 threads and the time >> > needed to emit a function is not negligible. It will be a great boost of >> > performances, if I can create one execution engine by thread. >> > >> > What do you think ? Do you have any comments on it ? >> > Thanks for reading ! >> > >> > Olivier. >> > >> > _______________________________________________ >> > LLVM Developers mailing list >> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > >> > > >
Thanks Jeffrey ! If possible, keep me inform (on revision number), I'm interested to see how you will do the unit test. (For my future patch... :) ). Thanks again. Olivier. On Wed, Feb 10, 2010 at 6:22 PM, Jeffrey Yasskin <jyasskin at google.com>wrote:> Thanks for the patch! I'll clean this up, convert your sample to a > unit test, and commit it for 2.7. > > On Sun, Feb 7, 2010 at 6:09 AM, Olivier Meurant > <meurant.olivier at gmail.com> wrote: > > Hi Jeffrey, > > > > Thanks for pointing me in the right direction ! > > I'm not using the JIT in lazy mode, but it was fun to understand the > > lazy-stub code. > > > > Attached you will find a patch which follow your 1st option : a map > > Stub_address -> JITResolver instance, except that the used map is a > > "std::map" to apply the same upper_bound trick as in the map > > CallSiteToFunctionMap of the ResolverState. (If it is necessary for > > call_site -> function, this should be necessary for call_site -> > resolver... > > Event if I'm not sure to master all subtle reasons - code > alignment/prologue > > ?) > > A mutex on the StubToResolverMap should prevent any possible race > > conditions. > > > > Every object (Emitter, Resolver, ResolverState) have now a private member > > instance of JIT (JIT *Jit). > > > > The bugpoint tool is using a extern 'C' function : > > "getPointerToNamedFunction" to get native pointer on function. This > function > > used the TheJit static pointer. To keep alive this functionality, I have > > added a static SmallVector<JIT*> JitPool (with its mutex JitPoolLock), > which > > keep pointers of available JIT instance. A new static method on JIT > object > > "getPointerToNamedFunctionOnJitPool" fills the need (iterate over > available > > JIT to find the function). > > > > Attached you will find a test, based on "HowToUseJit" example, which > > instantiate 2 JIT, in eager/lazy mode and retrieve native pointer via > > getPointerToNamedFunction function. Where does it belong ? Should I add > > something to llvm test suite ? To the test directory ? > > > > Let me know what you think ! > > Thanks. > > > > Olivier. > > > > On Thu, Feb 4, 2010 at 9:03 PM, Jeffrey Yasskin <jyasskin at google.com> > wrote: > >> > >> In eager compilation mode, I don't know of anything that would go > >> wrong with having multiple JITs in the process. However, in lazy > >> compilation mode, we need to map stub addresses to the JIT that knows > >> how to compile them. Right now, that's done by looking up the static > >> "TheJITResolver" variable and assuming it's the only JIT, but we could > >> 1) use a static DenseMap<stub_address, JITResolver*> instead, or 2) > >> include the JITResolver* inside the stub as an argument to the > >> compilation callback. Nobody's needed this enough to get it working > >> though. > >> > >> I think it'd make some sense to fix it for eager compilation even > >> before getting lazy compilation working. Would you like to write a > >> patch? You'll have to remove all uses of TheJIT and TheJITResolver > >> except for the one in JITResolver::JITCompilerFn, and change the "only > >> one JIT" check to say something about "only one JIT compiling lazily". > >> > >> I don't think this change will require passing an LLVMContext to the > >> JIT--it should just use the Context of the function it's jitting. > >> > >> The code freeze for llvm-2.7 is on Feb 21 (this probably isn't a > >> "major" change), so if you want it in the 2.7 release, please try to > >> mail the patch well ahead of that. > >> > >> On Thu, Feb 4, 2010 at 5:47 AM, Olivier Meurant > >> <meurant.olivier at gmail.com> wrote: > >> > Hi everyone ! > >> > > >> > If I call ExecutionEngine::createJIT (or EngineBuilder::create) more > >> > than > >> > one time, the second time fails on a assertion "Multiple JIT > >> > resolvers?". > >> > It seems that the JIT is designed to be a singleton in the process, > and > >> > I > >> > was wondering if it was something mandatory. > >> > How hard will it be to make it a non-singleton object ? Is this a > >> > JIT-only > >> > problem (work needed on JIT classes only) ? Is it a much wider design > >> > constraint (on codegen ?) ? > >> > > >> > In case, if you ask why I want to create more than one JIT, here is a > >> > reason > >> > : > >> > I have a JIT application which works fine, a thread is responsible to > >> > prepare code execution (work on module, add functions, optimize, call > >> > JIT...) and a another thread is responsible for executing the code > >> > produced > >> > in the first thread. > >> > It works great, but the preparation thread is heavily loaded. I decide > >> > to > >> > look at what happens, if I spawns 2 threads for preparation of the > code. > >> > I have a working prototype, which have one module and one context by > >> > thread, > >> > and it works. But the JIT is still shared between 2 threads and the > time > >> > needed to emit a function is not negligible. It will be a great boost > of > >> > performances, if I can create one execution engine by thread. > >> > > >> > What do you think ? Do you have any comments on it ? > >> > Thanks for reading ! > >> > > >> > Olivier. > >> > > >> > _______________________________________________ > >> > LLVM Developers mailing list > >> > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > >> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >> > > >> > > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20100210/d346b328/attachment.html>