Hi Koffie, I'd highly recommend switching to ORC from MCJIT. It is much more flexible when it comes to memory-management. 1. I took the approach of 1 execution engine with multiple modules (I'm not> removing modules once they have been added). During profiling, I noticed > that the memory usage is high with a lot of code. How can I reduce the > memory usage? Is one execution engine per module preferred? I would imagine > that this would take up more memory.Whether or not you need multiple ExecutionEngines depends on your use case, but for non-trivial use-cases yes: One ExecutionEngine per Module seems to be required. 2. When I add a module and jit it, can I invoke a remove module and still> be able to execute the compiled function?You can remove the module using ExecutionEngine::removeModule(M), but be aware that this will clear the global mappings associated with the Module, so any interfaces that use IR will no longer function. 3. Is it better to have one runtime mem manager that will be associated> with multiple execution engines (one engine per module)? Using this > approach I could throw away the execution engines once they have jitted the > code. I probably need to take ownership of the jitted code somehow. Are > there any examples available on how this could be achieved?The MemoryManager owns the JIT'd code. As long as it is alive the code will hang around. You will need to keep the symbol mappings if you need them. If you switch to ORC this tends to be easier: You have one stack and can add as many modules to it as you like. The type of Module pointer that you add determines the ownership semantics: Pass a raw pointer and the JIT does not take ownership. Pass a std::unique_ptr<Module> and the JIT does own the IR. Pass a shared_ptr<Module> and it's shared. The JIT will release the IR as soon as it is able to. ORC also frees the relocatable objects as soon as possible (which MCJIT does not do), so it's likely to have better memory performance over all. - Lang. On Mon, Jul 25, 2016 at 8:07 AM, David Blaikie <dblaikie at gmail.com> wrote:> +Lang for JIT things. > > On Sun, Jul 24, 2016 at 8:50 AM koffie drinker via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> Hi all, >> >> I'm building a runtime that can jit and execute code. I've followed the >> kaleidoscope tutorial and had a couple of questions. Basically I have a >> pre-compiler that compiles the code to cache objects. These cached objects >> are then persisted and used to reduce Jit compile time. >> >> 1. I took the approach of 1 execution engine with multiple modules (I'm >> not removing modules once they have been added). During profiling, I >> noticed that the memory usage is high with a lot of code. How can I reduce >> the memory usage? Is one execution engine per module preferred? I would >> imagine that this would take up more memory. >> >> 2. When I add a module and jit it, can I invoke a remove module and still >> be able to execute the compiled function? >> >> 3. Is it better to have one runtime mem manager that will be associated >> with multiple execution engines (one engine per module)? Using this >> approach I could throw away the execution engines once they have jitted the >> code. I probably need to take ownership of the jitted code somehow. Are >> there any examples available on how this could be achieved? >> >> cheers, >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160729/1ea4271a/attachment.html>
Hi Lang, I'm at nearing release deadline, so switching at this time is probably not a good idea. Unless Orc and MCJit are almost api compatible. Are there any docs available for making the transition ? With regards to the multiple ExecutionEngine: I have a runtime that can accept code that is known in advance (about 90% of the code). I'm able to cache the object to reduce jit time. For now I'm using one execution engine with multiple modules. What benefits would I gain with an execution engine per module ? I think this will be harder and more memory expensive, since code from different module can invoke each other. So MCJit is known for keeping memory used I assume (relocatable objects)? I will try to figure out which part of MCJiT is using the memory. Any pointers to keep at the back of my head during this ? On Fri, Jul 29, 2016 at 11:51 PM, Lang Hames <lhames at gmail.com> wrote:> Hi Koffie, > > I'd highly recommend switching to ORC from MCJIT. It is much more flexible > when it comes to memory-management. > > 1. I took the approach of 1 execution engine with multiple modules (I'm >> not removing modules once they have been added). During profiling, I >> noticed that the memory usage is high with a lot of code. How can I reduce >> the memory usage? Is one execution engine per module preferred? I would >> imagine that this would take up more memory. > > > Whether or not you need multiple ExecutionEngines depends on your > use case, but for non-trivial use-cases yes: One ExecutionEngine per Module > seems to be required. > > 2. When I add a module and jit it, can I invoke a remove module and still >> be able to execute the compiled function? > > > You can remove the module using ExecutionEngine::removeModule(M), but be > aware that this will clear the global mappings associated with the Module, > so any interfaces that use IR will no longer function. > > 3. Is it better to have one runtime mem manager that will be associated >> with multiple execution engines (one engine per module)? Using this >> approach I could throw away the execution engines once they have jitted the >> code. I probably need to take ownership of the jitted code somehow. Are >> there any examples available on how this could be achieved? > > > The MemoryManager owns the JIT'd code. As long as it is alive the code > will hang around. You will need to keep the symbol mappings if you need > them. > > > If you switch to ORC this tends to be easier: You have one stack and can > add as many modules to it as you like. The type of Module pointer that you > add determines the ownership semantics: Pass a raw pointer and the JIT does > not take ownership. Pass a std::unique_ptr<Module> and the JIT does own the > IR. Pass a shared_ptr<Module> and it's shared. The JIT will release the IR > as soon as it is able to. > > ORC also frees the relocatable objects as soon as possible (which MCJIT > does not do), so it's likely to have better memory performance over all. > > - Lang. > > > On Mon, Jul 25, 2016 at 8:07 AM, David Blaikie <dblaikie at gmail.com> wrote: > >> +Lang for JIT things. >> >> On Sun, Jul 24, 2016 at 8:50 AM koffie drinker via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> Hi all, >>> >>> I'm building a runtime that can jit and execute code. I've followed the >>> kaleidoscope tutorial and had a couple of questions. Basically I have a >>> pre-compiler that compiles the code to cache objects. These cached objects >>> are then persisted and used to reduce Jit compile time. >>> >>> 1. I took the approach of 1 execution engine with multiple modules (I'm >>> not removing modules once they have been added). During profiling, I >>> noticed that the memory usage is high with a lot of code. How can I reduce >>> the memory usage? Is one execution engine per module preferred? I would >>> imagine that this would take up more memory. >>> >>> 2. When I add a module and jit it, can I invoke a remove module and >>> still be able to execute the compiled function? >>> >>> 3. Is it better to have one runtime mem manager that will be associated >>> with multiple execution engines (one engine per module)? Using this >>> approach I could throw away the execution engines once they have jitted the >>> code. I probably need to take ownership of the jitted code somehow. Are >>> there any examples available on how this could be achieved? >>> >>> cheers, >>> >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160801/43d22cbe/attachment.html>
Hi Koffie, I'm at nearing release deadline, so switching at this time is probably not> a good idea. Unless Orc and MCJit are almost api compatible. Are there any > docs available for making the transition ?Orc's functionality is a superset of MCJIT's and it's generally an easy transition, however they're not API compatible: ORC doesn't derive from ExecutionEngine. If/when you want to try porting your code I would recommend checking out the new tutorial series http://llvm.org/docs/tutorial/BuildingAJIT1.html . There are also C bindings for the new API ( http://llvm.org/docs/doxygen/html/OrcBindings_8h.html) but it sounds like you're happy with the C++ API. With regards to the multiple ExecutionEngine: I have a runtime that can> accept code that is known in advance (about 90% of the code). I'm able to > cache the object to reduce jit time. For now I'm using one execution engine > with multiple modules. > What benefits would I gain with an execution engine per module ? I think > this will be harder and more memory expensive, since code from different > module can invoke each other.Some use-cases require multiple engines, but if yours doesn't I see no advantage in switching unless you use a different memory manager for each engine: If you use a different memory for each engine you can discard individual engines after you're done with them, freeing the underlying memory. So MCJit is known for keeping memory used I assume (relocatable objects)? I> will try to figure out which part of MCJiT is using the memory. Any > pointers to keep at the back of my head during this ?MCJIT keeps the relocatable objects alive, yes. It will also keep the IR alive unless you explicitly remove it using removeModule. You might be able to figure out a way to deallocate the objects by adding new API, but I'd recommend putting that effort into porting to ORC instead - you'll get the memory benefits natively, plus the new features that are in the works. - Lang. On Mon, Aug 1, 2016 at 1:53 PM, koffie drinker <gekkekoe at gmail.com> wrote:> Hi Lang, > > I'm at nearing release deadline, so switching at this time is probably not > a good idea. Unless Orc and MCJit are almost api compatible. Are there any > docs available for making the transition ? > > With regards to the multiple ExecutionEngine: I have a runtime that can > accept code that is known in advance (about 90% of the code). I'm able to > cache the object to reduce jit time. For now I'm using one execution engine > with multiple modules. > > What benefits would I gain with an execution engine per module ? I think > this will be harder and more memory expensive, since code from different > module can invoke each other. > > So MCJit is known for keeping memory used I assume (relocatable objects)? > I will try to figure out which part of MCJiT is using the memory. Any > pointers to keep at the back of my head during this ? > > > > > On Fri, Jul 29, 2016 at 11:51 PM, Lang Hames <lhames at gmail.com> wrote: > >> Hi Koffie, >> >> I'd highly recommend switching to ORC from MCJIT. It is much more >> flexible when it comes to memory-management. >> >> 1. I took the approach of 1 execution engine with multiple modules (I'm >>> not removing modules once they have been added). During profiling, I >>> noticed that the memory usage is high with a lot of code. How can I reduce >>> the memory usage? Is one execution engine per module preferred? I would >>> imagine that this would take up more memory. >> >> >> Whether or not you need multiple ExecutionEngines depends on your >> use case, but for non-trivial use-cases yes: One ExecutionEngine per Module >> seems to be required. >> >> 2. When I add a module and jit it, can I invoke a remove module and still >>> be able to execute the compiled function? >> >> >> You can remove the module using ExecutionEngine::removeModule(M), but be >> aware that this will clear the global mappings associated with the Module, >> so any interfaces that use IR will no longer function. >> >> 3. Is it better to have one runtime mem manager that will be associated >>> with multiple execution engines (one engine per module)? Using this >>> approach I could throw away the execution engines once they have jitted the >>> code. I probably need to take ownership of the jitted code somehow. Are >>> there any examples available on how this could be achieved? >> >> >> The MemoryManager owns the JIT'd code. As long as it is alive the code >> will hang around. You will need to keep the symbol mappings if you need >> them. >> >> >> If you switch to ORC this tends to be easier: You have one stack and can >> add as many modules to it as you like. The type of Module pointer that you >> add determines the ownership semantics: Pass a raw pointer and the JIT does >> not take ownership. Pass a std::unique_ptr<Module> and the JIT does own the >> IR. Pass a shared_ptr<Module> and it's shared. The JIT will release the IR >> as soon as it is able to. >> >> ORC also frees the relocatable objects as soon as possible (which MCJIT >> does not do), so it's likely to have better memory performance over all. >> >> - Lang. >> >> >> On Mon, Jul 25, 2016 at 8:07 AM, David Blaikie <dblaikie at gmail.com> >> wrote: >> >>> +Lang for JIT things. >>> >>> On Sun, Jul 24, 2016 at 8:50 AM koffie drinker via llvm-dev < >>> llvm-dev at lists.llvm.org> wrote: >>> >>>> Hi all, >>>> >>>> I'm building a runtime that can jit and execute code. I've followed >>>> the kaleidoscope tutorial and had a couple of questions. Basically I have a >>>> pre-compiler that compiles the code to cache objects. These cached objects >>>> are then persisted and used to reduce Jit compile time. >>>> >>>> 1. I took the approach of 1 execution engine with multiple modules (I'm >>>> not removing modules once they have been added). During profiling, I >>>> noticed that the memory usage is high with a lot of code. How can I reduce >>>> the memory usage? Is one execution engine per module preferred? I would >>>> imagine that this would take up more memory. >>>> >>>> 2. When I add a module and jit it, can I invoke a remove module and >>>> still be able to execute the compiled function? >>>> >>>> 3. Is it better to have one runtime mem manager that will be associated >>>> with multiple execution engines (one engine per module)? Using this >>>> approach I could throw away the execution engines once they have jitted the >>>> code. I probably need to take ownership of the jitted code somehow. Are >>>> there any examples available on how this could be achieved? >>>> >>>> cheers, >>>> >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> llvm-dev at lists.llvm.org >>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>>> >>> >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160801/ac3027e2/attachment.html>