Hello everyone, I need some advises about (re)using ExecutionEngine with MCJIT as a driver. I'm developing a service that receives a piece of high-level code, compiles it into LLVM IR function "main" and uses MCJIT to execute the function. It can happen that the same piece of code is sent to the service many times. I would like to cache the results (keep generated machine code alive) and do just the execution step. But I'm having problems with that. My first attempt was to cache ExecutionEngine instance and function address (from getFunctionAddress() method). Executing cached function second time crashes the process. I noticed that doing getFunctionAddress() each time helps a bit. There is no crash but results produced by executed function are incorrect and there are probably some memory access violations going on. Using the same function name for each code is probably a bad idea in case of MCJIT, so I changed the names to some random value. However, it did not help in any of previous problems. I thinking about using single instance of ExecutionEngine or share Memory Manager. Can I get any advice on that? Happy New Year, Paweł Bylica -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141229/a04d37ef/attachment.html>
Hi Pawel, I don't know much about MCJIT, but I did come across this blog post http://blog.llvm.org/2013/08/object-caching-with-kaleidoscope.html about caching the generated objects for subsequent execution, which apparently is a big win for MCJIT over the old jit execution engine in LLVM. There's example code in the LLVM source distro (examples/Kaleidoscope/MCJIT/ cached). HTH, Charlie. On 29 December 2014 at 13:23, Paweł Bylica <chfast at gmail.com> wrote:> Hello everyone, > > I need some advises about (re)using ExecutionEngine with MCJIT as a driver. > I'm developing a service that receives a piece of high-level code, compiles > it into LLVM IR function "main" and uses MCJIT to execute the function. > > It can happen that the same piece of code is sent to the service many times. > I would like to cache the results (keep generated machine code alive) and do > just the execution step. But I'm having problems with that. > > My first attempt was to cache ExecutionEngine instance and function address > (from getFunctionAddress() method). Executing cached function second time > crashes the process. > > I noticed that doing getFunctionAddress() each time helps a bit. There is no > crash but results produced by executed function are incorrect and there are > probably some memory access violations going on. > > Using the same function name for each code is probably a bad idea in case of > MCJIT, so I changed the names to some random value. However, it did not help > in any of previous problems. > > I thinking about using single instance of ExecutionEngine or share Memory > Manager. Can I get any advice on that? > > Happy New Year, > Paweł Bylica > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >
On 29 December 2014 at 14:47, Paweł Bylica <chfast at gmail.com> wrote:> Hi Charlie, > > Thanks for your response. I've already implemented object caching as > described in the article. It works great, but it was designed for caching > between process runs. When I'm using it now I still need to create LLVM > module (compile my code), copy some memory buffers from cache and do symbol > relocation. It saves time needed for machine code generation (huge gain) but > I believe I can do better.Ah, sorry for the noise, I'm sure someone else here will have some advice! Kind regards, Charlie.
On 12/29/2014 05:23 AM, Paweł Bylica wrote:> Hello everyone, > > I need some advises about (re)using ExecutionEngine with MCJIT as a > driver. I'm developing a service that receives a piece of high-level > code, compiles it into LLVM IR function "main" and uses MCJIT to > execute the function. > > It can happen that the same piece of code is sent to the service many > times. I would like to cache the results (keep generated machine code > alive) and do just the execution step. But I'm having problems with that. > > My first attempt was to cache ExecutionEngine instance and function > address (from getFunctionAddress() method). Executing cached function > second time crashes the process. > > I noticed that doing getFunctionAddress() each time helps a bit. There > is no crash but results produced by executed function are incorrect > and there are probably some memory access violations going on. > > Using the same function name for each code is probably a bad idea in > case of MCJIT, so I changed the names to some random value. However, > it did not help in any of previous problems. > > I thinking about using single instance of ExecutionEngine or share > Memory Manager. Can I get any advice on that?My suggestion would be to use a single long lived instance of EE and MM. Use some hashing mechanism to map your high level requests to a unique key. If you've already generated it, just reuse the existing code. Otherwise, create a new module, add it to the EE, and compile. This will cause you to "leak" memory for code that isn't being reused. I don't know of a good solution to that within the framework of MCJIT, but you can get something reasonable by simply recreating your EE and MM instances every N (10000?) compiles. Until you have something like that working, I won't worry about trying to improve the memory caching strategy. p.s. You can also look at using the on-disk cache capabilities. I have never used that and have no idea how useful it is.> Happy New Year, > Paweł Bylica > > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141229/153252f8/attachment.html>
My response would be similar to some of the previous replies. In my experience with MCJIT, the best way is to have one instance of the ExecutionEngine. Everytime you create a new llvm IR Module, add to the MCJIT ExectionEngine using addModule . It would be a good idea to create unique function names everytime you create a new llvm IR Module (I dont know what happens if there is a clash). With respect to hashing, I would implement hashing from your high-level code to the function generated outside of the MCJIT Execution Engine. Once you get the function name from your hash function you can get the function address using getFunctionAddress. On Mon, Dec 29, 2014 at 10:36 AM, Philip Reames <listmail at philipreames.com> wrote:> > On 12/29/2014 05:23 AM, Paweł Bylica wrote: > > Hello everyone, > > I need some advises about (re)using ExecutionEngine with MCJIT as a > driver. I'm developing a service that receives a piece of high-level code, > compiles it into LLVM IR function "main" and uses MCJIT to execute the > function. > > It can happen that the same piece of code is sent to the service many > times. I would like to cache the results (keep generated machine code > alive) and do just the execution step. But I'm having problems with that. > > My first attempt was to cache ExecutionEngine instance and function > address (from getFunctionAddress() method). Executing cached function > second time crashes the process. > > I noticed that doing getFunctionAddress() each time helps a bit. There > is no crash but results produced by executed function are incorrect and > there are probably some memory access violations going on. > > Using the same function name for each code is probably a bad idea in > case of MCJIT, so I changed the names to some random value. However, it did > not help in any of previous problems. > > I thinking about using single instance of ExecutionEngine or share > Memory Manager. Can I get any advice on that? > > My suggestion would be to use a single long lived instance of EE and MM. > Use some hashing mechanism to map your high level requests to a unique > key. If you've already generated it, just reuse the existing code. > Otherwise, create a new module, add it to the EE, and compile. > > This will cause you to "leak" memory for code that isn't being reused. I > don't know of a good solution to that within the framework of MCJIT, but > you can get something reasonable by simply recreating your EE and MM > instances every N (10000?) compiles. Until you have something like that > working, I won't worry about trying to improve the memory caching strategy. > > p.s. You can also look at using the on-disk cache capabilities. I have > never used that and have no idea how useful it is. > > Happy New Year, > Paweł Bylica > > > > > _______________________________________________ > LLVM Developers mailing listLLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.eduhttp://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-- Mahesh -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150104/d0fc9020/attachment.html>