I have an app that's dynamically generating and JITing code, and will have many such cases in the course of its run. They need to be JITed separately, because I need to execute the first batch before I know what the second one will be. Of course, I *still* need to execute the first after the need for the second arises, so I need to retain the JITed machine code for all the functions I dynamically compile. I think I just discovered the hard way that I can't keep adding to a single Module and ExecutionEngine, and re-optimizing -- certain combinations of optimization passes evidently are not happy doing this. In particular, I seem to have special problems with templated (but not explicitly 'inline') C++ functions, when llvm inlining passes are used. It seems to JIT fine, no errors, but I end up with my JITed code crashing upon execution (seg faults, and occasionally unknown opcodes). First question: can anybody confirm my theory, that it is not in fact safe/proper to add more functions to an already-optimized Module/ExecutionEngine and then perform more passes on it again? Assuming that is the case... I'm very concerned about leaks and resource usage (including memory) if I need to create a new module and/or EE for every one of these sets of functions that I need to dynamically compile. Second question: what's the best way to proceed? What's the approved way to completely free a Module/EE? And can I do so and still retain the JITed machine code to continue executing? -- Larry Gritz lg at larrygritz.com (a.k.a. lg at imageworks.com)
On Aug 16, 2010, at 3:47 PM, Larry Gritz wrote:> I have an app that's dynamically generating and JITing code, and will have many such cases in the course of its run. They need to be JITed separately, because I need to execute the first batch before I know what the second one will be. Of course, I *still* need to execute the first after the need for the second arises, so I need to retain the JITed machine code for all the functions I dynamically compile. > > I think I just discovered the hard way that I can't keep adding to a single Module and ExecutionEngine, and re-optimizing -- certain combinations of optimization passes evidently are not happy doing this. In particular, I seem to have special problems with templated (but not explicitly 'inline') C++ functions, when llvm inlining passes are used. It seems to JIT fine, no errors, but I end up with my JITed code crashing upon execution (seg faults, and occasionally unknown opcodes).In principle this ought to work, if you're careful. Are you sure you're not generating code that calls into functions that got totally inlined away? Are you running the Verifier pass at regular intervals? --Owen
On Aug 17, 2010, at 10:11 AM, Owen Anderson wrote:> In principle this ought to work, if you're careful. Are you sure you're not generating code that calls into functions that got totally inlined away?How would I know?> Are you running the Verifier pass at regular intervals?Yes, both before and after the set of optimization passes. So let me clarify what I'm doing: I have a whole bunch of C++ functions (including some inline and templated functions) that the dynamically-compiled user code may need, so I precompile that into bitcode with llvm-g++, and seed the Module with that bitcode, then add the user code (translated by our app from a custom programming language into LLVM IR) and then optimize. But I need to do this several times, for several user programs, which can come along irregularly during the execution of our app, and I never want to throw away the JITed machine code, though once I JIT I should not need the IR I generated for the user program. Also, an update: I've found that I'm totally safe if I create (and seed) a new Module and ExecutionEngine every time I need to JIT user code. But that's very slow and also a pig on memory use since I never free the Module or EE. I also found by experimentation that I seem to be able to keep a single Module and merely make a new EE for each new bit of user code -- but that is only safe if I move certain routines (that the user code may call) from the llvm-g++-ified code into a module statically compiled by g++. That is, if those few routines are just function calls to LLVM and not attempted to be inlined, all seems well as long as I don't reuse the EE's. This approach seems stable for now and saves most of the memory, but still smells like I'm working around an LLVM bug. So far I've had a hard time actually reproducing a crash in anything other than our full app, but if somebody wants to work with me on tracking the underlying bug, I can try to narrow it down to a simple example. In the mean time, perhaps somebody can answer these questions for me: who owns the machine code that is returned by ExecutionEngine::getPointerToFunction? Is that "static" once JITed? Owned by the EE? If I delete the Module and/or EE after JITing, can I still call the JITed code? If not, may I suggest as a future feature either the ability for the client app to take complete ownership of the JIT code, or else to ask the Module/EE to release as many resources and memory as possible except the callable JIT code? -- Larry Gritz lg at larrygritz.com
Hi Larry, I faced the same issue and solved it by writing my own JITMemoryManager. This is the class that is responsible for allocating memory for the JITed functions. So your derived implementation can let the application take ownership of this memory before destructing the entire execution engine. createJIT takes a JITMemoryManager as parameter, so it's straightforward to make it use your derived class. Good luck! Nicolas -----Original Message----- From: llvmdev-bounces at cs.uiuc.edu [mailto:llvmdev-bounces at cs.uiuc.edu] On Behalf Of Larry Gritz Sent: Tuesday, August 17, 2010 0:47 To: llvmdev at cs.uiuc.edu Subject: [LLVMdev] Module management questions I have an app that's dynamically generating and JITing code, and will have many such cases in the course of its run. They need to be JITed separately, because I need to execute the first batch before I know what the second one will be. Of course, I *still* need to execute the first after the need for the second arises, so I need to retain the JITed machine code for all the functions I dynamically compile. I think I just discovered the hard way that I can't keep adding to a single Module and ExecutionEngine, and re-optimizing -- certain combinations of optimization passes evidently are not happy doing this. In particular, I seem to have special problems with templated (but not explicitly 'inline') C++ functions, when llvm inlining passes are used. It seems to JIT fine, no errors, but I end up with my JITed code crashing upon execution (seg faults, and occasionally unknown opcodes). First question: can anybody confirm my theory, that it is not in fact safe/proper to add more functions to an already-optimized Module/ExecutionEngine and then perform more passes on it again? Assuming that is the case... I'm very concerned about leaks and resource usage (including memory) if I need to create a new module and/or EE for every one of these sets of functions that I need to dynamically compile. Second question: what's the best way to proceed? What's the approved way to completely free a Module/EE? And can I do so and still retain the JITed machine code to continue executing? -- Larry Gritz lg at larrygritz.com (a.k.a. lg at imageworks.com) _______________________________________________ LLVM Developers mailing list LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev