On Aug 17, 2010, at 10:11 AM, Owen Anderson wrote:> In principle this ought to work, if you're careful. Are you sure you're not generating code that calls into functions that got totally inlined away?How would I know?> Are you running the Verifier pass at regular intervals?Yes, both before and after the set of optimization passes. So let me clarify what I'm doing: I have a whole bunch of C++ functions (including some inline and templated functions) that the dynamically-compiled user code may need, so I precompile that into bitcode with llvm-g++, and seed the Module with that bitcode, then add the user code (translated by our app from a custom programming language into LLVM IR) and then optimize. But I need to do this several times, for several user programs, which can come along irregularly during the execution of our app, and I never want to throw away the JITed machine code, though once I JIT I should not need the IR I generated for the user program. Also, an update: I've found that I'm totally safe if I create (and seed) a new Module and ExecutionEngine every time I need to JIT user code. But that's very slow and also a pig on memory use since I never free the Module or EE. I also found by experimentation that I seem to be able to keep a single Module and merely make a new EE for each new bit of user code -- but that is only safe if I move certain routines (that the user code may call) from the llvm-g++-ified code into a module statically compiled by g++. That is, if those few routines are just function calls to LLVM and not attempted to be inlined, all seems well as long as I don't reuse the EE's. This approach seems stable for now and saves most of the memory, but still smells like I'm working around an LLVM bug. So far I've had a hard time actually reproducing a crash in anything other than our full app, but if somebody wants to work with me on tracking the underlying bug, I can try to narrow it down to a simple example. In the mean time, perhaps somebody can answer these questions for me: who owns the machine code that is returned by ExecutionEngine::getPointerToFunction? Is that "static" once JITed? Owned by the EE? If I delete the Module and/or EE after JITing, can I still call the JITed code? If not, may I suggest as a future feature either the ability for the client app to take complete ownership of the JIT code, or else to ask the Module/EE to release as many resources and memory as possible except the callable JIT code? -- Larry Gritz lg at larrygritz.com
On Tue, Aug 17, 2010 at 5:04 PM, Larry Gritz <lg at larrygritz.com> wrote:> On Aug 17, 2010, at 10:11 AM, Owen Anderson wrote: > >> In principle this ought to work, if you're careful. Are you sure you're not generating code that calls into functions that got totally inlined away? > > How would I know? > >> Are you running the Verifier pass at regular intervals? > > Yes, both before and after the set of optimization passes. > > So let me clarify what I'm doing: I have a whole bunch of C++ functions (including some inline and templated functions) that the dynamically-compiled user code may need, so I precompile that into bitcode with llvm-g++, and seed the Module with that bitcode, then add the user code (translated by our app from a custom programming language into LLVM IR) and then optimize. But I need to do this several times, for several user programs, which can come along irregularly during the execution of our app, and I never want to throw away the JITed machine code, though once I JIT I should not need the IR I generated for the user program.The llvm::Function object cannot be destroyed, because it is still used. However, you can free the IR by calling F->deleteBody() to save memory.> Also, an update: I've found that I'm totally safe if I create (and seed) a new Module and ExecutionEngine every time I need to JIT user code. But that's very slow and also a pig on memory use since I never free the Module or EE. > > I also found by experimentation that I seem to be able to keep a single Module and merely make a new EE for each new bit of user code -- but that is only safe if I move certain routines (that the user code may call) from the llvm-g++-ified code into a module statically compiled by g++. That is, if those few routines are just function calls to LLVM and not attempted to be inlined, all seems well as long as I don't reuse the EE's. This approach seems stable for now and saves most of the memory, but still smells like I'm working around an LLVM bug. > > So far I've had a hard time actually reproducing a crash in anything other than our full app, but if somebody wants to work with me on tracking the underlying bug, I can try to narrow it down to a simple example. > > In the mean time, perhaps somebody can answer these questions for me: who owns the machine code that is returned by ExecutionEngine::getPointerToFunction? Is that "static" once JITed? Owned by the EE? If I delete the Module and/or EE after JITing, can I still call the JITed code? If not, may I suggest as a future feature either the ability for the client app to take complete ownership of the JIT code, or else to ask the Module/EE to release as many resources and memory as possible except the callable JIT code?You can free the machine code yourself by saying EE->freeMachineCodeForFunction(F) . If you destroy the EE, it will also free the machine code. Reid
On Aug 18, 2010, at 10:24 AM, Reid Kleckner wrote:> You can free the machine code yourself by saying > EE->freeMachineCodeForFunction(F) . If you destroy the EE, it will > also free the machine code.Thanks, but unfortunately, this is exactly the opposite of what I want to do. I need to retain the machine code indefinitely, but I want to free all possible other resources that are not strictly needed simply to jump into the machine code and execute it. In particular, the Modules appear to be the culprit in the memory consumption. If I call ExecutionEngine::removeModule (after JITing a function from the module), does that mean that I can subsequently no longer call the function? Or can I? Can I delete the Module at that point and still use the machine code? -- Larry Gritz lg at larrygritz.com