Hi,
I ran into a problem migrating cling (finally!) to MCJIT: When an
("outer") MCJIT's finalization /
llvm::RuntimeDyldImpl::resolveExternalSymbols() is called and a symbol
is not known, cling can help by loading the suitable library and
providing the symbol.
It compiles the relevant C++ header as part of loading the library. This
compilation emits symbols through the MCJIT. That "inner"
MCJIT'ing then
needs to finalize to resolve its symbols and to run the initializers. My
problem is that this also finalizes the outer MCJIT, e.g. marking *all*
memory pages as non-writable + executable - which in turn makes the
outer MCJIT bomb as soon as it tries to write the newly determined
relocation address to (now protected) memory.
Here is what I guess I'm looking for:
- outer resolveExternalSymbols() has a missing symbol,
- we compile and emit into a new "memory section"
- we finalize only that section
- we run static init of that secion
- we pass that now resolved symbol address to the outer MCJIT.
I tried to have the LinkingMemManager's ClientMM stack memory managers
upon recursion, so each of them can operate on their own memory regions.
But that seems to mess up the section counts / symbol offset tables :-(
What would you recommend me to do?
Cheers, Axel.
+Lang, owner of JITs On Thu, Jan 22, 2015 at 6:05 AM, Axel Naumann <Axel.Naumann at cern.ch> wrote:> Hi, > > I ran into a problem migrating cling (finally!) to MCJIT: When an > ("outer") MCJIT's finalization / > llvm::RuntimeDyldImpl::resolveExternalSymbols() is called and a symbol > is not known, cling can help by loading the suitable library and > providing the symbol. > > It compiles the relevant C++ header as part of loading the library. This > compilation emits symbols through the MCJIT. That "inner" MCJIT'ing then > needs to finalize to resolve its symbols and to run the initializers. My > problem is that this also finalizes the outer MCJIT, e.g. marking *all* > memory pages as non-writable + executable - which in turn makes the > outer MCJIT bomb as soon as it tries to write the newly determined > relocation address to (now protected) memory. > > Here is what I guess I'm looking for: > - outer resolveExternalSymbols() has a missing symbol, > - we compile and emit into a new "memory section" > - we finalize only that section > - we run static init of that secion > - we pass that now resolved symbol address to the outer MCJIT. > > I tried to have the LinkingMemManager's ClientMM stack memory managers > upon recursion, so each of them can operate on their own memory regions. > But that seems to mess up the section counts / symbol offset tables :-( > > What would you recommend me to do? > > Cheers, Axel. > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150122/d0a121d9/attachment.html>
Hi Axel, You could use multiple MCJIT instances, each with its own RTDyldMemoryManager instance. That would allow you to selectively finalize memory. Alternatively you could avoid finalizing the inner MCJIT (I don't think that's necessary for resolving symbols), and defer the initializer calls until after you've finalized the outer one? Possibly of interest: I came across the same problem while working on the new Orc APIs, and there I opted for a version of the first solution: Each Module actually gets its own RTDyldMemoryManager instance, so they can be finalized independently. Cheers, Lang. On Thu, Jan 22, 2015 at 9:25 AM, David Blaikie <dblaikie at gmail.com> wrote:> +Lang, owner of JITs > > On Thu, Jan 22, 2015 at 6:05 AM, Axel Naumann <Axel.Naumann at cern.ch> > wrote: > >> Hi, >> >> I ran into a problem migrating cling (finally!) to MCJIT: When an >> ("outer") MCJIT's finalization / >> llvm::RuntimeDyldImpl::resolveExternalSymbols() is called and a symbol >> is not known, cling can help by loading the suitable library and >> providing the symbol. >> >> It compiles the relevant C++ header as part of loading the library. This >> compilation emits symbols through the MCJIT. That "inner" MCJIT'ing then >> needs to finalize to resolve its symbols and to run the initializers. My >> problem is that this also finalizes the outer MCJIT, e.g. marking *all* >> memory pages as non-writable + executable - which in turn makes the >> outer MCJIT bomb as soon as it tries to write the newly determined >> relocation address to (now protected) memory. >> >> Here is what I guess I'm looking for: >> - outer resolveExternalSymbols() has a missing symbol, >> - we compile and emit into a new "memory section" >> - we finalize only that section >> - we run static init of that secion >> - we pass that now resolved symbol address to the outer MCJIT. >> >> I tried to have the LinkingMemManager's ClientMM stack memory managers >> upon recursion, so each of them can operate on their own memory regions. >> But that seems to mess up the section counts / symbol offset tables :-( >> >> What would you recommend me to do? >> >> Cheers, Axel. >> >> >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150126/818ed636/attachment.html>