search for: lazily

TL;DR - when linking from a lazily loaded module and using Linker::LinkOnlyNeeded, bodies of used functions aren't being copied during linking. Previously on one of our products, we would lazily load our runtime module (around 9000 functions), and link some user module into this (which is in all practical use cases much sma...

...added the LinkOnlyNeeded flag. On Wed, Apr 20, 2016 at 9:18 AM, Mehdi Amini via llvm-dev < llvm-dev at lists.llvm.org> wrote: > Hi Neil, > > On Apr 20, 2016, at 5:20 AM, Neil Henning via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > TL;DR - when linking from a lazily loaded module and using > Linker::LinkOnlyNeeded, bodies of used functions aren't being copied during > linking. > > Previously on one of our products, we would lazily load our runtime module > (around 9000 functions), and link some user module into this (which is in > all pra...

Hey all, For LinkModules, /*dest*/ is a fully materialized module, /*src*/ is a lazily loaded module. From what I understood, getLinkedToGlobal() is finding the function in /*src*/ that matches some function declaration in /*dest*/, and given that /*src*/ is lazily loaded it could be un-materialized. The functions I need brought in from /*src*//**/ into /*dest*/ are always decl...

...n led to an assertion failure. I was able to work around this by creating a new function pass, run by the pass manager in KaleidoscopeJIT, which looks up any called functions, sees if they're from the loaded module, and then materializes them. Is this the best way to handle JITing code from a lazily loaded module or is there a better way to handle this? It seems like this could be handled easily by moveFunctionBody if it just did something like: if (OrigF.isMaterializable()) { OrigF.materialize(); } Is there a way to add a hook into the JIT to handle this kind of materialization? T...

> > > I understood from his description that he reversed the destination and > source so that destination is the user code. > I assumed it was not lazy loaded, but that would explain the question then > :) > > Neil: can you clarify? If Teresa is right, why aren't you materializing > the destination module entirely? > > I don't think it has ever been tried

Hello, As the title line says, I'm getting assertions on recursive JIT compiation, like this: opt: JIT.cpp:627: void llvm::JIT::runJITOnFunctionUnlocked(llvm::Function*, const llvm::MutexGuard&): Assertion `!isAlreadyCodeGenerating && "Error: Recursive compilation detected!"' failed. 0 libLLVM-2.7.so 0x012eb2c8 Stack dump: 0. Program arguments: opt

...gt; system and I would mostly have to do the work myself. ModuleProvider is a very simple concept. You can either load an entire module at once (e.g. with ParseBitcodeFile) or you can load just the outline and load functions on demand (with getBitcodeModuleProvider). This allows clients to lazily load functions as they are needed instead of loading everything off the disk. The JIT is one example that knows how to use this, it lazily loads functions the first time a function is called or its address is taken. > What is the preferred way to inject a llvm:Function which contains...

...sm("section$end$__DATA$__my"); void examineSection() { const struct stuff* p; for (p = stuff_start; p < stuff_end; ++p) { // do stuff with p } } That is, there are magic symbol names which reference the beginning or ending of any particular section. To support this, the linker lazily creates atoms when references to these magic symbols are discovered during resolving. I have some hooks for this already in place in lld: 1) There is Writer::addFiles(). This method gives any writer a change to add files/atoms to the set of atoms the Resolver works on. The Writer::addFiles()...

...e { double a; } and class Two: public class Two { static double getDoubleFromOne(One arg) { return One.a; } } Here's the bytecode generated for the getDoubleFromOne method: ldarg 0 getfield "One", "a" return What happens in Java is that types are created lazily. Which means you can compile getDoubleFromOne without knowing the layout of the class One. Therefore, if you compile getDoubleFromOne without the layout information, the compiler will generate the code: r2 = arg0 r1 = getElementOffsetOf("One", "a"); return r1(r2); getElementOf...

...his has both plusses and minuses. I did wonder how much it costs to have names everywhere...) I think libjit only has one technical idea that is missing from LLVM. In libjit you can create a new function and get a pointer to it, but set things up so that the IR for the function is also created lazily. As I understand it, right now in LLVM you can make the IR and lazily compile it, but not lazily make the IR. This seems pretty handy, at least for my situation. It also looks pretty easy to add to LLVM :-) I don't want to get you down or anything. LLVM has many advantages over libjit as w...

Hi Nick, We have few symbols like __bss_start, __bss_end, which are Undefined symbols in the code. I want a way in the Reader to create specific atoms before the linker bootstraps. I didnt find a way to do that with the existing interfaces. The way it needs to work is as below :- 1) ReaderELF creates Absolute symbols (for __bss_start, __bss_end etc) 2) ReaderELF reads each file and adds

...;> example: >> >> BitcodeReader R(MBRef); >> Expected<bool> B = R.hasGlobalValueSummary(); What's this used for? Would there be a "readGlobalValueSummary()" similar to function summaries? >> std::unique_ptr<Module> M1 = R.getLazyModule(Ctx); // lazily load the >> first module >> R.next(); >> std::unique_ptr<Module> M2 = R.parseBitcodeFile(Ctx); // eagerly load the >> second module I'm very excited about the idea of storing multiple modules in a bitcode file, and the (thin)LTO and CFI goodness you're buildin...

...function on BitcodeReader. We would also have a next() > member function which would move to the next module in the file. For > example: > > BitcodeReader R(MBRef); > Expected<bool> B = R.hasGlobalValueSummary(); > std::unique_ptr<Module> M1 = R.getLazyModule(Ctx); // lazily load the > first module > R.next(); > std::unique_ptr<Module> M2 = R.parseBitcodeFile(Ctx); // eagerly load the > second module > > > > That makes the API quite stateful, you may have good implementation reason > for this, but they’re not clear to me. > I rather...

...the work myself. > > ModuleProvider is a very simple concept.  You can > either load an  > entire module at once (e.g. with ParseBitcodeFile) or you > can load  > just the outline and load functions on demand (with  > getBitcodeModuleProvider).  This allows clients to > lazily load  > functions as they are needed instead of loading everything > off the  > disk.  The JIT is one example that knows how to use > this, it lazily  > loads functions the first time a function is called or its > address is  > taken. > > > What is the preferred w...

...; > Here's a simple example: consider class One: > > public class One { > double a; > } > > and class Two: > public class Two { > static double getDoubleFromOne(One arg) { > return One.a; > } > } > What happens in Java is that types are created lazily. Which means you > can compile getDoubleFromOne without knowing the layout of the class > One. Therefore, if you compile getDoubleFromOne without the layout > information, the compiler will generate the code: I'm missing something here. If you lazily compile getDoubleFromOne the fir...

...ext couple of >> days. > > > As I expected, there were issues. I fixed the following two problems: > > 1. The JIT didn't know where a call came from when it was a tail call > (jmp > doesn't push a return address). > 2. The JIT was clobbering EAX/EDX when lazily compiling a function, and > X86 fastcc wants to pass values in those registers. > > I think things are working now, give it a spin and lemme know if you > hit any problems. > > -Chris > VC++ build is broken as a result of this change. I'll get it fixed soon.

I have tracing the calls to materializeFunction in the LLVM code in hopes of determining how to properly utilize this function but from my explorations I gather it's just a hook which is called by the JIT system and I would mostly have to do the work myself. What is the preferred way to inject a llvm:Function which contains basic blocks into the Module + JIT? My understanding (perhaps

...t; > void examineSection() { > const struct stuff* p; > for (p = stuff_start; p < stuff_end; ++p) { > // do stuff with p > } > } > > That is, there are magic symbol names which reference the beginning or ending of any particular section. To support this, the linker lazily creates atoms when references to these magic symbols are discovered during resolving. > > I have some hooks for this already in place in lld: > > 1) There is Writer::addFiles(). This method gives any writer a change to add files/atoms to the set of atoms the Resolver works on. The Wr...

...> similar to function summaries? > There would be a getModuleSummaryIndex() which again would be similar to llvm::getModuleSummaryIndex(). Note that the module summary already covers all global values, not just functions. >> std::unique_ptr<Module> M1 = R.getLazyModule(Ctx); // lazily load the > >> first module > >> R.next(); > >> std::unique_ptr<Module> M2 = R.parseBitcodeFile(Ctx); // eagerly load > the > >> second module > > I'm very excited about the idea of storing multiple modules in a > bitcode file, and the (thin)...