Gaier, Bjoern via llvm-dev
2020-Sep-29 07:03 UTC
[llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?
Hey Lang, Thank you for your help and your patience – also for your answers in the “ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup” mail. Both problems have the same origin so I keep writing about it here, to avoid duplication. My big problem is still handling cross references between modules with “our” name scheme. Since our old loader loads object files, we resolved those references with object files and since they were already compiled, we knew all addresses right away. With the LLJIT as I finally understand, I will only get the addresses when I have resolved every references, which makes the code way safer. However, another thing of our system is, that each object file was loaded from a different process, so sometimes not all symbols for ModuleA were present because ModuleB was not loaded/requested yet. That was okay, so we kept resolving the undefined references of ModuleA until ModuleB was loaded and everything was fine. If I get it right… This would change now to having a single LLJIT representing the entire system. Each process would get it’s own DyLib for there module. However, I would need to check on IR-Level now which symbols would be undefined – correct? Because if I wait until “DefinitionGenerator::tryToGenerate” is called and have to wait for a module that might never be loaded, then I’m stuck there forever. 1.) If I find a symbol that is undefined – and it has our name scheme, then I would jump to for example ModuleB, which is also not jitted yet and would do a “replaceAllUsesWith” on the Symbol of ModuleA to ModuleB – right? - Would that mean, when I add ModuleA to DyLibA – is ModuleB then part of DyLibA as well? 1.1.) Alternatively I could rename the symbol 2.) If the ModuleB is already jitted, then I can take the address to do the “replaceAllUsesWith” right? When I resolved all those references, then I can add the IR Module to my DyLib and compile it. However is it a good idea to use “replaceAllUsesWith” with addresses? Seems like the DefinitionGenerator would be jobless… Writing this… I actually wondered about something else but this feels like a terrible approach… But now I’m curious xD So… if I load an IR-Module, could I use the static LLVM compiler to compile it to an object file and then use the source code of the LLD to load and resolve the symbols the same way/kind we did in the past? That sounds more like the “addObjectFile” function of the LLJIT… And then I guess I have to write a LinkLayer or something? That is where my knowledge ends… Disclaimer: I don’t like that approach but it would be interesting to know (also cause some people here would be happy with it .w.”) Thank you so far! Kind greetings Björn From: Lang Hames <lhames at gmail.com> Sent: 29 September 2020 01:47 To: Gaier, Bjoern <Bjoern.Gaier at horiba.com> Cc: LLVM Developers Mailing List <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup? Hi Bjoern, Even though the "tryToGenerate" function of my DefinitionGenerator returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I got an address for "? That's because you're issuing the lookup with RequiredState == SymbolState::Resolved. This means that your query will return as soon as "?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s) addresses are assigned before external references are looked up. So after your lookup returns the linker attempts to find "?_Plansch_test@@3HA", fails, and so moves "?Sampler@@YAXXZ" to the error state. You almost always want to issue your lookups with RequiredState == SymbolState::Ready. This ensures that the query will not return until / unless the requested symbols (and all their dependencies) are successfully linked into the target process and ready to execute. Question 1.) Is there any way to reset the error state of "?Sampler@@YAXXZ" at this point? No. However, the removable code feature will allow you to remove failed materialization units once it lands in the mainline. - After my first call I used the "define" function of the JITDylib to define ?_Plansch_test@@3HA and then I tried calling the lookup function again and again, however I only got the error: "Failed to materialize symbols" even though "?_Plansch_test@@3HA" was defined now... - Changing the order of the "define" and the "lookup" call works of course, but I'm interested in the case where I don't know the address yet. The JIT doesn't re-try linking. Once a symbol has failed to link it remains in the error state. In theory, once removable code is added you could choose to remove and then re-add "?Sampler@@YAXXZ" after "?_Plansch_test@@3HA" is defined. The real solution though is just to make sure that "?_Plansch_test@@3HA" is defined (either directly or via a generator) before you look up "?Sampler@@YAXXZ". Out of curiosity I repeated the previous scenario - but added "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error from my DefinitionGenerator... Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a SymbolsNotFound error sent to your error reporter in the first scenario too, followed by a failure-to-materialize error for "?Sampler@@YAXXZ". ... and "?Sampler@@YAXXZ" is stuck as a pending query in the MaterializingInfos entries. Huh. That sounds like a bug: All references to the query should be removed from the state machine before it returns its result (in this case an error). I'll see if I can reproduce this locally and fix it up, but it doesn't affect the discussion here. When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed and give me the addresses. Also I'm able to execute the code now. This is great! However... When a lookup fails we try to restore the ExecutionSession state to what it was prior to the query. This is why the sequence "lookup -> symbols not found -> define -> lookup again" worked. Question 2.) Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" like in the first scenario? I expected it would return an address for it. A lookup must match against all symbols before anything is JIT'd. When it failed to match "?_Plansch_test@@3HA" we immediately bailed out with an error. There was no further attempt to compile "?Sampler@@YAXXZ". Question 3.) Can I somehow combine both behaviours? Getting the address for all the symbols (like in scenario 1) while still being able to provide definitions later (like in scenario 2)? Sort of. Definition generators allow you to provide a definition at the last minute (i.e. in response to a query). The best mental model though is: "All definitions that a generator can generate are part of the interface of the dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols from a dynamic library containing "foo", "bar" and "baz" then you should think of your JITDylib as containing definitions for "foo", "bar" and "baz", even if the generator hasn't actually added them to the JITDylib yet. The reason is that it will add them in response to any query for them, so it's indistinguishable (except for timing and debug logging) from the case where they're already present. If you need to be able to defer adding a "real" definition beyond the initial lookup then your only option (and this only applies to functions) is a lazy-reexport. This allows you to provide a definition for a function while deferring lookup until the first execution of the re-export at runtime. I wouldn't generally use this to break dependencies though: You want a definition of the real function body for "?_Plansch_test@@3HA" already added to your JIT because (in general) you never know when JIT'd code will need it. My turn to ask a question: How is "?_Plansch_test@@3HA" created, and why not just add it up-front? :) -- Lang. On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: Hey everyone, I felt this question is different from my other question - hope this is okay. So - I was playing around with the lookup function of the ExecutionSession and there are some things I don't understand. I have a .BC file with a function "?Sampler@@YAXXZ" referencing a value "?_Plansch_test@@3HA" that is not defined in that module itself. I first planed on not providing an address for "?_Plansch_test@@3HA" but wanted to know the address of "?Sampler@@YAXXZ". So I issued something like that: auto &ES = this->jit->getExecutionSession(); SymbolLookupSet lookupSet; lookupSet.add("?Sampler@@YAXXZ", llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol); ES.lookup({{&jit->getMainJITDylib(), llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved); Even though the "tryToGenerate" function of my DefinitionGenerator returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I got an address for "?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, that the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I guess. Question 1.) Is there any way to reset the error state of "?Sampler@@YAXXZ" at this point? - After my first call I used the "define" function of the JITDylib to define ?_Plansch_test@@3HA and then I tried calling the lookup function again and again, however I only got the error: "Failed to materialize symbols" even though "?_Plansch_test@@3HA" was defined now... - Changing the order of the "define" and the "lookup" call works of course, but I'm interested in the case where I don't know the address yet. Out of curiosity I repeated the previous scenario - but added "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a pending query in the MaterializingInfos entries. When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed and give me the addresses. Also I'm able to execute the code now. This is great! However... Question 2.) Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" like in the first scenario? I expected it would return an address for it. Question 3.) Can I somehow combine both behaviours? Getting the address for all the symbols (like in scenario 1) while still being able to provide definitions later (like in scenario 2)? Thank you in advance and kind greetings, Björn Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz. _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200929/94c8bac7/attachment-0001.html>
Lang Hames via llvm-dev
2020-Sep-29 16:55 UTC
[llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?
Hi Bjoern, However, another thing of our system is, that each object file was loaded> from a different process,Do you mean that the object file is produced by another process and is being loaded into your JIT process for execution, or that you want your JIT to produce code for several different processes? These are different problems with different solutions. I'll wait until I understand your use case to answer further. Writing this… I actually wondered about something else but this feels like> a terrible approach… But now I’m curious xD > So… if I load an IR-Module, could I use the static LLVM compiler to > compile it to an object file and then use the source code of the LLD to > load and resolve the symbols the same way/kind we did in the past? > That sounds more like the “addObjectFile” function of the LLJIT… And then > I guess I have to write a LinkLayer or something? That is where my > knowledge ends… > Disclaimer: I don’t like that approach but it would be interesting to know > (also cause some people here would be happy with it .w.”)You're trying to do all this on Hard Mode. ;) ORC takes care of all this kind of stuff for you: Don't re-write IR. Leave references as symbolic -- they will be fixed up in the JIT linker. You don't need to write your own JIT linker. LLJIT has one built in. When you add things to the JIT: - If you have a program representation (module, object file, etc.) and you want to add it then just go ahead. - If your program representations contain external references then they must be resolvable or linking will fail (there's no getting around that, in a JIT or in a regular compile), BUT... - You can always add new definitions in response to a query by using a definition generator. If your definition generator can find/create a definition then great. If it can't then the reference really is unresolved and linking really should fail. The part of your use case that is the most opaque to me is the renaming. When you see a reference to "test" in some object, how do you decide that it should resolve to the definition of "test" in, for example, Planschi, as opposed to some other module? Do you just have a list of modules that you check in-order until you find a matching symbol name? -- Lang. On Tue, Sep 29, 2020 at 2:30 AM Gaier, Bjoern <Bjoern.Gaier at horiba.com> wrote:> Hey Lang, > > > > Thank you for your help and your patience – also for your answers in the > “ORC JIT - Can modules independently managed with one LLJIT instance? + > problems with ExecutionSession.lookup” mail. Both problems have the same > origin so I keep writing about it here, to avoid duplication. > > My big problem is still handling cross references between modules with > “our” name scheme. Since our old loader loads object files, we resolved > those references with object files and since they were already compiled, we > knew all addresses right away. With the LLJIT as I finally understand, I > will only get the addresses when I have resolved every references, which > makes the code way safer. > > However, another thing of our system is, that each object file was loaded > from a different process, so sometimes not all symbols for ModuleA were > present because ModuleB was not loaded/requested yet. That was okay, so we > kept resolving the undefined references of ModuleA until ModuleB was loaded > and everything was fine. > > > > If I get it right… This would change now to having a single LLJIT > representing the entire system. Each process would get it’s own DyLib for > there module. However, I would need to check on IR-Level now which symbols > would be undefined – correct? Because if I wait until > “DefinitionGenerator::tryToGenerate” is called and have to wait for a > module that might never be loaded, then I’m stuck there forever. > > 1.) If I find a symbol that is undefined – and it has our name scheme, > then I would jump to for example ModuleB, which is also not jitted yet and > would do a “replaceAllUsesWith” on the Symbol of ModuleA to ModuleB – > right? > > - Would that mean, when I add ModuleA to DyLibA – is > ModuleB then part of DyLibA as well? > > 1.1.) Alternatively I could rename the symbol > > 2.) If the ModuleB is already jitted, then I can take the address to do > the “replaceAllUsesWith” right? > > > > When I resolved all those references, then I can add the IR Module to my > DyLib and compile it. However is it a good idea to use “replaceAllUsesWith” > with addresses? Seems like the DefinitionGenerator would be jobless… > > > > Writing this… I actually wondered about something else but this feels like > a terrible approach… But now I’m curious xD > > So… if I load an IR-Module, could I use the static LLVM compiler to > compile it to an object file and then use the source code of the LLD to > load and resolve the symbols the same way/kind we did in the past? > > That sounds more like the “addObjectFile” function of the LLJIT… And then > I guess I have to write a LinkLayer or something? That is where my > knowledge ends… > Disclaimer: I don’t like that approach but it would be interesting to know > (also cause some people here would be happy with it .w.”) > > > > Thank you so far! > > > > Kind greetings > > Björn > > > > > > *From:* Lang Hames <lhames at gmail.com> > *Sent:* 29 September 2020 01:47 > *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com> > *Cc:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org> > *Subject:* Re: [llvm-dev] ORC JIT - different behaviour of > ExecutionSession.lookup? > > > > Hi Bjoern, > > > > Even though the "tryToGenerate" function of my DefinitionGenerator > returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I > got an address for "? > > > > That's because you're issuing the lookup with RequiredState => SymbolState::Resolved. This means that your query will return as soon as > "?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s) addresses > are assigned before external references are looked up. So after your lookup > returns the linker attempts to find "?_Plansch_test@@3HA", fails, and so > moves "?Sampler@@YAXXZ" to the error state. > > > > You almost always want to issue your lookups with RequiredState => SymbolState::Ready. This ensures that the query will not return until / > unless the requested symbols (and all their dependencies) are successfully > linked into the target process and ready to execute. > > > > Question 1.) > Is there any way to reset the error state of "?Sampler@@YAXXZ" at this > point? > > > > No. However, the removable code feature will allow you to remove failed > materialization units once it lands in the mainline. > > > > - After my first call I used the "define" function of the JITDylib to > define ?_Plansch_test@@3HA and then I tried calling the lookup function > again and again, however I only got the error: "Failed to materialize > symbols" even though "?_Plansch_test@@3HA" was defined now... > - Changing the order of the "define" and the "lookup" call works of > course, but I'm interested in the case where I don't know the address yet. > > > > The JIT doesn't re-try linking. Once a symbol has failed to link it > remains in the error state. In theory, once removable code is added you > could choose to remove and then re-add "?Sampler@@YAXXZ" > after "?_Plansch_test@@3HA" is defined. The real solution though is just > to make sure that "?_Plansch_test@@3HA" is defined (either directly or > via a generator) before you look up "?Sampler@@YAXXZ". > > > > Out of curiosity I repeated the previous scenario - but added > "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. > When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error > from my DefinitionGenerator... > > > > Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a > SymbolsNotFound error sent to your error reporter in the first scenario > too, followed by a failure-to-materialize error for "?Sampler@@YAXXZ". > > > > ... and "?Sampler@@YAXXZ" is stuck as a pending query in the > MaterializingInfos entries. > > > > Huh. That sounds like a bug: All references to the query should be removed > from the state machine before it returns its result (in this case an > error). I'll see if I can reproduce this locally and fix it up, but it > doesn't affect the discussion here. > > > > When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" > the second time, it will succeed and give me the addresses. Also I'm able > to execute the code now. This is great! However... > > > > When a lookup fails we try to restore the ExecutionSession state to what > it was prior to the query. This is why the sequence "lookup -> symbols not > found -> define -> lookup again" worked. > > > > Question 2.) > Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" > like in the first scenario? I expected it would return an address for it. > > > > A lookup must match against all symbols before anything is JIT'd. When it > failed to match "?_Plansch_test@@3HA" we immediately bailed out with an > error. There was no further attempt to compile "?Sampler@@YAXXZ". > > > > Question 3.) > Can I somehow combine both behaviours? Getting the address for all the > symbols (like in scenario 1) while still being able to provide definitions > later (like in scenario 2)? > > > > *Sort of.* > > > > Definition generators allow you to provide a definition at the last minute > (i.e. in response to a query). The best mental model though is: "All > definitions that a generator can generate are part of the interface of the > dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols > from a dynamic library containing "foo", "bar" and "baz" then you should > think of your JITDylib as containing definitions for "foo", "bar" and > "baz", even if the generator hasn't actually added them to the JITDylib > yet. The reason is that it will add them in response to any query for them, > so it's indistinguishable (except for timing and debug logging) from the > case where they're already present. > > > > If you need to be able to defer adding a "real" definition beyond the > initial lookup then your only option (and this only applies to functions) > is a lazy-reexport. This allows you to provide a definition for a function > while deferring lookup until the first execution of the re-export at > runtime. I wouldn't generally use this to break dependencies though: You > want a definition of the real function body for "?_Plansch_test@@3HA" > already added to your JIT because (in general) you never know when JIT'd > code will need it. > > > > My turn to ask a question: How is "?_Plansch_test@@3HA" created, and why > not just add it up-front? :) > > > > -- Lang. > > > > On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > Hey everyone, > > > > I felt this question is different from my other question - hope this is > okay. > > > > So - I was playing around with the lookup function of the ExecutionSession > and there are some things I don't understand. > > I have a .BC file with a function "?Sampler@@YAXXZ" referencing a value > "?_Plansch_test@@3HA" that is not defined in that module itself. I first > planed on not providing an address for "?_Plansch_test@@3HA" but wanted > to know the address of "?Sampler@@YAXXZ". So I issued something like that: > > > > auto &ES = this->jit->getExecutionSession(); > > SymbolLookupSet lookupSet; > > > > lookupSet.add("?Sampler@@YAXXZ", > llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol); > > ES.lookup({{&jit->getMainJITDylib(), > llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, > llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved); > > > > Even though the "tryToGenerate" function of my DefinitionGenerator > returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I > got an address for "?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, > that the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I > guess. > > > > Question 1.) > > Is there any way to reset the error state of "?Sampler@@YAXXZ" at this > point? > > - After my first call I used the "define" function of the JITDylib to > define ?_Plansch_test@@3HA and then I tried calling the lookup function > again and again, however I only got the error: "Failed to materialize > symbols" even though "?_Plansch_test@@3HA" was defined now... > > - Changing the order of the "define" and the "lookup" call works of > course, but I'm interested in the case where I don't know the address yet. > > > > Out of curiosity I repeated the previous scenario - but added > "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. > When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error > from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a pending > query in the MaterializingInfos entries. When I then add a definition for > "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed > and give me the addresses. Also I'm able to execute the code now. This is > great! However... > > > > Question 2.) > > Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" > like in the first scenario? I expected it would return an address for it. > > > > Question 3.) > > Can I somehow combine both behaviours? Getting the address for all the > symbols (like in scenario 1) while still being able to provide definitions > later (like in scenario 2)? > > > > Thank you in advance and kind greetings, > > Björn > > Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, > USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert > Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin > Cansiz. > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, > USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert > Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin > Cansiz. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200929/0d8141e3/attachment.html>
Lang Hames via llvm-dev
2020-Sep-29 18:36 UTC
[llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?
Hi Bjoern,> ... and "?Sampler@@YAXXZ" is stuck as a pending query in the >> MaterializingInfos entries. > >Huh. That sounds like a bug: All references to the query should be removed> from the state machine before it returns its result (in this case an > error). I'll see if I can reproduce this locally and fix it up, but it > doesn't affect the discussion here.I tried to reproduce this locally by defining an absolute symbol "foo", then issuing a lookup for two symbols "foo" and "bar". The error was produced as expected, but I couldn't see any queries left attached to the ExecutionSession. Did you definitely see a failed query still attached? If so, and you see this again could you log your output and file a bug? -- Lang. On Tue, Sep 29, 2020 at 9:55 AM Lang Hames <lhames at gmail.com> wrote:> Hi Bjoern, > > However, another thing of our system is, that each object file was loaded >> from a different process, > > > Do you mean that the object file is produced by another process and is > being loaded into your JIT process for execution, or that you want your JIT > to produce code for several different processes? These are different > problems with different solutions. I'll wait until I understand your use > case to answer further. > > Writing this… I actually wondered about something else but this feels like >> a terrible approach… But now I’m curious xD >> So… if I load an IR-Module, could I use the static LLVM compiler to >> compile it to an object file and then use the source code of the LLD to >> load and resolve the symbols the same way/kind we did in the past? >> That sounds more like the “addObjectFile” function of the LLJIT… And then >> I guess I have to write a LinkLayer or something? That is where my >> knowledge ends… >> Disclaimer: I don’t like that approach but it would be interesting to >> know (also cause some people here would be happy with it .w.”) > > > You're trying to do all this on Hard Mode. ;) > > ORC takes care of all this kind of stuff for you: > Don't re-write IR. Leave references as symbolic -- they will be fixed up > in the JIT linker. > You don't need to write your own JIT linker. LLJIT has one built in. > > When you add things to the JIT: > - If you have a program representation (module, object file, etc.) and > you want to add it then just go ahead. > - If your program representations contain external references then they > must be resolvable or linking will fail (there's no getting around that, in > a JIT or in a regular compile), BUT... > - You can always add new definitions in response to a query by using a > definition generator. If your definition generator can find/create a > definition then great. If it can't then the reference really is unresolved > and linking really should fail. > > The part of your use case that is the most opaque to me is the renaming. > When you see a reference to "test" in some object, how do you decide that > it should resolve to the definition of "test" in, for example, Planschi, as > opposed to some other module? Do you just have a list of modules that you > check in-order until you find a matching symbol name? > > -- Lang. > > On Tue, Sep 29, 2020 at 2:30 AM Gaier, Bjoern <Bjoern.Gaier at horiba.com> > wrote: > >> Hey Lang, >> >> >> >> Thank you for your help and your patience – also for your answers in the >> “ORC JIT - Can modules independently managed with one LLJIT instance? + >> problems with ExecutionSession.lookup” mail. Both problems have the same >> origin so I keep writing about it here, to avoid duplication. >> >> My big problem is still handling cross references between modules with >> “our” name scheme. Since our old loader loads object files, we resolved >> those references with object files and since they were already compiled, we >> knew all addresses right away. With the LLJIT as I finally understand, I >> will only get the addresses when I have resolved every references, which >> makes the code way safer. >> >> However, another thing of our system is, that each object file was loaded >> from a different process, so sometimes not all symbols for ModuleA were >> present because ModuleB was not loaded/requested yet. That was okay, so we >> kept resolving the undefined references of ModuleA until ModuleB was loaded >> and everything was fine. >> >> >> >> If I get it right… This would change now to having a single LLJIT >> representing the entire system. Each process would get it’s own DyLib for >> there module. However, I would need to check on IR-Level now which symbols >> would be undefined – correct? Because if I wait until >> “DefinitionGenerator::tryToGenerate” is called and have to wait for a >> module that might never be loaded, then I’m stuck there forever. >> >> 1.) If I find a symbol that is undefined – and it has our name scheme, >> then I would jump to for example ModuleB, which is also not jitted yet and >> would do a “replaceAllUsesWith” on the Symbol of ModuleA to ModuleB – >> right? >> >> - Would that mean, when I add ModuleA to DyLibA – is >> ModuleB then part of DyLibA as well? >> >> 1.1.) Alternatively I could rename the symbol >> >> 2.) If the ModuleB is already jitted, then I can take the address to do >> the “replaceAllUsesWith” right? >> >> >> >> When I resolved all those references, then I can add the IR Module to my >> DyLib and compile it. However is it a good idea to use “replaceAllUsesWith” >> with addresses? Seems like the DefinitionGenerator would be jobless… >> >> >> >> Writing this… I actually wondered about something else but this feels >> like a terrible approach… But now I’m curious xD >> >> So… if I load an IR-Module, could I use the static LLVM compiler to >> compile it to an object file and then use the source code of the LLD to >> load and resolve the symbols the same way/kind we did in the past? >> >> That sounds more like the “addObjectFile” function of the LLJIT… And then >> I guess I have to write a LinkLayer or something? That is where my >> knowledge ends… >> Disclaimer: I don’t like that approach but it would be interesting to >> know (also cause some people here would be happy with it .w.”) >> >> >> >> Thank you so far! >> >> >> >> Kind greetings >> >> Björn >> >> >> >> >> >> *From:* Lang Hames <lhames at gmail.com> >> *Sent:* 29 September 2020 01:47 >> *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com> >> *Cc:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org> >> *Subject:* Re: [llvm-dev] ORC JIT - different behaviour of >> ExecutionSession.lookup? >> >> >> >> Hi Bjoern, >> >> >> >> Even though the "tryToGenerate" function of my DefinitionGenerator >> returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I >> got an address for "? >> >> >> >> That's because you're issuing the lookup with RequiredState =>> SymbolState::Resolved. This means that your query will return as soon as >> "?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s) addresses >> are assigned before external references are looked up. So after your lookup >> returns the linker attempts to find "?_Plansch_test@@3HA", fails, and so >> moves "?Sampler@@YAXXZ" to the error state. >> >> >> >> You almost always want to issue your lookups with RequiredState =>> SymbolState::Ready. This ensures that the query will not return until / >> unless the requested symbols (and all their dependencies) are successfully >> linked into the target process and ready to execute. >> >> >> >> Question 1.) >> Is there any way to reset the error state of "?Sampler@@YAXXZ" at this >> point? >> >> >> >> No. However, the removable code feature will allow you to remove failed >> materialization units once it lands in the mainline. >> >> >> >> - After my first call I used the "define" function of the JITDylib to >> define ?_Plansch_test@@3HA and then I tried calling the lookup function >> again and again, however I only got the error: "Failed to materialize >> symbols" even though "?_Plansch_test@@3HA" was defined now... >> - Changing the order of the "define" and the "lookup" call works of >> course, but I'm interested in the case where I don't know the address yet. >> >> >> >> The JIT doesn't re-try linking. Once a symbol has failed to link it >> remains in the error state. In theory, once removable code is added you >> could choose to remove and then re-add "?Sampler@@YAXXZ" >> after "?_Plansch_test@@3HA" is defined. The real solution though is just >> to make sure that "?_Plansch_test@@3HA" is defined (either directly or >> via a generator) before you look up "?Sampler@@YAXXZ". >> >> >> >> Out of curiosity I repeated the previous scenario - but added >> "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. >> When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error >> from my DefinitionGenerator... >> >> >> >> Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a >> SymbolsNotFound error sent to your error reporter in the first scenario >> too, followed by a failure-to-materialize error for "?Sampler@@YAXXZ". >> >> >> >> ... and "?Sampler@@YAXXZ" is stuck as a pending query in the >> MaterializingInfos entries. >> >> >> >> Huh. That sounds like a bug: All references to the query should be >> removed from the state machine before it returns its result (in this case >> an error). I'll see if I can reproduce this locally and fix it up, but it >> doesn't affect the discussion here. >> >> >> >> When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" >> the second time, it will succeed and give me the addresses. Also I'm able >> to execute the code now. This is great! However... >> >> >> >> When a lookup fails we try to restore the ExecutionSession state to what >> it was prior to the query. This is why the sequence "lookup -> symbols not >> found -> define -> lookup again" worked. >> >> >> >> Question 2.) >> Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" >> like in the first scenario? I expected it would return an address for it. >> >> >> >> A lookup must match against all symbols before anything is JIT'd. When it >> failed to match "?_Plansch_test@@3HA" we immediately bailed out with an >> error. There was no further attempt to compile "?Sampler@@YAXXZ". >> >> >> >> Question 3.) >> Can I somehow combine both behaviours? Getting the address for all the >> symbols (like in scenario 1) while still being able to provide definitions >> later (like in scenario 2)? >> >> >> >> *Sort of.* >> >> >> >> Definition generators allow you to provide a definition at the last >> minute (i.e. in response to a query). The best mental model though is: "All >> definitions that a generator can generate are part of the interface of the >> dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols >> from a dynamic library containing "foo", "bar" and "baz" then you should >> think of your JITDylib as containing definitions for "foo", "bar" and >> "baz", even if the generator hasn't actually added them to the JITDylib >> yet. The reason is that it will add them in response to any query for them, >> so it's indistinguishable (except for timing and debug logging) from the >> case where they're already present. >> >> >> >> If you need to be able to defer adding a "real" definition beyond the >> initial lookup then your only option (and this only applies to functions) >> is a lazy-reexport. This allows you to provide a definition for a function >> while deferring lookup until the first execution of the re-export at >> runtime. I wouldn't generally use this to break dependencies though: You >> want a definition of the real function body for "?_Plansch_test@@3HA" >> already added to your JIT because (in general) you never know when JIT'd >> code will need it. >> >> >> >> My turn to ask a question: How is "?_Plansch_test@@3HA" created, and why >> not just add it up-front? :) >> >> >> >> -- Lang. >> >> >> >> On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >> Hey everyone, >> >> >> >> I felt this question is different from my other question - hope this is >> okay. >> >> >> >> So - I was playing around with the lookup function of the >> ExecutionSession and there are some things I don't understand. >> >> I have a .BC file with a function "?Sampler@@YAXXZ" referencing a value >> "?_Plansch_test@@3HA" that is not defined in that module itself. I first >> planed on not providing an address for "?_Plansch_test@@3HA" but wanted >> to know the address of "?Sampler@@YAXXZ". So I issued something like >> that: >> >> >> >> auto &ES = this->jit->getExecutionSession(); >> >> SymbolLookupSet lookupSet; >> >> >> >> lookupSet.add("?Sampler@@YAXXZ", >> llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol); >> >> ES.lookup({{&jit->getMainJITDylib(), >> llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, >> llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved); >> >> >> >> Even though the "tryToGenerate" function of my DefinitionGenerator >> returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I >> got an address for "?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, >> that the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I >> guess. >> >> >> >> Question 1.) >> >> Is there any way to reset the error state of "?Sampler@@YAXXZ" at this >> point? >> >> - After my first call I used the "define" function of the JITDylib to >> define ?_Plansch_test@@3HA and then I tried calling the lookup function >> again and again, however I only got the error: "Failed to materialize >> symbols" even though "?_Plansch_test@@3HA" was defined now... >> >> - Changing the order of the "define" and the "lookup" call works of >> course, but I'm interested in the case where I don't know the address yet. >> >> >> >> Out of curiosity I repeated the previous scenario - but added >> "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. >> When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error >> from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a pending >> query in the MaterializingInfos entries. When I then add a definition for >> "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed >> and give me the addresses. Also I'm able to execute the code now. This is >> great! However... >> >> >> >> Question 2.) >> >> Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" >> like in the first scenario? I expected it would return an address for it. >> >> >> >> Question 3.) >> >> Can I somehow combine both behaviours? Getting the address for all the >> symbols (like in scenario 1) while still being able to provide definitions >> later (like in scenario 2)? >> >> >> >> Thank you in advance and kind greetings, >> >> Björn >> >> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, >> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert >> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin >> Cansiz. >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, >> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert >> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin >> Cansiz. >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200929/afbc03c0/attachment-0001.html>
Gaier, Bjoern via llvm-dev
2020-Sep-30 07:05 UTC
[llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup?
Hey Lang,> Do you mean that the object file is produced by another process and is being loaded into your JIT process for execution, or that you want your JIT to produce code for several different processes? These are different problems with different solutions. I'll wait until I understand your use case to answer further.In the current state we don’t have a JIT only an handcrafted object loader. All our object files are pre-compiled but will be loaded by different processes into the same shared memory for the main process to execute them. We would like to still have the files pre-compiled but we want to remove the handcrafted object loader. However, since we already create those files with Clang, it is not such a big change to compile them to IR.> You're trying to do all this on Hard Mode. ;)Yeah it was a silly thought… sorry xO> The part of your use case that is the most opaque to me is the renaming. When you see a reference to "test" in some object, how do you decide that it should resolve to the definition of "test" in, for example, Planschi, as opposed to some other module? Do you just have a list of modules that you check in-order until you find a matching symbol name?You wouldn’t find a reference to “test”, you would find “Planschi_test”. We would first try resolving that with the standard C library functions. If this fails, then we will ask the object file “Planschi” if it has “test” and resolve it. If “Planschi” is not loaded for some reason, then we go on with the next symbol and mark “Planschi_test” as undefined. So… we fail linking. However, our code allows to just try the linking again – so we do and do until it succeeds. Only when the undefined references hit 0 we mark the object file as executable. However, if the module you depend on has still undefined references you can be screwed. Cause our code will not check the dependencies. Another reason why we want to move away. I think… but this is probably not thought through well: I need to do those steps on the IR level now – because the DefinitionGenerator needs to resolve all symbols, but I might have to wait for a different module to pop up first. So I would iterate over the globals and the functions of the Module: I would rename each symbol I encounter to have a unique name. This would be easy because when we load stuff, we give them a hierarchical name like “Parent_Child_Child_Dino” So "?Sampler@@YAXXZ" Could be renamed to "Parent_Child_Child_Dino_?Sampler@@YAXXZ". If I encounter an undefined reference that is not part of the standard library like "?_Plansch_test@@3HA" Then I would extract the relative path and convert it to an absolute one, getting: “Parent_Child__Plansch_? test@@3HA”. When the “Plansch” module was loaded, I would have give it an unique name as well so they would fit. I would then add them to there own DyLibs and would be happy… would I? 1.) When all symbols have such a unique name now, couldn’t I just add them all to the same DyLib and use the RessourceTracker to unload them later? This would spare me searching the right DyLibs. 2.) Can I ask a DyLib if it has something for the “Parent_Child__Plansch_? test@@3HA" loaded? My biggest worry is, I do all those fancy renaming and such, but I still have no guarantee that the missing symbols are already present, but when I look up my symbols of interest it is to late… cause if something is missing because the module was not loaded yet, then I get in an error state and I’m stuck there :c 3.) Isn’t the process of renaming those symbols like… really memory costly? Could I run into limitations of creating a too long function name? I hope that makes sense… cause I’m not good in explaining anything, especially not when it is in a different language :c Thank you again :D Kind greetings Björn From: Lang Hames <lhames at gmail.com> Sent: 29 September 2020 18:55 To: Gaier, Bjoern <Bjoern.Gaier at horiba.com> Cc: LLVM Developers Mailing List <llvm-dev at lists.llvm.org> Subject: Re: [llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup? Hi Bjoern, However, another thing of our system is, that each object file was loaded from a different process, Do you mean that the object file is produced by another process and is being loaded into your JIT process for execution, or that you want your JIT to produce code for several different processes? These are different problems with different solutions. I'll wait until I understand your use case to answer further. Writing this… I actually wondered about something else but this feels like a terrible approach… But now I’m curious xD So… if I load an IR-Module, could I use the static LLVM compiler to compile it to an object file and then use the source code of the LLD to load and resolve the symbols the same way/kind we did in the past? That sounds more like the “addObjectFile” function of the LLJIT… And then I guess I have to write a LinkLayer or something? That is where my knowledge ends… Disclaimer: I don’t like that approach but it would be interesting to know (also cause some people here would be happy with it .w.”) You're trying to do all this on Hard Mode. ;) ORC takes care of all this kind of stuff for you: Don't re-write IR. Leave references as symbolic -- they will be fixed up in the JIT linker. You don't need to write your own JIT linker. LLJIT has one built in. When you add things to the JIT: - If you have a program representation (module, object file, etc.) and you want to add it then just go ahead. - If your program representations contain external references then they must be resolvable or linking will fail (there's no getting around that, in a JIT or in a regular compile), BUT... - You can always add new definitions in response to a query by using a definition generator. If your definition generator can find/create a definition then great. If it can't then the reference really is unresolved and linking really should fail. The part of your use case that is the most opaque to me is the renaming. When you see a reference to "test" in some object, how do you decide that it should resolve to the definition of "test" in, for example, Planschi, as opposed to some other module? Do you just have a list of modules that you check in-order until you find a matching symbol name? -- Lang. On Tue, Sep 29, 2020 at 2:30 AM Gaier, Bjoern <Bjoern.Gaier at horiba.com<mailto:Bjoern.Gaier at horiba.com>> wrote: Hey Lang, Thank you for your help and your patience – also for your answers in the “ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup” mail. Both problems have the same origin so I keep writing about it here, to avoid duplication. My big problem is still handling cross references between modules with “our” name scheme. Since our old loader loads object files, we resolved those references with object files and since they were already compiled, we knew all addresses right away. With the LLJIT as I finally understand, I will only get the addresses when I have resolved every references, which makes the code way safer. However, another thing of our system is, that each object file was loaded from a different process, so sometimes not all symbols for ModuleA were present because ModuleB was not loaded/requested yet. That was okay, so we kept resolving the undefined references of ModuleA until ModuleB was loaded and everything was fine. If I get it right… This would change now to having a single LLJIT representing the entire system. Each process would get it’s own DyLib for there module. However, I would need to check on IR-Level now which symbols would be undefined – correct? Because if I wait until “DefinitionGenerator::tryToGenerate” is called and have to wait for a module that might never be loaded, then I’m stuck there forever. 1.) If I find a symbol that is undefined – and it has our name scheme, then I would jump to for example ModuleB, which is also not jitted yet and would do a “replaceAllUsesWith” on the Symbol of ModuleA to ModuleB – right? - Would that mean, when I add ModuleA to DyLibA – is ModuleB then part of DyLibA as well? 1.1.) Alternatively I could rename the symbol 2.) If the ModuleB is already jitted, then I can take the address to do the “replaceAllUsesWith” right? When I resolved all those references, then I can add the IR Module to my DyLib and compile it. However is it a good idea to use “replaceAllUsesWith” with addresses? Seems like the DefinitionGenerator would be jobless… Writing this… I actually wondered about something else but this feels like a terrible approach… But now I’m curious xD So… if I load an IR-Module, could I use the static LLVM compiler to compile it to an object file and then use the source code of the LLD to load and resolve the symbols the same way/kind we did in the past? That sounds more like the “addObjectFile” function of the LLJIT… And then I guess I have to write a LinkLayer or something? That is where my knowledge ends… Disclaimer: I don’t like that approach but it would be interesting to know (also cause some people here would be happy with it .w.”) Thank you so far! Kind greetings Björn From: Lang Hames <lhames at gmail.com<mailto:lhames at gmail.com>> Sent: 29 September 2020 01:47 To: Gaier, Bjoern <Bjoern.Gaier at horiba.com<mailto:Bjoern.Gaier at horiba.com>> Cc: LLVM Developers Mailing List <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> Subject: Re: [llvm-dev] ORC JIT - different behaviour of ExecutionSession.lookup? Hi Bjoern, Even though the "tryToGenerate" function of my DefinitionGenerator returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I got an address for "? That's because you're issuing the lookup with RequiredState == SymbolState::Resolved. This means that your query will return as soon as "?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s) addresses are assigned before external references are looked up. So after your lookup returns the linker attempts to find "?_Plansch_test@@3HA", fails, and so moves "?Sampler@@YAXXZ" to the error state. You almost always want to issue your lookups with RequiredState == SymbolState::Ready. This ensures that the query will not return until / unless the requested symbols (and all their dependencies) are successfully linked into the target process and ready to execute. Question 1.) Is there any way to reset the error state of "?Sampler@@YAXXZ" at this point? No. However, the removable code feature will allow you to remove failed materialization units once it lands in the mainline. - After my first call I used the "define" function of the JITDylib to define ?_Plansch_test@@3HA and then I tried calling the lookup function again and again, however I only got the error: "Failed to materialize symbols" even though "?_Plansch_test@@3HA" was defined now... - Changing the order of the "define" and the "lookup" call works of course, but I'm interested in the case where I don't know the address yet. The JIT doesn't re-try linking. Once a symbol has failed to link it remains in the error state. In theory, once removable code is added you could choose to remove and then re-add "?Sampler@@YAXXZ" after "?_Plansch_test@@3HA" is defined. The real solution though is just to make sure that "?_Plansch_test@@3HA" is defined (either directly or via a generator) before you look up "?Sampler@@YAXXZ". Out of curiosity I repeated the previous scenario - but added "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error from my DefinitionGenerator... Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a SymbolsNotFound error sent to your error reporter in the first scenario too, followed by a failure-to-materialize error for "?Sampler@@YAXXZ". ... and "?Sampler@@YAXXZ" is stuck as a pending query in the MaterializingInfos entries. Huh. That sounds like a bug: All references to the query should be removed from the state machine before it returns its result (in this case an error). I'll see if I can reproduce this locally and fix it up, but it doesn't affect the discussion here. When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed and give me the addresses. Also I'm able to execute the code now. This is great! However... When a lookup fails we try to restore the ExecutionSession state to what it was prior to the query. This is why the sequence "lookup -> symbols not found -> define -> lookup again" worked. Question 2.) Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" like in the first scenario? I expected it would return an address for it. A lookup must match against all symbols before anything is JIT'd. When it failed to match "?_Plansch_test@@3HA" we immediately bailed out with an error. There was no further attempt to compile "?Sampler@@YAXXZ". Question 3.) Can I somehow combine both behaviours? Getting the address for all the symbols (like in scenario 1) while still being able to provide definitions later (like in scenario 2)? Sort of. Definition generators allow you to provide a definition at the last minute (i.e. in response to a query). The best mental model though is: "All definitions that a generator can generate are part of the interface of the dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols from a dynamic library containing "foo", "bar" and "baz" then you should think of your JITDylib as containing definitions for "foo", "bar" and "baz", even if the generator hasn't actually added them to the JITDylib yet. The reason is that it will add them in response to any query for them, so it's indistinguishable (except for timing and debug logging) from the case where they're already present. If you need to be able to defer adding a "real" definition beyond the initial lookup then your only option (and this only applies to functions) is a lazy-reexport. This allows you to provide a definition for a function while deferring lookup until the first execution of the re-export at runtime. I wouldn't generally use this to break dependencies though: You want a definition of the real function body for "?_Plansch_test@@3HA" already added to your JIT because (in general) you never know when JIT'd code will need it. My turn to ask a question: How is "?_Plansch_test@@3HA" created, and why not just add it up-front? :) -- Lang. On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote: Hey everyone, I felt this question is different from my other question - hope this is okay. So - I was playing around with the lookup function of the ExecutionSession and there are some things I don't understand. I have a .BC file with a function "?Sampler@@YAXXZ" referencing a value "?_Plansch_test@@3HA" that is not defined in that module itself. I first planed on not providing an address for "?_Plansch_test@@3HA" but wanted to know the address of "?Sampler@@YAXXZ". So I issued something like that: auto &ES = this->jit->getExecutionSession(); SymbolLookupSet lookupSet; lookupSet.add("?Sampler@@YAXXZ", llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol); ES.lookup({{&jit->getMainJITDylib(), llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved); Even though the "tryToGenerate" function of my DefinitionGenerator returned a "llvm::orc::SymbolsNotFound" for the "?_Plansch_test@@3HA", I got an address for "?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, that the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I guess. Question 1.) Is there any way to reset the error state of "?Sampler@@YAXXZ" at this point? - After my first call I used the "define" function of the JITDylib to define ?_Plansch_test@@3HA and then I tried calling the lookup function again and again, however I only got the error: "Failed to materialize symbols" even though "?_Plansch_test@@3HA" was defined now... - Changing the order of the "define" and the "lookup" call works of course, but I'm interested in the case where I don't know the address yet. Out of curiosity I repeated the previous scenario - but added "?_Plansch_test@@3HA" to the "lookupSet" which changed things drasticly. When executing "lookup" I now get the "llvm::orc::SymbolsNotFound" error from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a pending query in the MaterializingInfos entries. When I then add a definition for "?_Plansch_test@@3HA" and call "lookup" the second time, it will succeed and give me the addresses. Also I'm able to execute the code now. This is great! However... Question 2.) Why did the first call to lookup not return the address of "?Sampler@@YAXXZ" like in the first scenario? I expected it would return an address for it. Question 3.) Can I somehow combine both behaviours? Getting the address for all the symbols (like in scenario 1) while still being able to provide definitions later (like in scenario 2)? Thank you in advance and kind greetings, Björn Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz. _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz. Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200930/27282598/attachment-0001.html>