thr3ads.net - llvm dev - [llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup [Sep 2020]

If this information is useful, please help other people find it:
Share via:

Gaier, Bjoern via llvm-dev

2020-Sep-23 08:02 UTC

[llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

Hi Lang,

Thank you for your answer!  This helped me again a lot!! Also that
ResourceTracker is a really neat feature! Looking forward to it! :3
I changed the title cause… there is another issue I have (sorry about that…)

I’m finally allowed to investigate the ORC JIT for integration into our system,
which meant I got a few days to actually play around with it. However, another
problem arise which breaks my concept. It is the never ending story of “cross
references”. I have again two Modules, which are added to two different LLJIT
instances, but they are referencing each other. In the past you suggested me to
use the LookUp function of the ExecutionSession to get the addresses – so I
wrote this:

bool ModuleLoader :: resolve()
{
       auto &ES = this->jit->getExecutionSession();
       SymbolLookupSet lookupSet;

       this->undefinedReferences.clear();

       for(const auto &element : this->symbolsOfInterrest)
       {
             
lookupSet.add(element.second.name,llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
       }

auto result = ES.lookup({{&jit->getMainJITDylib(),
llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet,
llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);
       if(result)
       {
             for(const auto &element : *result)
             {
                    const llvm::StringRef &name = (*element.first);
                    const size_t           hash = calculateHash(name);

                    printf(">>>%s @ 0x%p\n", name.data(),
element.second.getAddress());
                    this->symbolsOfInterrest[hash].adr =
element.second.getAddress();
             }
       }

       else
       {
             ES.reportError(result.takeError());
       }

       this->mtx.unlock();
       return (this->undefinedReferences.size() == 0ull);
}

I also attached a reporter to the ES which will handle
“llvm::orc::SymbolsNotFound” by copying SymbolVector to the
“undefinedReferences”. If I call this function and have every symbol resolved,
then I can use the addresses to actually execute them. That is great! However,
when I have an undefined Reference, things get strange… The first call will
trigger my “tryToGenerate” function but it will not be able to resolve a certain
symbol. The reporter will be triggered and the “undefinedReferences”
SymbolVector will have size 1.
When I call the function a second time however, the “tryToGenerate” function
will not be called anymore, so my vector will be empty, the undefined reference
is still not resolved, but I return with a true and crash my program. So even if
I would have an address for that one symbol in the second run, I would have no
chance to tell “anyone” cause the “tryToGenerate” function was never called…
Said function looks like that though:

llvm::Error ReferenceManager::UndefinedReferenceResolver ::
tryToGenerate(llvm::orc::LookupKind K, llvm::orc::JITDylib &JD,
llvm::orc::JITDylibLookupFlags JDLookupFlags, const llvm::orc::SymbolLookupSet
&LookupSet)
{
       llvm::orc::SymbolNameVector notFound;
       llvm::orc::SymbolMap        NewSymbols;

       for(const auto &name : LookupSet)
       {
             printf("Generate!\n");
             const uintptr_t adr =
UndefinedReferenceResolver::lookup((*name.first).data());
             if(adr)
             {
                    NewSymbols[name.first] = llvm::JITEvaluatedSymbol(adr,
llvm::JITSymbolFlags::Absolute);
             }

             else
             {
                    notFound.push_back(name.first);
             }
       }

       JD.define(absoluteSymbols(std::move(NewSymbols)));
       return (notFound.size() == 0) ? llvm::Error::success() :
llvm::make_error<llvm::orc::SymbolsNotFound>(std::move(notFound));
}

When I use “llvm::orc::JITDylibLookupFlags::MatchExportedSymbolsOnly” then I
will get 1 undefined Reference in the first run, but a total of 9 in the second
run, because every symbol I wanted to lookup was now an undefined reference.

Thank you for the help in advance!

Kind greetings
Björn

From: Lang Hames <lhames at gmail.com>
Sent: 23 September 2020 07:54
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com>
Cc: LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
Subject: Re: ORC JIT - Can modules independently managed with one LLJIT
instance?

Hi Bjoern,

Sounds like you want the removable code feature that's under development in
the orcv1-removal branch of
https://github.com/lhames/llvm-project.git<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2flhames%2fllvm%2dproject.git&umid=7d73edbc-e566-4ffc-bd01-827f3d6c8c2e&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-918a46b7d72afae12d8ff60470e902dac471e605>.
I will be aiming to merge this work back into the mainline some time in the next
couple of weeks.

I have not added a "removeJITDylib" method to ExecutionSession with
this feature yet, but will try to design and publish that tomorrow. Then you
should be able to do exactly what you want.

In the new system you can also perform fine grained removal: It is possible to
track and remove individual modules from within a JITDylib. For an example of
this see:
https://github.com/lhames/llvm-project/blob/7ec9f8930f68760953a483157e010d0ff88285cd/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/toy.cpp#L1148<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2flhames%2fllvm%2dproject%2fblob%2f7ec9f8930f68760953a483157e010d0ff88285cd%2fllvm%2fexamples%2fKaleidoscope%2fBuildingAJIT%2fChapter1%2ftoy.cpp%23L1148&umid=7d73edbc-e566-4ffc-bd01-827f3d6c8c2e&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-c0d11d7f66acc6ff5b094b7166067e93a53cc1ff>,
from lines 1148 to line 1161.

Also a related but different question:
When using llvm::parseIRFile I need to pass it a LLVMContext - should this be a
unique one for every module I pass or can I use a global one?

It depends on whether you want to be able to compile concurrently in the JIT
process. If you do want to compile concurrently then each module should get its
own context. If you are happy to stick to single threaded compilation you can
load all modules on the same context, potentially saving some memory.

-- Lang.

On Thu, Sep 17, 2020 at 11:59 PM Gaier, Bjoern <Bjoern.Gaier at
horiba.com<mailto:Bjoern.Gaier at horiba.com>> wrote:
Hello everyone and Lang,

I have another design or "how-to" question about the ORC JIT. Sorry
for having so many about them, to me this is a really complicated yet
fascinating subject...

How would I design an ORC JIT with the following requirements?
At any time it should be possible to load a LLVM Module, every Module is
independent and is not allowed to be linked with the other modules, every module
can be removed at any time.

My first idea was to have an ORC JIT for every module I load, but then I
wondered if I could use a single ORC JIT for it.

So, I would create an ORC JIT:
- using llvm::orc::LLJITBuilder
- configurating it
- adding a custom memory manager that requests the entire memory size

Then I would call "getMainJITDylib" and fill it with symbols that are
valid for every module:
- adding printf, strlen, usw.

Now when I get a request to load a module:
- load module
- get symbols I want to look up
- create new DyLib and add module to that

Now I would do a lookup on that module, plus the main module to get symbol
addresses and stuff

If a new module is added, it goes to a new DyLib as well and so on.

-----

However, now I want to delete a module again. Killing the memory is fairly easy
with my custom memory manager, but I still have my DyLib of the now dead code...
Can I get rid of it without dumping the others or the entire ORC JIT?
Or is there a different approach to do this? My goal with this is to speed up
the setup phase for a module and also to reduce dynamic memory usage.

Also a related but different question:
When using llvm::parseIRFile I need to pass it a LLVMContext - should this be a
unique one for every module I pass or can I use a global one?

Kind greetings and thank you all again,
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200923/3c234b47/attachment.html>

Lang Hames via llvm-dev

2020-Sep-23 16:15 UTC

head link

[llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

Hi Bjoern,

If you had the removable code feature could you merge your LLJIT instances
and just have one instance with multiple JITDylibs? That will make your
life *much* easier. Interdependencies between modules in different
ExecutionSessions are dangerous: Dependencies are not tracked, and it will
be easy for everything to look as if it's working, but fail with thread
scheduling bugs. E.g. Thread 1 assigns an address to Symbol 1 but yields
before setting the memory permissions. Thread 2 is scheduled and sees that
Symbol 1 has been resolved, so proceeds to link Symbol 2 against it, set
Symbol 2's memory permissions, then jump to it. The program will crash when
execution of JIT'd code reaches symbol 1.

To be clear: Cross JIT references are ok as long as the dependencies form a
DAG. If there are any cycles then you're in trouble.

I'd strongly recommend switching to one LLJIT instance and using removable
JITDylibs to solve this.

If you want to continue with multiple LLJIT instances my
comments/recommendations would be:
  (1) Don't use error handling to track the unresolved symbols -- there are
places in ORC where we only report the first missing symbol rather than all
of them.
  (2) You shouldn't issue a call to ExecutionSession::lookup from inside
tryToGenerate: The tryToGenerate method is called under the session lock,
but ExecutionSession::lookup should only be called outside the lock.
Instead of issuing the lookup in place you should write a custom
MaterializationUnit and issue the lookup from its materialize method.

-- Lang.


On Wed, Sep 23, 2020 at 1:02 AM Gaier, Bjoern <Bjoern.Gaier at horiba.com>
wrote:
> Hi Lang,
>
>
>
> Thank you for your answer!  This helped me again a lot!! Also that
> ResourceTracker is a really neat feature! Looking forward to it! :3
>
> I changed the title cause… there is another issue I have (sorry about
> that…)
>
>
>
> I’m finally allowed to investigate the ORC JIT for integration into our
> system, which meant I got a few days to actually play around with it.
> However, another problem arise which breaks my concept. It is the never
> ending story of “cross references”. I have again two Modules, which are
> added to two different LLJIT instances, but they are referencing each
> other. In the past you suggested me to use the LookUp function of the
> ExecutionSession to get the addresses – so I wrote this:
>
>
>
> bool ModuleLoader :: resolve()
>
> {
>
>        auto &ES = this->jit->getExecutionSession();
>
>        SymbolLookupSet lookupSet;
>
>
>
>        this->undefinedReferences.clear();
>
>
>
>        for(const auto &element : this->symbolsOfInterrest)
>
>        {
>
>               lookupSet.add(element.second.name,llvm::orc::
> SymbolLookupFlags::WeaklyReferencedSymbol);
>
>        }
>
>
>
> auto result = ES.lookup({{&jit->getMainJITDylib(), llvm::orc::
> JITDylibLookupFlags::MatchAllSymbols}}, lookupSet, llvm::orc::LookupKind::
> Static, llvm::orc::SymbolState::Resolved);
>
>        if(result)
>
>        {
>
>              for(const auto &element : *result)
>
>              {
>
>                     const llvm::StringRef &name = (*element.first);
>
>                     const size_t           hash = calculateHash(name);
>
>
>
>                     printf(">>>%s @ 0x%p\n",
name.data(),
> element.second.getAddress());
>
>                     this->symbolsOfInterrest[hash].adr >
element.second.getAddress();
>
>              }
>
>        }
>
>
>
>        else
>
>        {
>
>              ES.reportError(result.takeError());
>
>        }
>
>
>
>        this->mtx.unlock();
>
>        return (this->undefinedReferences.size() == 0ull);
>
> }
>
>
>
> I also attached a reporter to the ES which will handle “llvm::orc::
> SymbolsNotFound” by copying SymbolVector to the “undefinedReferences”. If
> I call this function and have every symbol resolved, then I can use the
> addresses to actually execute them. That is great! However, when I have an
> undefined Reference, things get strange… The first call will trigger my
> “tryToGenerate” function but it will not be able to resolve a certain
> symbol. The reporter will be triggered and the “undefinedReferences”
> SymbolVector will have size 1.
> When I call the function a second time however, the “tryToGenerate”
> function will not be called anymore, so my vector will be empty, the
> undefined reference is still not resolved, but I return with a true and
> crash my program. So even if I would have an address for that one symbol in
> the second run, I would have no chance to tell “anyone” cause the
> “tryToGenerate” function was never called… Said function looks like that
> though:
>
>
>
> llvm::Error ReferenceManager::UndefinedReferenceResolver ::
> tryToGenerate(llvm::orc::LookupKind K, llvm::orc::JITDylib &JD,
> llvm::orc::JITDylibLookupFlags JDLookupFlags, const llvm::orc::
> SymbolLookupSet &LookupSet)
>
> {
>
>        llvm::orc::SymbolNameVector notFound;
>
>        llvm::orc::SymbolMap        NewSymbols;
>
>
>
>        for(const auto &name : LookupSet)
>
>        {
>
>              printf("Generate!\n");
>
>              const uintptr_t adr = UndefinedReferenceResolver::lookup((*
> name.first).data());
>
>              if(adr)
>
>              {
>
>                     NewSymbols[name.first] = llvm::JITEvaluatedSymbol(adr,
> llvm::JITSymbolFlags::Absolute);
>
>              }
>
>
>
>              else
>
>              {
>
>                     notFound.push_back(name.first);
>
>              }
>
>        }
>
>
>
>        JD.define(absoluteSymbols(std::move(NewSymbols)));
>
>        return (notFound.size() == 0) ? llvm::Error::success() :
> llvm::make_error<llvm::orc::SymbolsNotFound>(std::move(notFound));
>
> }
>
>
>
> When I use “llvm::orc::JITDylibLookupFlags::MatchExportedSymbolsOnly” then
> I will get 1 undefined Reference in the first run, but a total of 9 in the
> second run, because every symbol I wanted to lookup was now an undefined
> reference.
>
>
>
> Thank you for the help in advance!
>
>
>
> Kind greetings
>
> Björn
>
>
>
> *From:* Lang Hames <lhames at gmail.com>
> *Sent:* 23 September 2020 07:54
> *To:* Gaier, Bjoern <Bjoern.Gaier at horiba.com>
> *Cc:* LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
> *Subject:* Re: ORC JIT - Can modules independently managed with one LLJIT
> instance?
>
>
>
> Hi Bjoern,
>
>
>
> Sounds like you want the removable code feature that's under
development
> in the orcv1-removal branch of https://github.com/lhames/llvm-project.git
>
<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2flhames%2fllvm%2dproject.git&umid=7d73edbc-e566-4ffc-bd01-827f3d6c8c2e&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-918a46b7d72afae12d8ff60470e902dac471e605>.
> I will be aiming to merge this work back into the mainline some time in the
> next couple of weeks.
>
>
>
> I have not added a "removeJITDylib" method to ExecutionSession
with this
> feature yet, but will try to design and publish that tomorrow. Then you
> should be able to do exactly what you want.
>
>
>
> In the new system you can also perform fine grained removal: It is
> possible to track and remove individual modules from within a JITDylib. For
> an example of this see:
>
https://github.com/lhames/llvm-project/blob/7ec9f8930f68760953a483157e010d0ff88285cd/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/toy.cpp#L1148
>
<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2flhames%2fllvm%2dproject%2fblob%2f7ec9f8930f68760953a483157e010d0ff88285cd%2fllvm%2fexamples%2fKaleidoscope%2fBuildingAJIT%2fChapter1%2ftoy.cpp%23L1148&umid=7d73edbc-e566-4ffc-bd01-827f3d6c8c2e&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-c0d11d7f66acc6ff5b094b7166067e93a53cc1ff>,
> from lines 1148 to line 1161.
>
>
>
> Also a related but different question:
> When using llvm::parseIRFile I need to pass it a LLVMContext - should this
> be a unique one for every module I pass or can I use a global one?
>
>
>
> It depends on whether you want to be able to compile concurrently in the
> JIT process. If you do want to compile concurrently then each module should
> get its own context. If you are happy to stick to single threaded
> compilation you can load all modules on the same context, potentially
> saving some memory.
>
>
>
> -- Lang.
>
>
>
> On Thu, Sep 17, 2020 at 11:59 PM Gaier, Bjoern <Bjoern.Gaier at
horiba.com>
> wrote:
>
> Hello everyone and Lang,
>
>
>
> I have another design or "how-to" question about the ORC JIT.
Sorry for
> having so many about them, to me this is a really complicated yet
> fascinating subject...
>
>
>
> How would I design an ORC JIT with the following requirements?
>
> At any time it should be possible to load a LLVM Module, every Module is
> independent and is not allowed to be linked with the other modules, every
> module can be removed at any time.
>
>
>
> My first idea was to have an ORC JIT for every module I load, but then I
> wondered if I could use a single ORC JIT for it.
>
>
>
> So, I would create an ORC JIT:
>
> - using llvm::orc::LLJITBuilder
>
> - configurating it
>
> - adding a custom memory manager that requests the entire memory size
>
>
>
> Then I would call "getMainJITDylib" and fill it with symbols that
are
> valid for every module:
>
> - adding printf, strlen, usw.
>
>
>
> Now when I get a request to load a module:
>
> - load module
>
> - get symbols I want to look up
>
> - create new DyLib and add module to that
>
>
>
> Now I would do a lookup on that module, plus the main module to get symbol
> addresses and stuff
>
>
>
> If a new module is added, it goes to a new DyLib as well and so on.
>
>
>
> -----
>
>
>
> However, now I want to delete a module again. Killing the memory is fairly
> easy with my custom memory manager, but I still have my DyLib of the now
> dead code... Can I get rid of it without dumping the others or the entire
> ORC JIT?
>
> Or is there a different approach to do this? My goal with this is to speed
> up the setup phase for a module and also to reduce dynamic memory usage.
>
>
>
> Also a related but different question:
>
> When using llvm::parseIRFile I need to pass it a LLVMContext - should this
> be a unique one for every module I pass or can I use a global one?
>
>
>
> Kind greetings and thank you all again,
>
> Björn
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
>
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200923/bb979c5b/attachment.html>

Gaier, Bjoern via llvm-dev

2020-Sep-24 06:03 UTC

head link

[llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

Hey Lang,

I would be really happy to only have one LLJIT instance and using multiple
JITDylibs. However… it seems like that I don’t know enough to use them. So I
wonder…

  1.  When I add Module A to JITDylib A and Module B to JITDylib B – where will
those look for undefined symbols? Will Module A for example: will it only search
itself and the MainDylib? Or would it also search in JITDylib B?
  2.  If my current approach with using tryToGenerate and ES.lookUp is not
correct, how would I do this then? Our cross references don’t have the exact
symbol names where they want to be resolved to. Our modules are loaded into a
tree structure so you can use the symbol names to navigate that tree. This is
why I don’t want the LLJIT to do an automatic resolving of those symbols,
because it can’t know about our structure. However, how could I achieve this
then? Especially if it is not so straight forward to find the symbols that are
not resolved yet.

For example: Module A might reference Planschi_test Module B was loaded with
name “Planschi” and has a variable called test – so our old loader would then
resolve that “Planschi_test” reference with that address.
However those “pathes” can get really complex like navigating to a parent and
such.

  1.  Is there a way then, how I can lookup the addresses of the Module A, while
still having undefined references, but I’m allowed to provide those at a later
point?

About the multiple LLJIT approach:
When I used tryToGenerate for Module A I actually only looked in a hash map of
Module B which I filled with their ExecutionSession::lookup function – so
tryToGenerate didn’t called that function. However, like I said – a second call
to the Module A lookup didn’t triggered another call to tryToGenerate, giving me
no chance to ask Module Bs hash map again :c

Thank you again ^o^

Kind greetings
Björn

From: Lang Hames <lhames at gmail.com>
Sent: 23 September 2020 18:15
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com>
Cc: LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
Subject: Re: ORC JIT - Can modules independently managed with one LLJIT
instance? + problems with ExecutionSession.lookup

Hi Bjoern,

If you had the removable code feature could you merge your LLJIT instances and
just have one instance with multiple JITDylibs? That will make your life *much*
easier. Interdependencies between modules in different ExecutionSessions are
dangerous: Dependencies are not tracked, and it will be easy for everything to
look as if it's working, but fail with thread scheduling bugs. E.g. Thread 1
assigns an address to Symbol 1 but yields before setting the memory permissions.
Thread 2 is scheduled and sees that Symbol 1 has been resolved, so proceeds to
link Symbol 2 against it, set Symbol 2's memory permissions, then jump to
it. The program will crash when execution of JIT'd code reaches symbol 1.

To be clear: Cross JIT references are ok as long as the dependencies form a DAG.
If there are any cycles then you're in trouble.

I'd strongly recommend switching to one LLJIT instance and using removable
JITDylibs to solve this.

If you want to continue with multiple LLJIT instances my
comments/recommendations would be:
  (1) Don't use error handling to track the unresolved symbols -- there are
places in ORC where we only report the first missing symbol rather than all of
them.
  (2) You shouldn't issue a call to ExecutionSession::lookup from inside
tryToGenerate: The tryToGenerate method is called under the session lock, but
ExecutionSession::lookup should only be called outside the lock. Instead of
issuing the lookup in place you should write a custom MaterializationUnit and
issue the lookup from its materialize method.

-- Lang.


On Wed, Sep 23, 2020 at 1:02 AM Gaier, Bjoern <Bjoern.Gaier at
horiba.com<mailto:Bjoern.Gaier at horiba.com>> wrote:
Hi Lang,

Thank you for your answer!  This helped me again a lot!! Also that
ResourceTracker is a really neat feature! Looking forward to it! :3
I changed the title cause… there is another issue I have (sorry about that…)

I’m finally allowed to investigate the ORC JIT for integration into our system,
which meant I got a few days to actually play around with it. However, another
problem arise which breaks my concept. It is the never ending story of “cross
references”. I have again two Modules, which are added to two different LLJIT
instances, but they are referencing each other. In the past you suggested me to
use the LookUp function of the ExecutionSession to get the addresses – so I
wrote this:

bool ModuleLoader :: resolve()
{
       auto &ES = this->jit->getExecutionSession();
       SymbolLookupSet lookupSet;

       this->undefinedReferences.clear();

       for(const auto &element : this->symbolsOfInterrest)
       {
             
lookupSet.add(element.second.name<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f%2felement.second.name&umid=0e7ae036-6054-4cfd-8013-5cc37f16d67a&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-bedda45c2776595c390d63af4d6c24c16f889af2>,llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
       }

auto result = ES.lookup({{&jit->getMainJITDylib(),
llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet,
llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);
       if(result)
       {
             for(const auto &element : *result)
             {
                    const llvm::StringRef &name = (*element.first);
                    const size_t           hash = calculateHash(name);

                    printf(">>>%s @ 0x%p\n", name.data(),
element.second.getAddress());
                    this->symbolsOfInterrest[hash].adr =
element.second.getAddress();
             }
       }

       else
       {
             ES.reportError(result.takeError());
       }

       this->mtx.unlock();
       return (this->undefinedReferences.size() == 0ull);
}

I also attached a reporter to the ES which will handle
“llvm::orc::SymbolsNotFound” by copying SymbolVector to the
“undefinedReferences”. If I call this function and have every symbol resolved,
then I can use the addresses to actually execute them. That is great! However,
when I have an undefined Reference, things get strange… The first call will
trigger my “tryToGenerate” function but it will not be able to resolve a certain
symbol. The reporter will be triggered and the “undefinedReferences”
SymbolVector will have size 1.
When I call the function a second time however, the “tryToGenerate” function
will not be called anymore, so my vector will be empty, the undefined reference
is still not resolved, but I return with a true and crash my program. So even if
I would have an address for that one symbol in the second run, I would have no
chance to tell “anyone” cause the “tryToGenerate” function was never called…
Said function looks like that though:

llvm::Error ReferenceManager::UndefinedReferenceResolver ::
tryToGenerate(llvm::orc::LookupKind K, llvm::orc::JITDylib &JD,
llvm::orc::JITDylibLookupFlags JDLookupFlags, const llvm::orc::SymbolLookupSet
&LookupSet)
{
       llvm::orc::SymbolNameVector notFound;
       llvm::orc::SymbolMap        NewSymbols;

       for(const auto &name : LookupSet)
       {
             printf("Generate!\n");
             const uintptr_t adr =
UndefinedReferenceResolver::lookup((*name.first).data());
             if(adr)
             {
                    NewSymbols[name.first] = llvm::JITEvaluatedSymbol(adr,
llvm::JITSymbolFlags::Absolute);
             }

             else
             {
                    notFound.push_back(name.first);
             }
       }

       JD.define(absoluteSymbols(std::move(NewSymbols)));
       return (notFound.size() == 0) ? llvm::Error::success() :
llvm::make_error<llvm::orc::SymbolsNotFound>(std::move(notFound));
}

When I use “llvm::orc::JITDylibLookupFlags::MatchExportedSymbolsOnly” then I
will get 1 undefined Reference in the first run, but a total of 9 in the second
run, because every symbol I wanted to lookup was now an undefined reference.

Thank you for the help in advance!

Kind greetings
Björn

From: Lang Hames <lhames at gmail.com<mailto:lhames at gmail.com>>
Sent: 23 September 2020 07:54
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com<mailto:Bjoern.Gaier at
horiba.com>>
Cc: LLVM Developers Mailing List <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
Subject: Re: ORC JIT - Can modules independently managed with one LLJIT
instance?

Hi Bjoern,

Sounds like you want the removable code feature that's under development in
the orcv1-removal branch of
https://github.com/lhames/llvm-project.git<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2flhames%2fllvm%2dproject.git&umid=7d73edbc-e566-4ffc-bd01-827f3d6c8c2e&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-918a46b7d72afae12d8ff60470e902dac471e605>.
I will be aiming to merge this work back into the mainline some time in the next
couple of weeks.

I have not added a "removeJITDylib" method to ExecutionSession with
this feature yet, but will try to design and publish that tomorrow. Then you
should be able to do exactly what you want.

In the new system you can also perform fine grained removal: It is possible to
track and remove individual modules from within a JITDylib. For an example of
this see:
https://github.com/lhames/llvm-project/blob/7ec9f8930f68760953a483157e010d0ff88285cd/llvm/examples/Kaleidoscope/BuildingAJIT/Chapter1/toy.cpp#L1148<https://hes32-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fgithub.com%2flhames%2fllvm%2dproject%2fblob%2f7ec9f8930f68760953a483157e010d0ff88285cd%2fllvm%2fexamples%2fKaleidoscope%2fBuildingAJIT%2fChapter1%2ftoy.cpp%23L1148&umid=7d73edbc-e566-4ffc-bd01-827f3d6c8c2e&auth=b6f66d00f8195cc5198eee21f0dbabe6af0a3180-c0d11d7f66acc6ff5b094b7166067e93a53cc1ff>,
from lines 1148 to line 1161.

Also a related but different question:
When using llvm::parseIRFile I need to pass it a LLVMContext - should this be a
unique one for every module I pass or can I use a global one?

It depends on whether you want to be able to compile concurrently in the JIT
process. If you do want to compile concurrently then each module should get its
own context. If you are happy to stick to single threaded compilation you can
load all modules on the same context, potentially saving some memory.

-- Lang.

On Thu, Sep 17, 2020 at 11:59 PM Gaier, Bjoern <Bjoern.Gaier at
horiba.com<mailto:Bjoern.Gaier at horiba.com>> wrote:
Hello everyone and Lang,

I have another design or "how-to" question about the ORC JIT. Sorry
for having so many about them, to me this is a really complicated yet
fascinating subject...

How would I design an ORC JIT with the following requirements?
At any time it should be possible to load a LLVM Module, every Module is
independent and is not allowed to be linked with the other modules, every module
can be removed at any time.

My first idea was to have an ORC JIT for every module I load, but then I
wondered if I could use a single ORC JIT for it.

So, I would create an ORC JIT:
- using llvm::orc::LLJITBuilder
- configurating it
- adding a custom memory manager that requests the entire memory size

Then I would call "getMainJITDylib" and fill it with symbols that are
valid for every module:
- adding printf, strlen, usw.

Now when I get a request to load a module:
- load module
- get symbols I want to look up
- create new DyLib and add module to that

Now I would do a lookup on that module, plus the main module to get symbol
addresses and stuff

If a new module is added, it goes to a new DyLib as well and so on.

-----

However, now I want to delete a module again. Killing the memory is fairly easy
with my custom memory manager, but I still have my DyLib of the now dead code...
Can I get rid of it without dumping the others or the entire ORC JIT?
Or is there a different approach to do this? My goal with this is to speed up
the setup phase for a module and also to reduce dynamic memory usage.

Also a related but different question:
When using llvm::parseIRFile I need to pass it a LLVMContext - should this be a
unique one for every module I pass or can I use a global one?

Kind greetings and thank you all again,
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200924/428177b8/attachment-0001.html>

Possibly Parallel Threads

Search for more reasonably related threads

llvm dev - Sep 2020 - ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

[llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

[llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

[llvm-dev] ORC JIT - Can modules independently managed with one LLJIT instance? + problems with ExecutionSession.lookup

Possibly Parallel Threads