llvm dev - Sep 2020 - ORC JIT - different behaviour of ExecutionSession.lookup?

Hey everyone,

I felt this question is different from my other question - hope this is okay.

So - I was playing around with the lookup function of the ExecutionSession and
there are some things I don't understand.
I have a .BC file with a function "?Sampler@@YAXXZ" referencing a
value "?_Plansch_test@@3HA" that is not defined in that module itself.
I first planed on not providing an address for "?_Plansch_test@@3HA"
but wanted to know the address of "?Sampler@@YAXXZ". So I issued
something like that:

                auto &ES = this->jit->getExecutionSession();
                SymbolLookupSet lookupSet;

                lookupSet.add("?Sampler@@YAXXZ",
llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
                ES.lookup({{&jit->getMainJITDylib(),
llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet,
llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);

Even though the "tryToGenerate" function of my DefinitionGenerator
returned a "llvm::orc::SymbolsNotFound" for the
"?_Plansch_test@@3HA", I got an address for
"?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, that
the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I
guess.

Question 1.)
Is there any way to reset the error state of "?Sampler@@YAXXZ" at this
point?
- After my first call I used the "define" function of the JITDylib to
define ?_Plansch_test@@3HA and then I tried calling the lookup function again
and again, however I only got the error: "Failed to materialize
symbols" even though "?_Plansch_test@@3HA" was defined now...
- Changing the order of the "define" and the "lookup" call
works of course, but I'm interested in the case where I don't know the
address yet.

Out of curiosity I repeated the previous scenario - but added
"?_Plansch_test@@3HA" to the "lookupSet" which changed
things drasticly. When executing "lookup" I now get the
"llvm::orc::SymbolsNotFound" error from my DefinitionGenerator and
"?Sampler@@YAXXZ" is stuck as a pending query in the
MaterializingInfos entries. When I then add a definition for
"?_Plansch_test@@3HA" and call "lookup" the second time, it
will succeed and give me the addresses. Also I'm able to execute the code
now. This is great! However...

Question 2.)
Why did the first call to lookup not return the address of
"?Sampler@@YAXXZ" like in the first scenario? I expected it would
return an address for it.

Question 3.)
Can I somehow combine both behaviours? Getting the address for all the symbols
(like in scenario 1) while still being able to provide definitions later (like
in scenario 2)?

Thank you in advance and kind greetings,
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200928/471bf9ca/attachment.html>

Hi Bjoern,

Even though the "tryToGenerate" function of my DefinitionGenerator
returned
> a "llvm::orc::SymbolsNotFound" for the
"?_Plansch_test@@3HA", I got an
> address for "?

That's because you're issuing the lookup with RequiredState
=SymbolState::Resolved. This means that your query will return as soon as
"?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s)
addresses
are assigned before external references are looked up. So after your lookup
returns the linker attempts to find "?_Plansch_test@@3HA", fails, and
so
moves "?Sampler@@YAXXZ" to the error state.

You almost always want to issue your lookups with RequiredState
=SymbolState::Ready. This ensures that the query will not return until /
unless the requested symbols (and all their dependencies) are successfully
linked into the target process and ready to execute.

Question 1.)
> Is there any way to reset the error state of "?Sampler@@YAXXZ" at
this
> point?

No. However, the removable code feature will allow you to remove failed
materialization units once it lands in the mainline.


- After my first call I used the "define" function of the JITDylib
to
> define ?_Plansch_test@@3HA and then I tried calling the lookup function
> again and again, however I only got the error: "Failed to materialize
> symbols" even though "?_Plansch_test@@3HA" was defined
now...
> - Changing the order of the "define" and the "lookup"
call works of
> course, but I'm interested in the case where I don't know the
address yet.

The JIT doesn't re-try linking. Once a symbol has failed to link it remains
in the error state. In theory, once removable code is added you could
choose to remove and then re-add "?Sampler@@YAXXZ" after
"?_Plansch_test@@3HA"
is defined. The real solution though is just to make sure
that "?_Plansch_test@@3HA" is defined (either directly or via a
generator)
before you look up "?Sampler@@YAXXZ".


Out of curiosity I repeated the previous scenario - but
added
> "?_Plansch_test@@3HA" to the "lookupSet" which changed
things drasticly.
> When executing "lookup" I now get the
"llvm::orc::SymbolsNotFound" error
> from my DefinitionGenerator...

Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a
SymbolsNotFound error sent to your error reporter in the first scenario
too, followed by a failure-to-materialize error for "?Sampler@@YAXXZ".


... and "?Sampler@@YAXXZ" is stuck as a pending query in
the
> MaterializingInfos entries.

Huh. That sounds like a bug: All references to the query should be removed
from the state machine before it returns its result (in this case an
error). I'll see if I can reproduce this locally and fix it up, but it
doesn't affect the discussion here.


When I then add a definition for "?_Plansch_test@@3HA" and call
"lookup"
> the second time, it will succeed and give me the addresses. Also I'm
able
> to execute the code now. This is great! However...

When a lookup fails we try to restore the ExecutionSession state to what it
was prior to the query. This is why the sequence "lookup -> symbols not
found -> define -> lookup again" worked.


Question 2.)
> Why did the first call to lookup not return the address of
"?Sampler@@YAXXZ"
> like in the first scenario? I expected it would return an address for it.

A lookup must match against all symbols before anything is JIT'd. When it
failed to match "?_Plansch_test@@3HA" we immediately bailed out with
an
error. There was no further attempt to compile "?Sampler@@YAXXZ".


Question 3.)
> Can I somehow combine both behaviours? Getting the address for all the
> symbols (like in scenario 1) while still being able to provide definitions
> later (like in scenario 2)?

*Sort of.*


Definition generators allow you to provide a definition at the last minute
(i.e. in response to a query). The best mental model though is: "All
definitions that a generator can generate are part of the interface of the
dylib". E.g. if you use a DynamicLibrarySearchGenerator to mirror symbols
from a dynamic library containing "foo", "bar" and
"baz" then you should
think of your JITDylib as containing definitions for "foo",
"bar" and
"baz", even if the generator hasn't actually added them to the
JITDylib
yet. The reason is that it will add them in response to any query for them,
so it's indistinguishable (except for timing and debug logging) from the
case where they're already present.


If you need to be able to defer adding a "real" definition beyond the
initial lookup then your only option (and this only applies to functions)
is a lazy-reexport. This allows you to provide a definition for a function
while deferring lookup until the first execution of the re-export at
runtime. I wouldn't generally use this to break dependencies though: You
want a definition of the real function body for "?_Plansch_test@@3HA"
already added to your JIT because (in general) you never know when JIT'd
code will need it.


My turn to ask a question: How is "?_Plansch_test@@3HA" created, and
why
not just add it up-front? :)


-- Lang.

On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hey everyone,
>
>
>
> I felt this question is different from my other question - hope this is
> okay.
>
>
>
> So - I was playing around with the lookup function of the ExecutionSession
> and there are some things I don't understand.
>
> I have a .BC file with a function "?Sampler@@YAXXZ" referencing a
value
> "?_Plansch_test@@3HA" that is not defined in that module itself.
I first
> planed on not providing an address for "?_Plansch_test@@3HA" but
wanted
> to know the address of "?Sampler@@YAXXZ". So I issued something
like that:
>
>
>
>                 auto &ES = this->jit->getExecutionSession();
>
>                 SymbolLookupSet lookupSet;
>
>
>
>                 lookupSet.add("?Sampler@@YAXXZ",
> llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
>
>                 ES.lookup({{&jit->getMainJITDylib(),
> llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet,
> llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);
>
>
>
> Even though the "tryToGenerate" function of my
DefinitionGenerator
> returned a "llvm::orc::SymbolsNotFound" for the
"?_Plansch_test@@3HA", I
> got an address for "?Sampler@@YAXXZ". Dumping the
"MainJITDylib" I saw,
> that the "?Sampler@@YAXXZ" was in an Error state. Which made
sense - I
> guess.
>
>
>
> Question 1.)
>
> Is there any way to reset the error state of "?Sampler@@YAXXZ" at
this
> point?
>
> - After my first call I used the "define" function of the
JITDylib to
> define ?_Plansch_test@@3HA and then I tried calling the lookup function
> again and again, however I only got the error: "Failed to materialize
> symbols" even though "?_Plansch_test@@3HA" was defined
now...
>
> - Changing the order of the "define" and the "lookup"
call works of
> course, but I'm interested in the case where I don't know the
address yet.
>
>
>
> Out of curiosity I repeated the previous scenario - but added
> "?_Plansch_test@@3HA" to the "lookupSet" which changed
things drasticly.
> When executing "lookup" I now get the
"llvm::orc::SymbolsNotFound" error
> from my DefinitionGenerator and "?Sampler@@YAXXZ" is stuck as a
pending
> query in the MaterializingInfos entries. When I then add a definition for
> "?_Plansch_test@@3HA" and call "lookup" the second
time, it will succeed
> and give me the addresses. Also I'm able to execute the code now. This
is
> great! However...
>
>
>
> Question 2.)
>
> Why did the first call to lookup not return the address of
"?Sampler@@YAXXZ"
> like in the first scenario? I expected it would return an address for it.
>
>
>
> Question 3.)
>
> Can I somehow combine both behaviours? Getting the address for all the
> symbols (like in scenario 1) while still being able to provide definitions
> later (like in scenario 2)?
>
>
>
> Thank you in advance and kind greetings,
>
> Björn
> Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816,
> USt.ID-Nr. DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert
> Plank, Markus Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin
> Cansiz.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200928/3e6ec59f/attachment-0001.html>

Hey Lang,

Thank you for your help and your patience – also for your answers in the “ORC
JIT - Can modules independently managed with one LLJIT instance? + problems with
ExecutionSession.lookup” mail. Both problems have the same origin so I keep
writing about it here, to avoid duplication.
My big problem is still handling cross references between modules with “our”
name scheme. Since our old loader loads object files, we resolved those
references with object files and since they were already compiled, we knew all
addresses right away. With the LLJIT as I finally understand, I will only get
the addresses when I have resolved every references, which makes the code way
safer.
However, another thing of our system is, that each object file was loaded from a
different process, so sometimes not all symbols for ModuleA were present because
ModuleB was not loaded/requested yet. That was okay, so we kept resolving the
undefined references of ModuleA until ModuleB was loaded and everything was
fine.

If I get it right… This would change now to having a single LLJIT representing
the entire system. Each process would get it’s own DyLib for there module.
However, I would need to check on IR-Level now which symbols would be undefined
– correct? Because if I wait until “DefinitionGenerator::tryToGenerate” is
called and have to wait for a module that might never be loaded, then I’m stuck
there forever.
1.) If I find a symbol that is undefined – and it has our name scheme, then I
would jump to for example ModuleB, which is also not jitted yet and would do a
“replaceAllUsesWith” on the Symbol of ModuleA to ModuleB – right?
                - Would that mean, when I add ModuleA to DyLibA – is ModuleB
then part of DyLibA as well?
1.1.) Alternatively I could rename the symbol
2.) If the ModuleB is already jitted, then I can take the address to  do the
“replaceAllUsesWith” right?

When I resolved all those references, then I can add the IR Module to my DyLib
and compile it. However is it a good idea to use “replaceAllUsesWith” with
addresses? Seems like the DefinitionGenerator would be jobless…

Writing this… I actually wondered about something else but this feels like a
terrible approach… But now I’m curious xD
So… if I load an IR-Module, could I use the static LLVM compiler to compile it
to an object file and then use the source code of the LLD to load and resolve
the symbols the same way/kind we did in the past?
That sounds more like the “addObjectFile” function of the LLJIT… And then I
guess I have to write a LinkLayer or something? That is where my knowledge ends…
Disclaimer: I don’t like that approach but it would be interesting to know (also
cause some people here would be happy with it .w.”)

Thank you so far!

Kind greetings
Björn


From: Lang Hames <lhames at gmail.com>
Sent: 29 September 2020 01:47
To: Gaier, Bjoern <Bjoern.Gaier at horiba.com>
Cc: LLVM Developers Mailing List <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] ORC JIT - different behaviour of
ExecutionSession.lookup?

Hi Bjoern,

Even though the "tryToGenerate" function of my DefinitionGenerator
returned a "llvm::orc::SymbolsNotFound" for the
"?_Plansch_test@@3HA", I got an address for "?

That's because you're issuing the lookup with RequiredState ==
SymbolState::Resolved. This means that your query will return as soon as
"?Sampler@@YAXXZ" is assigned an address. In the JIT linker(s)
addresses are assigned before external references are looked up. So after your
lookup returns the linker attempts to find "?_Plansch_test@@3HA",
fails, and so moves "?Sampler@@YAXXZ" to the error state.

You almost always want to issue your lookups with RequiredState ==
SymbolState::Ready. This ensures that the query will not return until / unless
the requested symbols (and all their dependencies) are successfully linked into
the target process and ready to execute.

Question 1.)
Is there any way to reset the error state of "?Sampler@@YAXXZ" at this
point?

No. However, the removable code feature will allow you to remove failed
materialization units once it lands in the mainline.

- After my first call I used the "define" function of the JITDylib to
define ?_Plansch_test@@3HA and then I tried calling the lookup function again
and again, however I only got the error: "Failed to materialize
symbols" even though "?_Plansch_test@@3HA" was defined now...
- Changing the order of the "define" and the "lookup" call
works of course, but I'm interested in the case where I don't know the
address yet.

The JIT doesn't re-try linking. Once a symbol has failed to link it remains
in the error state. In theory, once removable code is added you could choose to
remove and then re-add "?Sampler@@YAXXZ" after
"?_Plansch_test@@3HA" is defined. The real solution though is just to
make sure that "?_Plansch_test@@3HA" is defined (either directly or
via a generator) before you look up "?Sampler@@YAXXZ".

Out of curiosity I repeated the previous scenario - but added
"?_Plansch_test@@3HA" to the "lookupSet" which changed
things drasticly. When executing "lookup" I now get the
"llvm::orc::SymbolsNotFound" error from my DefinitionGenerator...

Yes. Because "?_Plansch_test@@3HA" is not defined. You should see a
SymbolsNotFound error sent to your error reporter in the first scenario too,
followed by a failure-to-materialize error for "?Sampler@@YAXXZ".

... and "?Sampler@@YAXXZ" is stuck as a pending query in the
MaterializingInfos entries.

Huh. That sounds like a bug: All references to the query should be removed from
the state machine before it returns its result (in this case an error). I'll
see if I can reproduce this locally and fix it up, but it doesn't affect the
discussion here.

When I then add a definition for "?_Plansch_test@@3HA" and call
"lookup" the second time, it will succeed and give me the addresses.
Also I'm able to execute the code now. This is great! However...

When a lookup fails we try to restore the ExecutionSession state to what it was
prior to the query. This is why the sequence "lookup -> symbols not
found -> define -> lookup again" worked.

Question 2.)
Why did the first call to lookup not return the address of
"?Sampler@@YAXXZ" like in the first scenario? I expected it would
return an address for it.

A lookup must match against all symbols before anything is JIT'd. When it
failed to match "?_Plansch_test@@3HA" we immediately bailed out with
an error. There was no further attempt to compile "?Sampler@@YAXXZ".

Question 3.)
Can I somehow combine both behaviours? Getting the address for all the symbols
(like in scenario 1) while still being able to provide definitions later (like
in scenario 2)?

Sort of.

Definition generators allow you to provide a definition at the last minute (i.e.
in response to a query). The best mental model though is: "All definitions
that a generator can generate are part of the interface of the dylib". E.g.
if you use a DynamicLibrarySearchGenerator to mirror symbols from a dynamic
library containing "foo", "bar" and "baz" then you
should think of your JITDylib as containing definitions for "foo",
"bar" and "baz", even if the generator hasn't actually
added them to the JITDylib yet. The reason is that it will add them in response
to any query for them, so it's indistinguishable (except for timing and
debug logging) from the case where they're already present.

If you need to be able to defer adding a "real" definition beyond the
initial lookup then your only option (and this only applies to functions) is a
lazy-reexport. This allows you to provide a definition for a function while
deferring lookup until the first execution of the re-export at runtime. I
wouldn't generally use this to break dependencies though: You want a
definition of the real function body for "?_Plansch_test@@3HA" already
added to your JIT because (in general) you never know when JIT'd code will
need it.

My turn to ask a question: How is "?_Plansch_test@@3HA" created, and
why not just add it up-front? :)

-- Lang.

On Mon, Sep 28, 2020 at 4:57 AM Gaier, Bjoern via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hey everyone,

I felt this question is different from my other question - hope this is okay.

So - I was playing around with the lookup function of the ExecutionSession and
there are some things I don't understand.
I have a .BC file with a function "?Sampler@@YAXXZ" referencing a
value "?_Plansch_test@@3HA" that is not defined in that module itself.
I first planed on not providing an address for "?_Plansch_test@@3HA"
but wanted to know the address of "?Sampler@@YAXXZ". So I issued
something like that:

                auto &ES = this->jit->getExecutionSession();
                SymbolLookupSet lookupSet;

                lookupSet.add("?Sampler@@YAXXZ",
llvm::orc::SymbolLookupFlags::WeaklyReferencedSymbol);
                ES.lookup({{&jit->getMainJITDylib(),
llvm::orc::JITDylibLookupFlags::MatchAllSymbols}}, lookupSet,
llvm::orc::LookupKind::Static, llvm::orc::SymbolState::Resolved);

Even though the "tryToGenerate" function of my DefinitionGenerator
returned a "llvm::orc::SymbolsNotFound" for the
"?_Plansch_test@@3HA", I got an address for
"?Sampler@@YAXXZ". Dumping the "MainJITDylib" I saw, that
the "?Sampler@@YAXXZ" was in an Error state. Which made sense - I
guess.

Question 1.)
Is there any way to reset the error state of "?Sampler@@YAXXZ" at this
point?
- After my first call I used the "define" function of the JITDylib to
define ?_Plansch_test@@3HA and then I tried calling the lookup function again
and again, however I only got the error: "Failed to materialize
symbols" even though "?_Plansch_test@@3HA" was defined now...
- Changing the order of the "define" and the "lookup" call
works of course, but I'm interested in the case where I don't know the
address yet.

Out of curiosity I repeated the previous scenario - but added
"?_Plansch_test@@3HA" to the "lookupSet" which changed
things drasticly. When executing "lookup" I now get the
"llvm::orc::SymbolsNotFound" error from my DefinitionGenerator and
"?Sampler@@YAXXZ" is stuck as a pending query in the
MaterializingInfos entries. When I then add a definition for
"?_Plansch_test@@3HA" and call "lookup" the second time, it
will succeed and give me the addresses. Also I'm able to execute the code
now. This is great! However...

Question 2.)
Why did the first call to lookup not return the address of
"?Sampler@@YAXXZ" like in the first scenario? I expected it would
return an address for it.

Question 3.)
Can I somehow combine both behaviours? Getting the address for all the symbols
(like in scenario 1) while still being able to provide definitions later (like
in scenario 2)?

Thank you in advance and kind greetings,
Björn
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Als GmbH eingetragen im Handelsregister Bad Homburg v.d.H. HRB 9816, USt.ID-Nr.
DE 114 165 789 Geschäftsführer: Dr. Hiroshi Nakamura, Dr. Robert Plank, Markus
Bode, Heiko Lampert, Takashi Nagano, Junichi Tajika, Ergin Cansiz.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200929/94c8bac7/attachment-0001.html>