llvm dev - Aug 2019 - New ORC v2 LLJIT - global variables

We are trying to switch to the new orc v2 lljit engine at my work with
hopes of parallel jitting. We are switching from the ExecutionEngine w/
OrcMCJitReplacement. We are having a hard time with global variables. We
have found a way to create/emit new globals during jitting by using the old
ExecutionEngine::getOrEmitGlobalVariable. Is there an easier way to do this
with the new jit engine? We were hoping to create a new ThreadSafeModule,
add a GlobalVariable, and then add the module to the JIT engine through
addLazyIRModule(), but this doesn't seem to work.

2nd question; Is the lookup function supposed to work with global variables
by name? Looking through the CompileOnDemandLayer it seems to change the
names of globals to "__orc_lcl.<name><id>". Does this
affect the lookup
methods? I noticed that all of the globals we are looking for are tagged
with Internal linkage and end up getting emitted through extractSubModule().

Any help is appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190808/bf4f8ea4/attachment.html>

+Lang Hames <lhames at gmail.com>, Admiral of the Orcish Fleet (Lang's
the
primary developer/creator of the ORC JIT & might have some answers to your
question)

On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> We are trying to switch to the new orc v2 lljit engine at my work with
> hopes of parallel jitting. We are switching from the ExecutionEngine w/
> OrcMCJitReplacement. We are having a hard time with global variables. We
> have found a way to create/emit new globals during jitting by using the old
> ExecutionEngine::getOrEmitGlobalVariable. Is there an easier way to do this
> with the new jit engine? We were hoping to create a new ThreadSafeModule,
> add a GlobalVariable, and then add the module to the JIT engine through
> addLazyIRModule(), but this doesn't seem to work.
>
> 2nd question; Is the lookup function supposed to work with global
> variables by name? Looking through the CompileOnDemandLayer it seems to
> change the names of globals to
"__orc_lcl.<name><id>". Does this affect the
> lookup methods? I noticed that all of the globals we are looking for are
> tagged with Internal linkage and end up getting emitted through
> extractSubModule().
>
> Any help is appreciated.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190812/443cfd1e/attachment.html>

Hi Chris,
You approach to emit globals seems correct to me, there is also a
possibility that unused globals being removed. Could you please describe
your problem?
For question 2:
You can lookup the globals with their source name, besides ORC do internal
interning of Symbols.

On Mon, 12 Aug 2019 at 22:57, David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> +Lang Hames <lhames at gmail.com>, Admiral of the Orcish Fleet
(Lang's the
> primary developer/creator of the ORC JIT & might have some answers to
your
> question)
>
> On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> We are trying to switch to the new orc v2 lljit engine at my work with
>> hopes of parallel jitting. We are switching from the ExecutionEngine w/
>> OrcMCJitReplacement. We are having a hard time with global variables.
We
>> have found a way to create/emit new globals during jitting by using the
old
>> ExecutionEngine::getOrEmitGlobalVariable. Is there an easier way to do
this
>> with the new jit engine? We were hoping to create a new
ThreadSafeModule,
>> add a GlobalVariable, and then add the module to the JIT engine through
>> addLazyIRModule(), but this doesn't seem to work.
>>
>> 2nd question; Is the lookup function supposed to work with global
>> variables by name? Looking through the CompileOnDemandLayer it seems to
>> change the names of globals to
"__orc_lcl.<name><id>". Does this affect the
>> lookup methods? I noticed that all of the globals we are looking for
are
>> tagged with Internal linkage and end up getting emitted through
>> extractSubModule().
>>
>> Any help is appreciated.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190813/59057d94/attachment.html>

Hi Chris,

> Thanks so much for the quick help. I was able to find them after changing
> the linkage. Does this new orc JIT follow different standards than the
> MCJitReplacement? We used to set a lot of our functions/globals to internal
> linkage before doing a lookup.

They are different: ORCv2 follows the static linker's lookup rules, whereas
MCJITReplacement mimicked MCJIT's rules which were never clearly spelled
out, but in practice allowed you to search for internal symbols.

The advantage of the new scheme is that if you are compiling/running
something on the command line. E.g.:

$ llc -filetype=obj -o extra.o extra.ll
$ llc -filetype=obj -o prog.o prog.ll
$ clang -o prog prog.o extra.o
$ ./prog

Then you can add the same modules to the JIT and expect the resulting JIT'd
code to behave the same as the statically compiled code. E.g. (omitting
some error handling for clarity):

J->addIRModule(loadModule("extra.ll"));
J->addIRModule(loadModule("main.ll"));
auto Main = (int(*)())J->lookup("main").getAddress();
Main();

If you are generating code specifically for the JIT it is often easiest to
just use external linkage and default visibility for all symbols. However,
for people who want to hook up an existing front-end, or to use more
advanced features in their JIT (duplicate symbol names with internal
linkage, symbols with weak linkage, hidden visibility, etc.) this scheme
has the nice property that "What does linker-feature X do in the JIT?"
is
more-or-less equivalent to "What does linker-feature X do in a regular
build". The answer to the latter question is generally more widely known,
and is much easier to test from the command line.

When adding a new module with a single global variable in it, I get
an
> error during the addLazyIRModule.

Are you hitting an assertion, or is addLazyIRModule returning a non-success
value? If the latter, what is the error message?
(You can find this by calling 'logAllUnhandledErrors(std::move(Err),
errs(), "")', or, in tool code, using the ExitOnError utility:
http://llvm.org/docs/ProgrammersManual.html#using-exitonerror-to-simplify-tool-code
).

Right now we are treating the module as one big single module, and when
we
> go to lookup a symbol, I believe we are jitting the entire thing. Once we
> make the switch to LLLazyJIT, we want to start splitting up our modules.

If you are using MCJITReplacement you are definitely JITing the whole
module up-front, and in that case lazy compilation may help reduce your
compile time. I'd love to hear how it works for you!

Cheers,
Lang.

On Mon, Aug 12, 2019 at 10:26 AM David Blaikie <dblaikie at gmail.com>
wrote:
> +Lang Hames <lhames at gmail.com>, Admiral of the Orcish Fleet
(Lang's the
> primary developer/creator of the ORC JIT & might have some answers to
your
> question)
>
> On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> We are trying to switch to the new orc v2 lljit engine at my work with
>> hopes of parallel jitting. We are switching from the ExecutionEngine w/
>> OrcMCJitReplacement. We are having a hard time with global variables.
We
>> have found a way to create/emit new globals during jitting by using the
old
>> ExecutionEngine::getOrEmitGlobalVariable. Is there an easier way to do
this
>> with the new jit engine? We were hoping to create a new
ThreadSafeModule,
>> add a GlobalVariable, and then add the module to the JIT engine through
>> addLazyIRModule(), but this doesn't seem to work.
>>
>> 2nd question; Is the lookup function supposed to work with global
>> variables by name? Looking through the CompileOnDemandLayer it seems to
>> change the names of globals to
"__orc_lcl.<name><id>". Does this affect the
>> lookup methods? I noticed that all of the globals we are looking for
are
>> tagged with Internal linkage and end up getting emitted through
>> extractSubModule().
>>
>> Any help is appreciated.
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190812/f414b50d/attachment.html>

Hi Chris,

Thanks for clarifying that. That makes sense.
>

So I apologize. It was not an error I got on the AddLazyIRModule. The
issue
> is actually after I call AddLazyIRModule(). Performing a lookup on that
> global variable causes a hang. I've stepped through the debugger, and
it
> seems to hang inside Core.cpp, ExecutionSession::lookup().

Are you (a) trying to JIT concurrently and (b) loading all modules on the
same ThreadSafeContext? Notionally you should be able to do both, but in my
experience bugs in this area are the most frequent source of deadlocks in
lookup.

Are you able to get a backtrace for each thread in the debugger while the
lookup is deadlocked? If you're using LLDB you can use "thread
list" to
list the threads, "thread select N" to switch to each, and
"bt" to get the
backtrace. I am not sure what the equivalent commands in GDB are. Either
way, the lookup thread is the least interesting one: I expect to see some
other thread waiting on a context lock, and the thing we *really* want to
know is who owns the lock and how did they end up blocking while holding it.

Are you able to share any of your code / test cases?

Cheers,
Lang.

Prior to r367686, ORCv2 took out a lock on the context when a module was
emitted and did not relinquish it until that module had been fully emitted.
The problem with that is that if the module being emitted has dependencies
on other modules living on the same

On Tue, Aug 13, 2019 at 2:09 PM chris boese <chris107565 at gmail.com>
wrote:
> Hi Lang,
>
> Thanks for clarifying that. That makes sense.
>
> So I apologize. It was not an error I got on the AddLazyIRModule. The
> issue is actually after I call AddLazyIRModule(). Performing a lookup on
> that global variable causes a hang. I've stepped through the debugger,
and
> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>
> Thanks,
> Chris
>
> On Mon, Aug 12, 2019 at 2:33 PM Lang Hames <lhames at gmail.com>
wrote:
>
>> Hi Chris,
>>
>>
>>> Thanks so much for the quick help. I was able to find them after
>>> changing the linkage. Does this new orc JIT follow different
standards than
>>> the MCJitReplacement? We used to set a lot of our functions/globals
to
>>> internal linkage before doing a lookup.
>>
>>
>> They are different: ORCv2 follows the static linker's lookup rules,
>> whereas MCJITReplacement mimicked MCJIT's rules which were never
clearly
>> spelled out, but in practice allowed you to search for internal
symbols.
>>
>> The advantage of the new scheme is that if you are compiling/running
>> something on the command line. E.g.:
>>
>> $ llc -filetype=obj -o extra.o extra.ll
>> $ llc -filetype=obj -o prog.o prog.ll
>> $ clang -o prog prog.o extra.o
>> $ ./prog
>>
>> Then you can add the same modules to the JIT and expect the resulting
>> JIT'd code to behave the same as the statically compiled code. E.g.
>> (omitting some error handling for clarity):
>>
>> J->addIRModule(loadModule("extra.ll"));
>> J->addIRModule(loadModule("main.ll"));
>> auto Main = (int(*)())J->lookup("main").getAddress();
>> Main();
>>
>> If you are generating code specifically for the JIT it is often easiest
>> to just use external linkage and default visibility for all symbols.
>> However, for people who want to hook up an existing front-end, or to
use
>> more advanced features in their JIT (duplicate symbol names with
internal
>> linkage, symbols with weak linkage, hidden visibility, etc.) this
scheme
>> has the nice property that "What does linker-feature X do in the
JIT?" is
>> more-or-less equivalent to "What does linker-feature X do in a
regular
>> build". The answer to the latter question is generally more widely
known,
>> and is much easier to test from the command line.
>>
>> When adding a new module with a single global variable in it, I get an
>>> error during the addLazyIRModule.
>>
>>
>> Are you hitting an assertion, or is addLazyIRModule returning a
>> non-success value? If the latter, what is the error message?
>> (You can find this by calling
'logAllUnhandledErrors(std::move(Err),
>> errs(), "")', or, in tool code, using the ExitOnError
utility:
>>
http://llvm.org/docs/ProgrammersManual.html#using-exitonerror-to-simplify-tool-code
>> ).
>>
>> Right now we are treating the module as one big single module, and when
>>> we go to lookup a symbol, I believe we are jitting the entire
thing. Once
>>> we make the switch to LLLazyJIT, we want to start splitting up our
modules.
>>
>>
>> If you are using MCJITReplacement you are definitely JITing the whole
>> module up-front, and in that case lazy compilation may help reduce your
>> compile time. I'd love to hear how it works for you!
>>
>> Cheers,
>> Lang.
>>
>> On Mon, Aug 12, 2019 at 10:26 AM David Blaikie <dblaikie at
gmail.com>
>> wrote:
>>
>>> +Lang Hames <lhames at gmail.com>, Admiral of the Orcish
Fleet (Lang's the
>>> primary developer/creator of the ORC JIT & might have some
answers to your
>>> question)
>>>
>>> On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>> We are trying to switch to the new orc v2 lljit engine at my
work with
>>>> hopes of parallel jitting. We are switching from the
ExecutionEngine w/
>>>> OrcMCJitReplacement. We are having a hard time with global
variables. We
>>>> have found a way to create/emit new globals during jitting by
using the old
>>>> ExecutionEngine::getOrEmitGlobalVariable. Is there an easier
way to do this
>>>> with the new jit engine? We were hoping to create a new
ThreadSafeModule,
>>>> add a GlobalVariable, and then add the module to the JIT engine
through
>>>> addLazyIRModule(), but this doesn't seem to work.
>>>>
>>>> 2nd question; Is the lookup function supposed to work with
global
>>>> variables by name? Looking through the CompileOnDemandLayer it
seems to
>>>> change the names of globals to
"__orc_lcl.<name><id>". Does this affect the
>>>> lookup methods? I noticed that all of the globals we are
looking for are
>>>> tagged with Internal linkage and end up getting emitted through
>>>> extractSubModule().
>>>>
>>>> Any help is appreciated.
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190813/73deaded/attachment-0001.html>

Hi Chris,

1. Using the addIRModule method for adding new global variables, I'm
having
> to set them to external linkage, so that they can be found during lookup.
> One issue with this, is I have to manually loop through all the underlying
> globals/functions and change their linkage to external as well. If I
don't
> do that, I'll get a JIT Session Error about not being able to find the
> underlying globals/functions. Is there an easy way to set all of the
> underlying globals/functions to external linkage as well?

There is no "change all linkages" convenience function, you just have
to
loop over all the global values and set them individually. If you're doing
that though there's likely a design issue somewhere else: In general
languages will define which symbols get external linkage and which do not,
and the front-end should set up the linkages for you: you shouldn't need to
modify linkages in order to add something to the JIT. E.g. If you have the
following in C:

static void foo(void) { }
void bar(void) { foo(); }

then clang will translate that to:

define internal void @foo() { ... }
define void @bar() { ... }

This IR, with foo having internal linkage and bar external, can be added to
a JITDylib as-is and will result in one visible definition: bar. (ORC can
handle internal symbols just fine, they just don't appear in the public
interface for the JITDylib because that's what internal linkage means:
visible only from within the module).

The two main exceptions to the "you shouldn't need to promote linkage
yourself" rule are:

(1) Debuggers / REPLs, where you want to give a user the ability to call
directly into what would usually be an internal function. In that case you
should only need to promote the linkage on the internal symbols that you
want the user to be able to access. If that's everything, then promote
everything. If that's only certain functions/variables then just promote
those functions/variables.

(2) Manually breaking up modules: If you want to perform more fine grained
or parallel compilation you may choose to break up large modules. In that
case you need to promote internal symbols to external-hidden symbols so
that they can still be resolved across the new module boundaries. You will
usually have to rename them to avoid clashes too. If you're using LLLazyJIT
though you don't need to worry: all the symbol promoting and renaming
required for lazy compilation is taken care of internally.

So: Why is your front-end producing internal definitions for symbols that
you want to be able to call in the JIT? Should your front-end be producing
external definitions?

2. We've had this problem ever since we switched to
OrcMCJitReplacement.
> When jitting code on linux, we have to comment the body of
> registerEHFrames(). If we don't, then any exceptions that are thrown in
> jitted code will result in a segfault. Do you know what issue we might be
> running into here? RuntimeDyIdELF::registerEHFrames() is commented out for
> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
> OrcMCJitReplacement.

registerEHFrames is supposed to register the __eh_frames section with
libunwind. I know that RuntimeDyldMachO's __eh_frame support was weak, and
would often result in registering broken frames. I don't know
RuntimeDyldELF's __eh_frame support code well, but from a glance I suspect
it suffers the same problems. That is almost certainly why your code is
segfaulting when exceptions are thrown: the JIT is registering broken
frames.

The poor quality of RuntimeDyld's exception handling is why I turned it off
by default. The new JIT linker (JTILink) has much better eh_frame support,
but unfortunately there is no ELF JITLink implementation yet.

The upshot of all this is that at the moment it's not safe in general to
throw exceptions from/through JIT'd code on ELF. I'm hoping that JITLink
ELF variants will solve this eventually, but there's no clear timeline for
that work.

-- Lang.

On Mon, Aug 19, 2019 at 11:12 AM chris boese <chris107565 at gmail.com>
wrote:
> Hi Lang,
>
> Had a couple more questions I thought I'd run by you.
>
> 1. Using the addIRModule method for adding new global variables, I'm
> having to set them to external linkage, so that they can be found during
> lookup. One issue with this, is I have to manually loop through all the
> underlying globals/functions and change their linkage to external as well.
> If I don't do that, I'll get a JIT Session Error about not being
able to
> find the underlying globals/functions. Is there an easy way to set all of
> the underlying globals/functions to external linkage as well?
>
> 2. We've had this problem ever since we switched to
OrcMCJitReplacement.
> When jitting code on linux, we have to comment the body of
> registerEHFrames(). If we don't, then any exceptions that are thrown in
> jitted code will result in a segfault. Do you know what issue we might be
> running into here? RuntimeDyIdELF::registerEHFrames() is commented out for
> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
> OrcMCJitReplacement.
>
> Thanks,
> Chris
>
> On Thu, Aug 15, 2019 at 9:22 AM chris boese <chris107565 at
gmail.com> wrote:
>
>> Hi Lang,
>>
>> So that was my issue. Putting modules on the same ThreadSafeContext was
>> causing a deadlock. Our current version of LLVM does not have r367686
>> patch. I resolved by creating a separate ThreadSafeContext, and now
I'm
>> able to add/find globals by using the addLazyIRModule function.
>>
>> Thank you so much for your help. For now I believe I'm unblocked. I
will
>> reach out again if encountering any more issues.
>>
>> Thanks,
>> Chris
>>
>> On Tue, Aug 13, 2019 at 2:49 PM Lang Hames <lhames at gmail.com>
wrote:
>>
>>> Hi Chris,
>>>
>>> Thanks for clarifying that. That makes sense.
>>>>
>>>
>>>
>>> So I apologize. It was not an error I got on the AddLazyIRModule.
The
>>>> issue is actually after I call AddLazyIRModule(). Performing a
lookup on
>>>> that global variable causes a hang. I've stepped through
the debugger, and
>>>> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>>>
>>>
>>> Are you (a) trying to JIT concurrently and (b) loading all modules
on
>>> the same ThreadSafeContext? Notionally you should be able to do
both, but
>>> in my experience bugs in this area are the most frequent source of
>>> deadlocks in lookup.
>>>
>>> Are you able to get a backtrace for each thread in the debugger
while
>>> the lookup is deadlocked? If you're using LLDB you can use
"thread list" to
>>> list the threads, "thread select N" to switch to each,
and "bt" to get the
>>> backtrace. I am not sure what the equivalent commands in GDB are.
Either
>>> way, the lookup thread is the least interesting one: I expect to
see some
>>> other thread waiting on a context lock, and the thing we *really*
want to
>>> know is who owns the lock and how did they end up blocking while
holding it.
>>>
>>> Are you able to share any of your code / test cases?
>>>
>>> Cheers,
>>> Lang.
>>>
>>> Prior to r367686, ORCv2 took out a lock on the context when a
module was
>>> emitted and did not relinquish it until that module had been fully
emitted.
>>> The problem with that is that if the module being emitted has
dependencies
>>> on other modules living on the same
>>>
>>> On Tue, Aug 13, 2019 at 2:09 PM chris boese <chris107565 at
gmail.com>
>>> wrote:
>>>
>>>> Hi Lang,
>>>>
>>>> Thanks for clarifying that. That makes sense.
>>>>
>>>> So I apologize. It was not an error I got on the
AddLazyIRModule. The
>>>> issue is actually after I call AddLazyIRModule(). Performing a
lookup on
>>>> that global variable causes a hang. I've stepped through
the debugger, and
>>>> it seems to hang inside Core.cpp, ExecutionSession::lookup().
>>>>
>>>> Thanks,
>>>> Chris
>>>>
>>>> On Mon, Aug 12, 2019 at 2:33 PM Lang Hames <lhames at
gmail.com> wrote:
>>>>
>>>>> Hi Chris,
>>>>>
>>>>>
>>>>>> Thanks so much for the quick help. I was able to find
them after
>>>>>> changing the linkage. Does this new orc JIT follow
different standards than
>>>>>> the MCJitReplacement? We used to set a lot of our
functions/globals to
>>>>>> internal linkage before doing a lookup.
>>>>>
>>>>>
>>>>> They are different: ORCv2 follows the static linker's
lookup rules,
>>>>> whereas MCJITReplacement mimicked MCJIT's rules which
were never clearly
>>>>> spelled out, but in practice allowed you to search for
internal symbols.
>>>>>
>>>>> The advantage of the new scheme is that if you are
compiling/running
>>>>> something on the command line. E.g.:
>>>>>
>>>>> $ llc -filetype=obj -o extra.o extra.ll
>>>>> $ llc -filetype=obj -o prog.o prog.ll
>>>>> $ clang -o prog prog.o extra.o
>>>>> $ ./prog
>>>>>
>>>>> Then you can add the same modules to the JIT and expect the
resulting
>>>>> JIT'd code to behave the same as the statically
compiled code. E.g.
>>>>> (omitting some error handling for clarity):
>>>>>
>>>>> J->addIRModule(loadModule("extra.ll"));
>>>>> J->addIRModule(loadModule("main.ll"));
>>>>> auto Main =
(int(*)())J->lookup("main").getAddress();
>>>>> Main();
>>>>>
>>>>> If you are generating code specifically for the JIT it is
often
>>>>> easiest to just use external linkage and default visibility
for all
>>>>> symbols. However, for people who want to hook up an
existing front-end, or
>>>>> to use more advanced features in their JIT (duplicate
symbol names with
>>>>> internal linkage, symbols with weak linkage, hidden
visibility, etc.) this
>>>>> scheme has the nice property that "What does
linker-feature X do in the
>>>>> JIT?" is more-or-less equivalent to "What does
linker-feature X do in a
>>>>> regular build". The answer to the latter question is
generally more widely
>>>>> known, and is much easier to test from the command line.
>>>>>
>>>>> When adding a new module with a single global variable in
it, I get an
>>>>>> error during the addLazyIRModule.
>>>>>
>>>>>
>>>>> Are you hitting an assertion, or is addLazyIRModule
returning a
>>>>> non-success value? If the latter, what is the error
message?
>>>>> (You can find this by calling
'logAllUnhandledErrors(std::move(Err),
>>>>> errs(), "")', or, in tool code, using the
ExitOnError utility:
>>>>>
http://llvm.org/docs/ProgrammersManual.html#using-exitonerror-to-simplify-tool-code
>>>>> ).
>>>>>
>>>>> Right now we are treating the module as one big single
module, and
>>>>>> when we go to lookup a symbol, I believe we are jitting
the entire thing.
>>>>>> Once we make the switch to LLLazyJIT, we want to start
splitting up our
>>>>>> modules.
>>>>>
>>>>>
>>>>> If you are using MCJITReplacement you are definitely JITing
the whole
>>>>> module up-front, and in that case lazy compilation may help
reduce your
>>>>> compile time. I'd love to hear how it works for you!
>>>>>
>>>>> Cheers,
>>>>> Lang.
>>>>>
>>>>> On Mon, Aug 12, 2019 at 10:26 AM David Blaikie <dblaikie
at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> +Lang Hames <lhames at gmail.com>, Admiral of the
Orcish Fleet (Lang's
>>>>>> the primary developer/creator of the ORC JIT &
might have some answers to
>>>>>> your question)
>>>>>>
>>>>>> On Thu, Aug 8, 2019 at 4:15 PM chris boese via llvm-dev
<
>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> We are trying to switch to the new orc v2 lljit
engine at my work
>>>>>>> with hopes of parallel jitting. We are switching
from the ExecutionEngine
>>>>>>> w/ OrcMCJitReplacement. We are having a hard time
with global variables. We
>>>>>>> have found a way to create/emit new globals during
jitting by using the old
>>>>>>> ExecutionEngine::getOrEmitGlobalVariable. Is there
an easier way to do this
>>>>>>> with the new jit engine? We were hoping to create a
new ThreadSafeModule,
>>>>>>> add a GlobalVariable, and then add the module to
the JIT engine through
>>>>>>> addLazyIRModule(), but this doesn't seem to
work.
>>>>>>>
>>>>>>> 2nd question; Is the lookup function supposed to
work with global
>>>>>>> variables by name? Looking through the
CompileOnDemandLayer it seems to
>>>>>>> change the names of globals to
"__orc_lcl.<name><id>". Does this affect the
>>>>>>> lookup methods? I noticed that all of the globals
we are looking for are
>>>>>>> tagged with Internal linkage and end up getting
emitted through
>>>>>>> extractSubModule().
>>>>>>>
>>>>>>> Any help is appreciated.
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190819/abf6aea3/attachment.html>

Hi Lang,

Thanks for the info. I think we have issues with all of the JIT "prep"
work
we are currently doing around changing linkage for symbols before jitting.
I believe this was done to work with the OrcMCJitReplacement, but it
doesn't seem compatible with the new LLJIT. I will look into it further. On
second point, we will just turn it off for now.

Thanks,
Chris

On Mon, Aug 19, 2019 at 1:35 PM Lang Hames <lhames at gmail.com> wrote:
> Hi Chris,
>
> 1. Using the addIRModule method for adding new global variables, I'm
>> having to set them to external linkage, so that they can be found
during
>> lookup. One issue with this, is I have to manually loop through all the
>> underlying globals/functions and change their linkage to external as
well.
>> If I don't do that, I'll get a JIT Session Error about not
being able to
>> find the underlying globals/functions. Is there an easy way to set all
of
>> the underlying globals/functions to external linkage as well?
>
>
> There is no "change all linkages" convenience function, you just
have to
> loop over all the global values and set them individually. If you're
doing
> that though there's likely a design issue somewhere else: In general
> languages will define which symbols get external linkage and which do not,
> and the front-end should set up the linkages for you: you shouldn't
need to
> modify linkages in order to add something to the JIT. E.g. If you have the
> following in C:
>
> static void foo(void) { }
> void bar(void) { foo(); }
>
> then clang will translate that to:
>
> define internal void @foo() { ... }
> define void @bar() { ... }
>
> This IR, with foo having internal linkage and bar external, can be added
> to a JITDylib as-is and will result in one visible definition: bar. (ORC
> can handle internal symbols just fine, they just don't appear in the
public
> interface for the JITDylib because that's what internal linkage means:
> visible only from within the module).
>
> The two main exceptions to the "you shouldn't need to promote
linkage
> yourself" rule are:
>
> (1) Debuggers / REPLs, where you want to give a user the ability to call
> directly into what would usually be an internal function. In that case you
> should only need to promote the linkage on the internal symbols that you
> want the user to be able to access. If that's everything, then promote
> everything. If that's only certain functions/variables then just
promote
> those functions/variables.
>
> (2) Manually breaking up modules: If you want to perform more fine grained
> or parallel compilation you may choose to break up large modules. In that
> case you need to promote internal symbols to external-hidden symbols so
> that they can still be resolved across the new module boundaries. You will
> usually have to rename them to avoid clashes too. If you're using
LLLazyJIT
> though you don't need to worry: all the symbol promoting and renaming
> required for lazy compilation is taken care of internally.
>
> So: Why is your front-end producing internal definitions for symbols that
> you want to be able to call in the JIT? Should your front-end be producing
> external definitions?
>
> 2. We've had this problem ever since we switched to
OrcMCJitReplacement.
>> When jitting code on linux, we have to comment the body of
>> registerEHFrames(). If we don't, then any exceptions that are
thrown in
>> jitted code will result in a segfault. Do you know what issue we might
be
>> running into here? RuntimeDyIdELF::registerEHFrames() is commented out
for
>> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
>> OrcMCJitReplacement.
>
>
> registerEHFrames is supposed to register the __eh_frames section with
> libunwind. I know that RuntimeDyldMachO's __eh_frame support was weak,
and
> would often result in registering broken frames. I don't know
> RuntimeDyldELF's __eh_frame support code well, but from a glance I
suspect
> it suffers the same problems. That is almost certainly why your code is
> segfaulting when exceptions are thrown: the JIT is registering broken
> frames.
>
> The poor quality of RuntimeDyld's exception handling is why I turned it
> off by default. The new JIT linker (JTILink) has much better eh_frame
> support, but unfortunately there is no ELF JITLink implementation yet.
>
> The upshot of all this is that at the moment it's not safe in general
to
> throw exceptions from/through JIT'd code on ELF. I'm hoping that
JITLink
> ELF variants will solve this eventually, but there's no clear timeline
for
> that work.
>
> -- Lang.
>
> On Mon, Aug 19, 2019 at 11:12 AM chris boese <chris107565 at
gmail.com>
> wrote:
>
>> Hi Lang,
>>
>> Had a couple more questions I thought I'd run by you.
>>
>> 1. Using the addIRModule method for adding new global variables,
I'm
>> having to set them to external linkage, so that they can be found
during
>> lookup. One issue with this, is I have to manually loop through all the
>> underlying globals/functions and change their linkage to external as
well.
>> If I don't do that, I'll get a JIT Session Error about not
being able to
>> find the underlying globals/functions. Is there an easy way to set all
of
>> the underlying globals/functions to external linkage as well?
>>
>> 2. We've had this problem ever since we switched to
OrcMCJitReplacement.
>> When jitting code on linux, we have to comment the body of
>> registerEHFrames(). If we don't, then any exceptions that are
thrown in
>> jitted code will result in a segfault. Do you know what issue we might
be
>> running into here? RuntimeDyIdELF::registerEHFrames() is commented out
for
>> new LLJIT. OrcMCJitReplacement::registerEHFrames when using
>> OrcMCJitReplacement.
>>
>> Thanks,
>> Chris
>>
>> On Thu, Aug 15, 2019 at 9:22 AM chris boese <chris107565 at
gmail.com>
>> wrote:
>>
>>> Hi Lang,
>>>
>>> So that was my issue. Putting modules on the same ThreadSafeContext
was
>>> causing a deadlock. Our current version of LLVM does not have
r367686
>>> patch. I resolved by creating a separate ThreadSafeContext, and now
I'm
>>> able to add/find globals by using the addLazyIRModule function.
>>>
>>> Thank you so much for your help. For now I believe I'm
unblocked. I will
>>> reach out again if encountering any more issues.
>>>
>>> Thanks,
>>> Chris
>>>
>>> On Tue, Aug 13, 2019 at 2:49 PM Lang Hames <lhames at
gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Thanks for clarifying that. That makes sense.
>>>>>
>>>>
>>>>
>>>> So I apologize. It was not an error I got on the
AddLazyIRModule. The
>>>>> issue is actually after I call AddLazyIRModule().
Performing a lookup on
>>>>> that global variable causes a hang. I've stepped
through the debugger, and
>>>>> it seems to hang inside Core.cpp,
ExecutionSession::lookup().
>>>>
>>>>
>>>> Are you (a) trying to JIT concurrently and (b) loading all
modules on
>>>> the same ThreadSafeContext? Notionally you should be able to do
both, but
>>>> in my experience bugs in this area are the most frequent source
of
>>>> deadlocks in lookup.
>>>>
>>>> Are you able to get a backtrace for each thread in the debugger
while
>>>> the lookup is deadlocked? If you're using LLDB you can use
"thread list" to
>>>> list the threads, "thread select N" to switch to
each, and "bt" to get the
>>>> backtrace. I am not sure what the equivalent commands in GDB
are. Either
>>>> way, the lookup thread is the least interesting one: I expect
to see some
>>>> other thread waiting on a context lock, and the thing we
*really* want to
>>>> know is who owns the lock and how did they end up blocking
while holding it.
>>>>
>>>> Are you able to share any of your code / test cases?
>>>>
>>>> Cheers,
>>>> Lang.
>>>>
>>>> Prior to r367686, ORCv2 took out a lock on the context when a
module
>>>> was emitted and did not relinquish it until that module had
been fully
>>>> emitted. The problem with that is that if the module being
emitted has
>>>> dependencies on other modules living on the same
>>>>
>>>> On Tue, Aug 13, 2019 at 2:09 PM chris boese <chris107565 at
gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Lang,
>>>>>
>>>>> Thanks for clarifying that. That makes sense.
>>>>>
>>>>> So I apologize. It was not an error I got on the
AddLazyIRModule. The
>>>>> issue is actually after I call AddLazyIRModule().
Performing a lookup on
>>>>> that global variable causes a hang. I've stepped
through the debugger, and
>>>>> it seems to hang inside Core.cpp,
ExecutionSession::lookup().
>>>>>
>>>>> Thanks,
>>>>> Chris
>>>>>
>>>>> On Mon, Aug 12, 2019 at 2:33 PM Lang Hames <lhames at
gmail.com> wrote:
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>>
>>>>>>> Thanks so much for the quick help. I was able to
find them after
>>>>>>> changing the linkage. Does this new orc JIT follow
different standards than
>>>>>>> the MCJitReplacement? We used to set a lot of our
functions/globals to
>>>>>>> internal linkage before doing a lookup.
>>>>>>
>>>>>>
>>>>>> They are different: ORCv2 follows the static
linker's lookup rules,
>>>>>> whereas MCJITReplacement mimicked MCJIT's rules
which were never clearly
>>>>>> spelled out, but in practice allowed you to search for
internal symbols.
>>>>>>
>>>>>> The advantage of the new scheme is that if you are
compiling/running
>>>>>> something on the command line. E.g.:
>>>>>>
>>>>>> $ llc -filetype=obj -o extra.o extra.ll
>>>>>> $ llc -filetype=obj -o prog.o prog.ll
>>>>>> $ clang -o prog prog.o extra.o
>>>>>> $ ./prog
>>>>>>
>>>>>> Then you can add the same modules to the JIT and expect
the resulting
>>>>>> JIT'd code to behave the same as the statically
compiled code. E.g.
>>>>>> (omitting some error handling for clarity):
>>>>>>
>>>>>> J->addIRModule(loadModule("extra.ll"));
>>>>>> J->addIRModule(loadModule("main.ll"));
>>>>>> auto Main =
(int(*)())J->lookup("main").getAddress();
>>>>>> Main();
>>>>>>
>>>>>> If you are generating code specifically for the JIT it
is often
>>>>>> easiest to just use external linkage and default
visibility for all
>>>>>> symbols. However, for people who want to hook up an
existing front-end, or
>>>>>> to use more advanced features in their JIT (duplicate
symbol names with
>>>>>> internal linkage, symbols with weak linkage, hidden
visibility, etc.) this
>>>>>> scheme has the nice property that "What does
linker-feature X do in the
>>>>>> JIT?" is more-or-less equivalent to "What
does linker-feature X do in a
>>>>>> regular build". The answer to the latter question
is generally more widely
>>>>>> known, and is much easier to test from the command
line.
>>>>>>
>>>>>> When adding a new module with a single global variable
in it, I get
>>>>>>> an error during the addLazyIRModule.
>>>>>>
>>>>>>
>>>>>> Are you hitting an assertion, or is addLazyIRModule
returning a
>>>>>> non-success value? If the latter, what is the error
message?
>>>>>> (You can find this by calling
'logAllUnhandledErrors(std::move(Err),
>>>>>> errs(), "")', or, in tool code, using the
ExitOnError utility:
>>>>>>
http://llvm.org/docs/ProgrammersManual.html#using-exitonerror-to-simplify-tool-code
>>>>>> ).
>>>>>>
>>>>>> Right now we are treating the module as one big single
module, and
>>>>>>> when we go to lookup a symbol, I believe we are
jitting the entire thing.
>>>>>>> Once we make the switch to LLLazyJIT, we want to
start splitting up our
>>>>>>> modules.
>>>>>>
>>>>>>
>>>>>> If you are using MCJITReplacement you are definitely
JITing the whole
>>>>>> module up-front, and in that case lazy compilation may
help reduce your
>>>>>> compile time. I'd love to hear how it works for
you!
>>>>>>
>>>>>> Cheers,
>>>>>> Lang.
>>>>>>
>>>>>> On Mon, Aug 12, 2019 at 10:26 AM David Blaikie
<dblaikie at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> +Lang Hames <lhames at gmail.com>, Admiral of
the Orcish Fleet (Lang's
>>>>>>> the primary developer/creator of the ORC JIT &
might have some answers to
>>>>>>> your question)
>>>>>>>
>>>>>>> On Thu, Aug 8, 2019 at 4:15 PM chris boese via
llvm-dev <
>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> We are trying to switch to the new orc v2 lljit
engine at my work
>>>>>>>> with hopes of parallel jitting. We are
switching from the ExecutionEngine
>>>>>>>> w/ OrcMCJitReplacement. We are having a hard
time with global variables. We
>>>>>>>> have found a way to create/emit new globals
during jitting by using the old
>>>>>>>> ExecutionEngine::getOrEmitGlobalVariable. Is
there an easier way to do this
>>>>>>>> with the new jit engine? We were hoping to
create a new ThreadSafeModule,
>>>>>>>> add a GlobalVariable, and then add the module
to the JIT engine through
>>>>>>>> addLazyIRModule(), but this doesn't seem to
work.
>>>>>>>>
>>>>>>>> 2nd question; Is the lookup function supposed
to work with global
>>>>>>>> variables by name? Looking through the
CompileOnDemandLayer it seems to
>>>>>>>> change the names of globals to
"__orc_lcl.<name><id>". Does this affect the
>>>>>>>> lookup methods? I noticed that all of the
globals we are looking for are
>>>>>>>> tagged with Internal linkage and end up getting
emitted through
>>>>>>>> extractSubModule().
>>>>>>>>
>>>>>>>> Any help is appreciated.
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>
>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190821/ebc8d660/attachment.html>