llvm dev - Apr 2020 - LLVM 10 ORC2 issue with symbol resolution

Hi Dibyendu,

I had a look - to be honest I am not sure where the error is
being
> reported from. It looks like this:
> JIT session error: Symbols not found: [ luaV_tointeger_, luaG_runerror ]
> I don't think it is from any logging I am doing.

Errors in ORC may trigger cascading failures. E.g. if two different modules
M1 and M2 both reference a function "foo" which cannot be resolved,
then
you have three errors: A failure to resolve "foo", A failure to
materialize
the symbols in M1, and a failure to materialize the symbols in M2. When
such a cascade occurs ORC will usually report the original error(s) via
ExecutionSession::reportError (which logs them to stderr by default) and
return a derived error (e.g. failure-to-materialize) to callers. I believe
you're seeing the logged error, and you will see a failure-to-materialize
error if you log the result that reaches cantFail in your example above.

-- Lang.


On Mon, Apr 13, 2020 at 1:18 PM Dibyendu Majumdar <mobile at
majumdar.org.uk>
wrote:
> Hi Lang,
>
>
> > > Can you share the specific errors that you're getting back? I
would
> recommend replacing
> > >
> > > cantFail(callToFailingFunc(...));
> > >
> > > with
> > >
> > > logAllUnhandledErrors(callToFailingFunc(...), errs(),
> "callToFailingFunc(...) failed:");
> > >
> >
> > Okay thank you - I will do that and report back.
>
>
> I had a look - to be honest I am not sure where the error is being
> reported from. It looks like this:
>
> JIT session error: Symbols not found: [ luaV_tointeger_, luaG_runerror ]
>
> I don't think it is from any logging I am doing.
>
> >
> > > On Mon, Apr 13, 2020 at 10:06 AM Dibyendu Majumdar <
> mobile at majumdar.org.uk> wrote:
> > >>
> > >> Hi,
> > >>
> > >> I updated my project to LLVM 10.0 today and I am getting JIT
symbol
> > >> resolution errors.
> > >> I could not find any example or updated tutorial or
documentation that
> > >> describes the new api - as all documentation seems out of
date.
> > >>
> > >> I paste below some code snippets that show what I am doing:
> > >>
> > >> /* global syms is a array mapping names to function addresses
*/
> > >>
> > >> ES->createJITDylib("<main>");
> > >> MainJD = ES->getJITDylibByName("<main>");
> > >> MainJD->addGenerator(
> > >>
>  cantFail(llvm::orc::DynamicLibrarySearchGenerator::GetForCurrentProcess(
> > >>         DL->getGlobalPrefix())));
> > >> auto &JD = *MainJD;
> > >> llvm::orc::MangleAndInterner mangle(*ES, *this->DL);
> > >> llvm::orc::SymbolMap Symbols;
> > >> for (int i = 0; global_syms[i].name != nullptr; i++) {
> > >>   Symbols.insert({mangle(global_syms[i].name),
> > >>
> > >>
>
llvm::JITEvaluatedSymbol(llvm::pointerToJITTargetAddress(global_syms[i].address),
> > >>
> > >>
llvm::JITSymbolFlags(llvm::JITSymbolFlags::FlagNames::Absolute))});
> > >> }
> > >>
llvm::cantFail(JD.define(llvm::orc::absoluteSymbols(Symbols)), "Failed
> > >> to install extern symbols");
> > >>
> > >> FYI - my previous post on similar issue.
> > >> Basically I haven't been able to get ORC v2 working -
tried in 8, 9
> and now 10.
> > >>
> > >>
>
http://llvm.1065342.n5.nabble.com/llvm-dev-ORC-v2-question-tp130489p130601.html
> > >>
> > >> Thanks and Regards
> > >> Dibyendu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200413/5fde3108/attachment.html>

Hi Lang,

On Mon, 13 Apr 2020 at 21:26, Lang Hames <lhames at gmail.com>
wrote:
>
>> I had a look - to be honest I am not sure where the error is being
>> reported from. It looks like this:
>> JIT session error: Symbols not found: [ luaV_tointeger_, luaG_runerror
]
>> I don't think it is from any logging I am doing.
>
>
> Errors in ORC may trigger cascading failures. E.g. if two different modules
M1 and M2 both reference a function "foo" which cannot be resolved,
then you have three errors: A failure to resolve "foo", A failure to
materialize the symbols in M1, and a failure to materialize the symbols in M2.
When such a cascade occurs ORC will usually report the original error(s) via
ExecutionSession::reportError (which logs them to stderr by default) and return
a derived error (e.g. failure-to-materialize) to callers. I believe you're
seeing the logged error, and you will see a failure-to-materialize error if you
log the result that reaches cantFail in your example above.
>
I am a bit confused about this. I assumed that cantFail() will abort
the process on error - but my process doesn't abort.
I have cantFail() in a few places, I am not sure if you are suggesting
adding the error logging everywhere ...

Thanks and Regards
Dibyendu

Hi Dibyendu,

I am a bit confused about this. I assumed that cantFail() will
abort
> the process on error - but my process doesn't abort.
> I have cantFail() in a few places, I am not sure if you are suggesting
> adding the error logging everywhere ...

Are you building LLVM in +Asserts mode? I believe that cantFail should
abort in +Asserts builds, but it's effectively a no-op if asserts are
disabled.

If you are building in +Asserts mode it could also be consumeError: that
function is intended to silently consume errors.

Both cantFail and consumeError should be used very carefully. The cantFail
function should only be used if you are 100% certain that an error can
never occur at runtime if program invariants hold. E.g. adding a symbol to
a newly created JITDylib can't fail because there are no existing
definitions to clash with. The consumeError function should only be used if
you have genuinely dealt with whatever error condition arose. If you think
that an error is being silently dropped please see if you can trace it by
putting a breakpoint on the ErrorInfoBase constructor and destructor, then
following any errors generated up the stack to the point where they're
dropped. Silently dropped errors (in +Asserts mode) are likely bugs in the
llvm::Error system and we definitely want to get them fixed.

Regards,
Lang.

On Mon, Apr 13, 2020 at 1:46 PM Dibyendu Majumdar <mobile at
majumdar.org.uk>
wrote:
> Hi Lang,
>
> On Mon, 13 Apr 2020 at 21:26, Lang Hames <lhames at gmail.com> wrote:
> >
> >> I had a look - to be honest I am not sure where the error is being
> >> reported from. It looks like this:
> >> JIT session error: Symbols not found: [ luaV_tointeger_,
luaG_runerror ]
> >> I don't think it is from any logging I am doing.
> >
> >
> > Errors in ORC may trigger cascading failures. E.g. if two different
> modules M1 and M2 both reference a function "foo" which cannot be
resolved,
> then you have three errors: A failure to resolve "foo", A failure
to
> materialize the symbols in M1, and a failure to materialize the symbols in
> M2. When such a cascade occurs ORC will usually report the original
> error(s) via ExecutionSession::reportError (which logs them to stderr by
> default) and return a derived error (e.g. failure-to-materialize) to
> callers. I believe you're seeing the logged error, and you will see a
> failure-to-materialize error if you log the result that reaches cantFail in
> your example above.
> >
>
> I am a bit confused about this. I assumed that cantFail() will abort
> the process on error - but my process doesn't abort.
> I have cantFail() in a few places, I am not sure if you are suggesting
> adding the error logging everywhere ...
>
> Thanks and Regards
> Dibyendu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200413/6f53b898/attachment.html>