llvm dev - Sep 2019 - Orc JIT vs. implicit template instanciation in LLVM 8

I have found the cause of our problems with implicit template
instantiation, and it looks to me like a bug in Orc JIT in LLVM 8.
(Perhaps Lang could confirm?)

We use createLegacyLookupResolver() to create our symbol resolver,
passing it a lookup function. Amongst other things, our function calls
LegacyRTDyldObjectLinkingLayer::findSymbol() to look up symbols in the
JIT.

When a module is finalized, the legacy lookup resolver calls
getResponsibilitySetWithLegacyFn() to find any symbols with weak
linkage (e.g. function templates) that need to be materialized; that
function calls our symbol resolver, which calls
LegacyRTDyldObjectLinkingLayer::findSymbol(). The problem is,
findSymbol() returns a null JITSymbol (address 0) if the symbol has
not been materialized yet. And naturally it hasn't, since we are
trying to determine which symbols need to be materialized...

My quick fix is to change these lines in
LegacyRTDyldObjectLinkingLayerBase::LinkedObject::getSymbol():

      if (!Finalized)
        return JITSymbol(getSymbolMaterializer(Name),
                         SymEntry->second.getFlags());

to:

      if (!Finalized || SymEntry->second.getAddress() == 0)
        return JITSymbol(getSymbolMaterializer(Name),
                         SymEntry->second.getFlags());

That works for me, but I am not sure it is the right way to do it...
Perhaps somebody else has a better idea?

Geoff

On Mon, Sep 16, 2019 at 3:56 PM Geoff Levner <glevner at gmail.com>
wrote:
>
> In my test case it seems my module does define the symbol for the
> missing function template, with weak linkage. But when the module is
> finalized, LegacyRTDyldObjectLinkingLayer fails to assign it an
> address, for some reason.
>
> I will continue to dig, but if anyone with a better understanding of
> the code has any ideas, I would appreciate it...
>
> Geoff
>
> On Sun, Sep 15, 2019 at 6:01 PM Geoff Levner <glevner at gmail.com>
wrote:
> >
> > Well, I agree the front end must be responsible for (not)
> > instantiating the function templates. However the problem shows up
> > only when JIT compiling, for some reason, and only with LLVM 8. Is
> > there perhaps something else that needs to be done before resolving
> > symbols, in addition to running constructors, to ensure that
> > instantiated function templates are taken into account?
> >
> > Or perhaps there is some option we are not passing to the front end
> > (clang -std=c++11) that tells it, "do instantiate function
templates
> > as needed"...?
> >
> > On Fri, Sep 13, 2019 at 8:13 PM David Blaikie <dblaikie at
gmail.com> wrote:
> > >
> > > This sounds like it's beyond the domain of ORC though - if
the
> > > function definition is missing, ORC (at least the bits in LLVM)
know
> > > nothing about your frontend or how to request it to manifest that
> > > function. In your old version, when/where/how did your frontend
> > > generate LLVM IR for this function? I'd check that and then
see why a
> > > similar thing isn't happening in the new version.
> > >
> > > - Dave
> > >
> > > On Fri, Sep 13, 2019 at 8:03 AM Geoff Levner via llvm-dev
> > > <llvm-dev at lists.llvm.org> wrote:
> > > >
> > > > The short version:
> > > >
> > > > When I port our application from LLVM 7 to 8, some
JIT-compiled code
> > > > seems to break the implicit instanciation of C++ templates.
That is,
> > > > when I try to resolve a symbol defined by the code, I get
undefined
> > > > symbols corresponding to methods that are actually function
templates.
> > > > If the code adds explicit template instanciations, the
problem goes
> > > > away.
> > > >
> > > > Does anybody have an intuition as to what might cause this?
> > > >
> > > > Now the longer version:
> > > >
> > > > Specifically, we use the Orc JIT (v1) to compile and execute
C++ code
> > > > on the fly. To move to LLVM 8, I am using the corresponding
"legacy"
> > > > classes before attempting the move to Orc v2. But when I do
that, some
> > > > code which compiles fine with LLVM 7 breaks. I might get an
undefined
> > > > symbol like the following when I try to resolve a symbol
defined by
> > > > the code, after running constructors (successfully):
> > > >
> > > >     _ZNSt6vectorIfSaIfEE17_M_default_appendEm
> > > >
> > > > Unmangled, that's:
> > > >
> > > >     std::vector<float, std::allocator<float>
>::_M_default_append(unsigned long)
> > > >
> > > > which is a function template defined by the C++ vector
header file. If
> > > > I add an explicit instanciation like so:
> > > >
> > > >     template class std::vector<float>;
> > > >
> > > > the problem goes away. I have tried to reproduce the problem
with the
> > > > Clang front end to no avail, so I assume it is specific to
symbol
> > > > resolution in Orc JIT. Has something changed there between
LLVM 7 and
> > > > 8?
> > > >
> > > > Any help would be much appreciated!
> > > >
> > > > Thanks,
> > > > Geoff
> > > > _______________________________________________
> > > > LLVM Developers mailing list
> > > > llvm-dev at lists.llvm.org
> > > > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi Geoff,

Oof. I don't know what this is yet, but I bet it's going to be awful.
Quick
apology up-front: I'm sorry about the state of weak symbol support in ORCv1
/ Legacy: it was always best effort. ORCv2's solution to these problems is
much more robust, but required fundamentally changing the resolution model
(i.e. introducing the idea of responsibility sets), which may have shaken
loose other bugs in ORCv1/Legacy handling of these symbols.

The problem is, findSymbol() returns a null JITSymbol (address 0) if
the
> symbol has not been materialized yet. And naturally it hasn't, since we
are
> trying to determine which symbols need to be materialized...

If Finalized == false, then we should be hitting this:

  if (!Finalized)
    return JITSymbol(getSymbolMaterializer(Name),
                     SymEntry->second.getFlags());

That's not a null symbol, it's a lazy symbol with a materializer
attached.
It *should* compare non-null in the following test
in getResponsibilitySetWithLegacyFn, which in turn should cause it to be
added to the responsibility set:

    if (JITSymbol Sym = FindSymbol(*S)) {
      if (!Sym.getFlags().isStrong())
        Result.insert(S);                  //
<- _ZNSt6vectorIfSaIfEE17_M_default_appendEm should be added here.
    } else if (auto Err = Sym.takeError())
      return std::move(Err);

Is that not happening?

Cheers,
Lang.


On Mon, Sep 16, 2019 at 9:36 AM Geoff Levner <glevner at gmail.com> wrote:
> I have found the cause of our problems with implicit template
> instantiation, and it looks to me like a bug in Orc JIT in LLVM 8.
> (Perhaps Lang could confirm?)
>
> We use createLegacyLookupResolver() to create our symbol resolver,
> passing it a lookup function. Amongst other things, our function calls
> LegacyRTDyldObjectLinkingLayer::findSymbol() to look up symbols in the
> JIT.
>
> When a module is finalized, the legacy lookup resolver calls
> getResponsibilitySetWithLegacyFn() to find any symbols with weak
> linkage (e.g. function templates) that need to be materialized; that
> function calls our symbol resolver, which calls
> LegacyRTDyldObjectLinkingLayer::findSymbol(). The problem is,
> findSymbol() returns a null JITSymbol (address 0) if the symbol has
> not been materialized yet. And naturally it hasn't, since we are
> trying to determine which symbols need to be materialized...
>
> My quick fix is to change these lines in
> LegacyRTDyldObjectLinkingLayerBase::LinkedObject::getSymbol():
>
>       if (!Finalized)
>         return JITSymbol(getSymbolMaterializer(Name),
>                          SymEntry->second.getFlags());
>
> to:
>
>       if (!Finalized || SymEntry->second.getAddress() == 0)
>         return JITSymbol(getSymbolMaterializer(Name),
>                          SymEntry->second.getFlags());
>
> That works for me, but I am not sure it is the right way to do it...
> Perhaps somebody else has a better idea?
>
> Geoff
>
> On Mon, Sep 16, 2019 at 3:56 PM Geoff Levner <glevner at gmail.com>
wrote:
> >
> > In my test case it seems my module does define the symbol for the
> > missing function template, with weak linkage. But when the module is
> > finalized, LegacyRTDyldObjectLinkingLayer fails to assign it an
> > address, for some reason.
> >
> > I will continue to dig, but if anyone with a better understanding of
> > the code has any ideas, I would appreciate it...
> >
> > Geoff
> >
> > On Sun, Sep 15, 2019 at 6:01 PM Geoff Levner <glevner at
gmail.com> wrote:
> > >
> > > Well, I agree the front end must be responsible for (not)
> > > instantiating the function templates. However the problem shows
up
> > > only when JIT compiling, for some reason, and only with LLVM 8.
Is
> > > there perhaps something else that needs to be done before
resolving
> > > symbols, in addition to running constructors, to ensure that
> > > instantiated function templates are taken into account?
> > >
> > > Or perhaps there is some option we are not passing to the front
end
> > > (clang -std=c++11) that tells it, "do instantiate function
templates
> > > as needed"...?
> > >
> > > On Fri, Sep 13, 2019 at 8:13 PM David Blaikie <dblaikie at
gmail.com>
> wrote:
> > > >
> > > > This sounds like it's beyond the domain of ORC though -
if the
> > > > function definition is missing, ORC (at least the bits in
LLVM) know
> > > > nothing about your frontend or how to request it to manifest
that
> > > > function. In your old version, when/where/how did your
frontend
> > > > generate LLVM IR for this function? I'd check that and
then see why a
> > > > similar thing isn't happening in the new version.
> > > >
> > > > - Dave
> > > >
> > > > On Fri, Sep 13, 2019 at 8:03 AM Geoff Levner via llvm-dev
> > > > <llvm-dev at lists.llvm.org> wrote:
> > > > >
> > > > > The short version:
> > > > >
> > > > > When I port our application from LLVM 7 to 8, some
JIT-compiled
> code
> > > > > seems to break the implicit instanciation of C++
templates. That
> is,
> > > > > when I try to resolve a symbol defined by the code, I
get undefined
> > > > > symbols corresponding to methods that are actually
function
> templates.
> > > > > If the code adds explicit template instanciations, the
problem goes
> > > > > away.
> > > > >
> > > > > Does anybody have an intuition as to what might cause
this?
> > > > >
> > > > > Now the longer version:
> > > > >
> > > > > Specifically, we use the Orc JIT (v1) to compile and
execute C++
> code
> > > > > on the fly. To move to LLVM 8, I am using the
corresponding
> "legacy"
> > > > > classes before attempting the move to Orc v2. But when
I do that,
> some
> > > > > code which compiles fine with LLVM 7 breaks. I might
get an
> undefined
> > > > > symbol like the following when I try to resolve a
symbol defined by
> > > > > the code, after running constructors (successfully):
> > > > >
> > > > >     _ZNSt6vectorIfSaIfEE17_M_default_appendEm
> > > > >
> > > > > Unmangled, that's:
> > > > >
> > > > >     std::vector<float, std::allocator<float>
> >::_M_default_append(unsigned long)
> > > > >
> > > > > which is a function template defined by the C++ vector
header
> file. If
> > > > > I add an explicit instanciation like so:
> > > > >
> > > > >     template class std::vector<float>;
> > > > >
> > > > > the problem goes away. I have tried to reproduce the
problem with
> the
> > > > > Clang front end to no avail, so I assume it is specific
to symbol
> > > > > resolution in Orc JIT. Has something changed there
between LLVM 7
> and
> > > > > 8?
> > > > >
> > > > > Any help would be much appreciated!
> > > > >
> > > > > Thanks,
> > > > > Geoff
> > > > > _______________________________________________
> > > > > LLVM Developers mailing list
> > > > > llvm-dev at lists.llvm.org
> > > > >
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190916/57193942/attachment.html>

No, the problem is that Finalized is true. It is set to true at the start
of the finalize() method. That's why I added a test causing that line to be
executed if the symbol's address is zero...

On Tue, 17 Sep 2019, 00:05 Lang Hames, <lhames at gmail.com> wrote:
> Hi Geoff,
>
> Oof. I don't know what this is yet, but I bet it's going to be
awful.
> Quick apology up-front: I'm sorry about the state of weak symbol
support in
> ORCv1 / Legacy: it was always best effort. ORCv2's solution to these
> problems is much more robust, but required fundamentally changing the
> resolution model (i.e. introducing the idea of responsibility sets), which
> may have shaken loose other bugs in ORCv1/Legacy handling of these symbols.
>
> The problem is, findSymbol() returns a null JITSymbol (address 0) if the
>> symbol has not been materialized yet. And naturally it hasn't,
since we are
>> trying to determine which symbols need to be materialized...
>
>
> If Finalized == false, then we should be hitting this:
>
>   if (!Finalized)
>     return JITSymbol(getSymbolMaterializer(Name),
>                      SymEntry->second.getFlags());
>
> That's not a null symbol, it's a lazy symbol with a materializer
attached.
> It *should* compare non-null in the following test
> in getResponsibilitySetWithLegacyFn, which in turn should cause it to be
> added to the responsibility set:
>
>     if (JITSymbol Sym = FindSymbol(*S)) {
>       if (!Sym.getFlags().isStrong())
>         Result.insert(S);                  //
> <- _ZNSt6vectorIfSaIfEE17_M_default_appendEm should be added here.
>     } else if (auto Err = Sym.takeError())
>       return std::move(Err);
>
> Is that not happening?
>
> Cheers,
> Lang.
>
>
> On Mon, Sep 16, 2019 at 9:36 AM Geoff Levner <glevner at gmail.com>
wrote:
>
>> I have found the cause of our problems with implicit template
>> instantiation, and it looks to me like a bug in Orc JIT in LLVM 8.
>> (Perhaps Lang could confirm?)
>>
>> We use createLegacyLookupResolver() to create our symbol resolver,
>> passing it a lookup function. Amongst other things, our function calls
>> LegacyRTDyldObjectLinkingLayer::findSymbol() to look up symbols in the
>> JIT.
>>
>> When a module is finalized, the legacy lookup resolver calls
>> getResponsibilitySetWithLegacyFn() to find any symbols with weak
>> linkage (e.g. function templates) that need to be materialized; that
>> function calls our symbol resolver, which calls
>> LegacyRTDyldObjectLinkingLayer::findSymbol(). The problem is,
>> findSymbol() returns a null JITSymbol (address 0) if the symbol has
>> not been materialized yet. And naturally it hasn't, since we are
>> trying to determine which symbols need to be materialized...
>>
>> My quick fix is to change these lines in
>> LegacyRTDyldObjectLinkingLayerBase::LinkedObject::getSymbol():
>>
>>       if (!Finalized)
>>         return JITSymbol(getSymbolMaterializer(Name),
>>                          SymEntry->second.getFlags());
>>
>> to:
>>
>>       if (!Finalized || SymEntry->second.getAddress() == 0)
>>         return JITSymbol(getSymbolMaterializer(Name),
>>                          SymEntry->second.getFlags());
>>
>> That works for me, but I am not sure it is the right way to do it...
>> Perhaps somebody else has a better idea?
>>
>> Geoff
>>
>> On Mon, Sep 16, 2019 at 3:56 PM Geoff Levner <glevner at
gmail.com> wrote:
>> >
>> > In my test case it seems my module does define the symbol for the
>> > missing function template, with weak linkage. But when the module
is
>> > finalized, LegacyRTDyldObjectLinkingLayer fails to assign it an
>> > address, for some reason.
>> >
>> > I will continue to dig, but if anyone with a better understanding
of
>> > the code has any ideas, I would appreciate it...
>> >
>> > Geoff
>> >
>> > On Sun, Sep 15, 2019 at 6:01 PM Geoff Levner <glevner at
gmail.com> wrote:
>> > >
>> > > Well, I agree the front end must be responsible for (not)
>> > > instantiating the function templates. However the problem
shows up
>> > > only when JIT compiling, for some reason, and only with LLVM
8. Is
>> > > there perhaps something else that needs to be done before
resolving
>> > > symbols, in addition to running constructors, to ensure that
>> > > instantiated function templates are taken into account?
>> > >
>> > > Or perhaps there is some option we are not passing to the
front end
>> > > (clang -std=c++11) that tells it, "do instantiate
function templates
>> > > as needed"...?
>> > >
>> > > On Fri, Sep 13, 2019 at 8:13 PM David Blaikie <dblaikie at
gmail.com>
>> wrote:
>> > > >
>> > > > This sounds like it's beyond the domain of ORC
though - if the
>> > > > function definition is missing, ORC (at least the bits
in LLVM) know
>> > > > nothing about your frontend or how to request it to
manifest that
>> > > > function. In your old version, when/where/how did your
frontend
>> > > > generate LLVM IR for this function? I'd check that
and then see why
>> a
>> > > > similar thing isn't happening in the new version.
>> > > >
>> > > > - Dave
>> > > >
>> > > > On Fri, Sep 13, 2019 at 8:03 AM Geoff Levner via
llvm-dev
>> > > > <llvm-dev at lists.llvm.org> wrote:
>> > > > >
>> > > > > The short version:
>> > > > >
>> > > > > When I port our application from LLVM 7 to 8, some
JIT-compiled
>> code
>> > > > > seems to break the implicit instanciation of C++
templates. That
>> is,
>> > > > > when I try to resolve a symbol defined by the code,
I get
>> undefined
>> > > > > symbols corresponding to methods that are actually
function
>> templates.
>> > > > > If the code adds explicit template instanciations,
the problem
>> goes
>> > > > > away.
>> > > > >
>> > > > > Does anybody have an intuition as to what might
cause this?
>> > > > >
>> > > > > Now the longer version:
>> > > > >
>> > > > > Specifically, we use the Orc JIT (v1) to compile
and execute C++
>> code
>> > > > > on the fly. To move to LLVM 8, I am using the
corresponding
>> "legacy"
>> > > > > classes before attempting the move to Orc v2. But
when I do that,
>> some
>> > > > > code which compiles fine with LLVM 7 breaks. I
might get an
>> undefined
>> > > > > symbol like the following when I try to resolve a
symbol defined
>> by
>> > > > > the code, after running constructors
(successfully):
>> > > > >
>> > > > >     _ZNSt6vectorIfSaIfEE17_M_default_appendEm
>> > > > >
>> > > > > Unmangled, that's:
>> > > > >
>> > > > >     std::vector<float,
std::allocator<float>
>> >::_M_default_append(unsigned long)
>> > > > >
>> > > > > which is a function template defined by the C++
vector header
>> file. If
>> > > > > I add an explicit instanciation like so:
>> > > > >
>> > > > >     template class std::vector<float>;
>> > > > >
>> > > > > the problem goes away. I have tried to reproduce
the problem with
>> the
>> > > > > Clang front end to no avail, so I assume it is
specific to symbol
>> > > > > resolution in Orc JIT. Has something changed there
between LLVM 7
>> and
>> > > > > 8?
>> > > > >
>> > > > > Any help would be much appreciated!
>> > > > >
>> > > > > Thanks,
>> > > > > Geoff
>> > > > > _______________________________________________
>> > > > > LLVM Developers mailing list
>> > > > > llvm-dev at lists.llvm.org
>> > > > >
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190917/6c681ab6/attachment.html>