llvm dev - Jul 2020 - ORC error: No callback manager available

Hi LLVM Developers,

I have a query about an LLVM error message that I'm encountering and I
hope it's appropriate to post to this mailing list about it.

Postgres can be built with LLVM-based JIT support.
I find that when running on Linux (RHEL8) on IBM Z hardware, using
LLVM8+, I'm seeing many instances of the following LLVM error message
appearing in the Postgres server log:

    ORC error: No callback manager available for s390x-ibm-linux

(strangely, it otherwise doesn't seem to be causing any problems)

Looking through the LLVM source code, it appears that for the
"systemz" architecture, there is  currently no local CCMgr
implementation available, and this then prevents indirection support
and COD layer support. BUT - apparently it can still function (maybe
less efficiently?) without these?
Is somebody able to clarify exactly what that error message means and
what the limitations are? Is it unwise to use LLVM on architectures on
which this error is reported?
I looked through the LLVM V10 source code and there still seems to be
this issue for systemz.

Secondly, LLVM sends this error message to standard error. As a caller
of the LLVM library, it would be nice to be able to redirect such
messages from the library - is it possible?

Hope you can advise,
Greg

+Lang Hames, Lord of ORCs

On Thu, Jun 25, 2020 at 4:49 AM Greg Nancarrow via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hi LLVM Developers,
>
> I have a query about an LLVM error message that I'm encountering and I
> hope it's appropriate to post to this mailing list about it.
>
> Postgres can be built with LLVM-based JIT support.
> I find that when running on Linux (RHEL8) on IBM Z hardware, using
> LLVM8+, I'm seeing many instances of the following LLVM error message
> appearing in the Postgres server log:
>
>     ORC error: No callback manager available for s390x-ibm-linux
>
> (strangely, it otherwise doesn't seem to be causing any problems)
>
> Looking through the LLVM source code, it appears that for the
> "systemz" architecture, there is  currently no local CCMgr
> implementation available, and this then prevents indirection support
> and COD layer support. BUT - apparently it can still function (maybe
> less efficiently?) without these?
> Is somebody able to clarify exactly what that error message means and
> what the limitations are? Is it unwise to use LLVM on architectures on
> which this error is reported?
> I looked through the LLVM V10 source code and there still seems to be
> this issue for systemz.
>
> Secondly, LLVM sends this error message to standard error. As a caller
> of the LLVM library, it would be nice to be able to redirect such
> messages from the library - is it possible?
>
> Hope you can advise,
> Greg
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi,

Any update on this? Anyone?

Greg


On Fri, Jun 26, 2020 at 3:04 AM David Blaikie <dblaikie at gmail.com>
wrote:
>
> +Lang Hames, Lord of ORCs
>
> On Thu, Jun 25, 2020 at 4:49 AM Greg Nancarrow via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > Hi LLVM Developers,
> >
> > I have a query about an LLVM error message that I'm encountering
and I
> > hope it's appropriate to post to this mailing list about it.
> >
> > Postgres can be built with LLVM-based JIT support.
> > I find that when running on Linux (RHEL8) on IBM Z hardware, using
> > LLVM8+, I'm seeing many instances of the following LLVM error
message
> > appearing in the Postgres server log:
> >
> >     ORC error: No callback manager available for s390x-ibm-linux
> >
> > (strangely, it otherwise doesn't seem to be causing any problems)
> >
> > Looking through the LLVM source code, it appears that for the
> > "systemz" architecture, there is  currently no local CCMgr
> > implementation available, and this then prevents indirection support
> > and COD layer support. BUT - apparently it can still function (maybe
> > less efficiently?) without these?
> > Is somebody able to clarify exactly what that error message means and
> > what the limitations are? Is it unwise to use LLVM on architectures on
> > which this error is reported?
> > I looked through the LLVM V10 source code and there still seems to be
> > this issue for systemz.
> >
> > Secondly, LLVM sends this error message to standard error. As a caller
> > of the LLVM library, it would be nice to be able to redirect such
> > messages from the library - is it possible?
> >
> > Hope you can advise,
> > Greg
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi Greg,

Thanks for your patience.

Looking through the LLVM source code, it appears that for
the
> "systemz" architecture, there is  currently no local CCMgr
> implementation available, and this then prevents indirection support
> and COD layer support. BUT - apparently it can still function (maybe
> less efficiently?) without these?
> Is somebody able to clarify exactly what that error message means and
> what the limitations are? Is it unwise to use LLVM on architectures on
> which this error is reported?
> I looked through the LLVM V10 source code and there still seems to be
> this issue for systemz.

You are correct: The error is generated when a call is made to the
createLocalCompileCallbackManager function. Platforms that don't support a
callback manager (yet) can not use lazy compilation, but can still JIT
whole modules.

I suspect that postgres is using the older ORC C bindings. These always try
to enable lazy compilation support and direct any errors to stderr.
Postgress probably isn't using the lazy compilation support anyway, which
is why everything works just fine despite the error.

These older APIs are deprecated, so the best solution would be for postgres
to move to the newer C APIs (see
https://github.com/llvm/llvm-project/blob/master/llvm/include/llvm-c/Orc.h).

Are you working on Postgres yourself, or just building it?

Regards,
Lang.

On Thu, Jun 25, 2020 at 4:49 AM Greg Nancarrow via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi LLVM Developers,
>
> I have a query about an LLVM error message that I'm encountering and I
> hope it's appropriate to post to this mailing list about it.
>
> Postgres can be built with LLVM-based JIT support.
> I find that when running on Linux (RHEL8) on IBM Z hardware, using
> LLVM8+, I'm seeing many instances of the following LLVM error message
> appearing in the Postgres server log:
>
>     ORC error: No callback manager available for s390x-ibm-linux
>
> (strangely, it otherwise doesn't seem to be causing any problems)
>
> Looking through the LLVM source code, it appears that for the
> "systemz" architecture, there is  currently no local CCMgr
> implementation available, and this then prevents indirection support
> and COD layer support. BUT - apparently it can still function (maybe
> less efficiently?) without these?
> Is somebody able to clarify exactly what that error message means and
> what the limitations are? Is it unwise to use LLVM on architectures on
> which this error is reported?
> I looked through the LLVM V10 source code and there still seems to be
> this issue for systemz.
>
> Secondly, LLVM sends this error message to standard error. As a caller
> of the LLVM library, it would be nice to be able to redirect such
> messages from the library - is it possible?
>
> Hope you can advise,
> Greg
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200708/d863db56/attachment-0001.html>

Hi Lang,

Thanks for your reply.
I work on a product that is largely based on Postgres but we do
contribute the odd fix/improvement/feature to the open-source Postgres
where we can.

It seems to me that the newer ORC APIs only exist in LLVM10.
Were the older ORC APIs deprecated in LLVM10 only?
Unfortunately our product has a dependency on LLVM8 (e.g. on RHEL8 on
the systemz platform, the default LLVM version installed by the system
package manager "yum" is version 8).
For LLVM8, is there any other API that can be used to avoid the LLVM
"No callback manager available" error seen on systemz?

Regards,
Greg

On Thu, Jul 9, 2020 at 2:29 PM Lang Hames <lhames at gmail.com>
wrote:
>
> Hi Greg,
>
> Thanks for your patience.
>
>> Looking through the LLVM source code, it appears that for the
>> "systemz" architecture, there is  currently no local CCMgr
>> implementation available, and this then prevents indirection support
>> and COD layer support. BUT - apparently it can still function (maybe
>> less efficiently?) without these?
>> Is somebody able to clarify exactly what that error message means and
>> what the limitations are? Is it unwise to use LLVM on architectures on
>> which this error is reported?
>> I looked through the LLVM V10 source code and there still seems to be
>> this issue for systemz.
>
>
> You are correct: The error is generated when a call is made to the
createLocalCompileCallbackManager function. Platforms that don't support a
callback manager (yet) can not use lazy compilation, but can still JIT whole
modules.
>
> I suspect that postgres is using the older ORC C bindings. These always try
to enable lazy compilation support and direct any errors to stderr. Postgress
probably isn't using the lazy compilation support anyway, which is why
everything works just fine despite the error.
>
> These older APIs are deprecated, so the best solution would be for postgres
to move to the newer C APIs (see
https://github.com/llvm/llvm-project/blob/master/llvm/include/llvm-c/Orc.h).
>
> Are you working on Postgres yourself, or just building it?
>
> Regards,
> Lang.
>
> On Thu, Jun 25, 2020 at 4:49 AM Greg Nancarrow via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>
>> Hi LLVM Developers,
>>
>> I have a query about an LLVM error message that I'm encountering
and I
>> hope it's appropriate to post to this mailing list about it.
>>
>> Postgres can be built with LLVM-based JIT support.
>> I find that when running on Linux (RHEL8) on IBM Z hardware, using
>> LLVM8+, I'm seeing many instances of the following LLVM error
message
>> appearing in the Postgres server log:
>>
>>     ORC error: No callback manager available for s390x-ibm-linux
>>
>> (strangely, it otherwise doesn't seem to be causing any problems)
>>
>> Looking through the LLVM source code, it appears that for the
>> "systemz" architecture, there is  currently no local CCMgr
>> implementation available, and this then prevents indirection support
>> and COD layer support. BUT - apparently it can still function (maybe
>> less efficiently?) without these?
>> Is somebody able to clarify exactly what that error message means and
>> what the limitations are? Is it unwise to use LLVM on architectures on
>> which this error is reported?
>> I looked through the LLVM V10 source code and there still seems to be
>> this issue for systemz.
>>
>> Secondly, LLVM sends this error message to standard error. As a caller
>> of the LLVM library, it would be nice to be able to redirect such
>> messages from the library - is it possible?
>>
>> Hope you can advise,
>> Greg
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev