thr3ads.net - llvm dev - [llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch? [Aug 2020]

If this information is useful, please help other people find it:
Share via:

Geoff Levner via llvm-dev

2020-Aug-10 15:24 UTC

[llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?

Thanks, Joseph and Johannes.

I have not merged in anything, I am using the code from the repository
as is. What is this -debug-only option, and to whom would I pass it? I
am running our own JIT application, which uses clang to compile
modules on the fly via clang::CompilerInstance::ExecuteAction().

Working on the assumption that there is a mismatch in the declared
type of an OpenMP runtime function, can you give me a hint as to where
I might find which function it is, and where it would be declared?
What I am seeing in gdb is that a two-parameter function is indeed
being passed two arguments, but the first argument is not of the
expected type. The call originates in
CGOpenMPRuntime::emitForStaticFinish(), with DKind
=llvm::omp::Directive::OMPD_for, at this line:

  CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
                          CGM.getModule(), OMPRTL___kmpc_for_static_fini),
                      Args);

But I don't know where to go from there... Help?

Geoff

On Mon, Aug 10, 2020 at 4:43 PM Johannes Doerfert
<johannesdoerfert at gmail.com> wrote:>
> This looks strange but it makes sense that you see the assertion
> "because of" that patch.
>
> We use this code path to generate OpenMP code for a while nowthat is why
> I am not convinced there is not another (undiagnosed) problem which is
> the actual cause.
>
> Could you run it with `-debug-only='openmp-opt,openmp-ir-builder'`
> please? Is it possible you merged in an old module or some user code
> with OpenMP declarations?
>
>
> Thanks,
>
>    Johannes
>
>
>
> On 8/10/20 8:46 AM, Huber, Joseph wrote:
> > Hi,
> >
> > That patch was from an ongoing effort to consolidate OpenMP generation
in clang. If memory serves the implementation there is still a little
incomplete. It's supposed to use types from OMPConstants rather than ones it
defined itself and the methods used to create the functions shouldn't need
to be static. However attempting this caused a lot of errors so there might be
an underlying problem related to the issues you're seeing. Another one of
the errors I had to deal with was dealing with conflicting function
declarations, for example declaring omp_get_thread_num extern in the file.
Judging from the assertions you got previously this might be the problem. What
would happen is if the name of an OpenMP runtime function was specified in the
file the types wouldn't match up when it tried to find the declaration and
cause an error. The solution to this was to always attempt to typecast the
function type  found in the module to the one defined in OMPKinds.def. There
should be some debug clauses there that will tell you the types it's finding
if that's the issue.
> >
> > Joseph Huber
> > ________________________________
> > From: Geoff Levner <glevner at gmail.com>
> > Sent: Monday, August 10, 2020 4:34 AM
> > To: Lang Hames <lhames at gmail.com>
> > Cc: llvm-dev <llvm-dev at lists.llvm.org>; Huber, Joseph
<huberjn at ornl.gov>
> > Subject: [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?
> >
> > Thanks, Lang.
> >
> > I think I have identified the commit that breaks our application,
> > although I haven't quite finished verifying (it takes a while to
> > compile and test a debug version of LLVM). It's commit
> > 1a4fb2edcb908d6c9141036d29b46a347b1b6f18, dated June 1:
> >
> > -----
> > Summary: This changes Clang's generation of OpenMP runtime
functions
> > to use the types and functions defined in OpenMPKinds and
> > OpenMPConstants. New OpenMP runtime function information should now be
> > added to OMPKinds.def. This patch also changed the definitions of
> > __kmpc_push_num_teams and __kmpc_copyprivate to match those found in
> > the runtime.
> > -----
> >
> > The review of the changes is here: https://reviews.llvm.org/D80222
> >
> > I am CCing the author.
> >
> > Geoff
> >
> >
> > On Mon, Aug 10, 2020 at 6:54 AM Lang Hames <lhames at gmail.com>
wrote:
> >> Hi Geoff,
> >>
> >> Nothing in that backtrace leaps out at me. Based on the stack
trace and description my first guess would be a clang misconfiguration rather
than a JIT bug.
> >>
> >> How is that clang invocation being made? Is it from inside a
callback from ORC, or is it before you add your module to the JIT?
> >>
> >> -- Lang.
> >>
> >> On Mon, Aug 3, 2020 at 5:41 AM Geoff Levner <glevner at
gmail.com> wrote:
> >>> Here, by the way, is the stack trace leading to the failed
assertion.
> >>> Note #9:
clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish().
> >>>
> >>> #0  0x00007fb70fa44870 in abort () from /usr/lib64/libc.so.6
> >>> #1  0x00007fb70fa3c0e6 in __assert_fail_base () from
/usr/lib64/libc.so.6
> >>> #2  0x00007fb70fa3c192 in __assert_fail () from
/usr/lib64/libc.so.6
> >>> #3  0x00007fb715ca1a68 in llvm::CallInst::init
(this=0x51f6e10,
> >>> FTy=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
> >>>      NameStr=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/lib/IR/Instructions.cpp:453
> >>> #4  0x00007fb7116a9e4e in llvm::CallInst::CallInst
(this=0x51f6e10,
> >>> Ty=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
> >>>      NameStr=..., InsertBefore=0x0) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1681
> >>> #5  0x00007fb7116a9cf2 in llvm::CallInst::Create
(Ty=0x51fd280,
> >>> Func=0x51f6d28, Args=..., Bundles=..., NameStr=...,
> >>>      InsertBefore=0x0) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1489
> >>> #6  0x00007fb71171f1e0 in llvm::IRBuilderBase::CreateCall
> >>> (this=0x7fffd4fdaac8, FTy=0x51fd280, Callee=0x51f6d28,
Args=...,
> >>>      OpBundles=..., Name=..., FPMathTag=0x0) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2338
> >>> #7  0x00007fb71171f2d5 in llvm::IRBuilderBase::CreateCall
> >>> (this=0x7fffd4fdaac8, Callee=..., Args=..., OpBundles=...,
Name=...,
> >>>      FPMathTag=0x0) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2356
> >>> #8  0x00007fb7117124eb in
> >>> clang::CodeGen::CodeGenFunction::EmitRuntimeCall
(this=0x7fffd4fda9d0,
> >>> callee=..., args=...,
> >>>      name=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGCall.cpp:4051
> >>> #9  0x00007fb7118b601f in
> >>> clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish
(this=0x4614460,
> >>> CGF=..., Loc=...,
> >>>      DKind=llvm::omp::OMPD_for) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:2875
> >>> #10 0x00007fb71197c294 in
> >>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction
> >>> &) const (__closure=0x7fffd4fd8340, CGF=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3048
> >>> #11 0x00007fb71199754d in
> >>>
llvm::function_ref<void(clang::CodeGen::CodeGenFunction&)>::callback_fn<clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(const
> >>> clang::OMPLoopDirective&, clang::Expr*, const
CodeGenLoopBoundsTy&,
> >>> const
CodeGenDispatchBoundsTy&)::<lambda(clang::CodeGen::CodeGenFunction&)>
> >>>> (intptr_t, clang::CodeGen::CodeGenFunction &)
> >>> (callable=140736766772032,
> >>>      params#0=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:186
> >>> #12 0x00007fb71199ef91 in llvm::function_ref<void
> >>>
(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction&)
> >>> const (this=0x7fffd4fd7b40, params#0=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:203
> >>> #13 0x00007fb71199d79a in
> >>>
clang::CodeGen::CodeGenFunction::OpenMPCancelExitStack::emitExit(clang::CodeGen::CodeGenFunction&,
> >>> llvm::omp::Directive, llvm::function_ref<void
> >>> (clang::CodeGen::CodeGenFunction&)>)
(this=0x7fffd4fdb678, CGF=...,
> >>>      Kind=llvm::omp::OMPD_for, CodeGen=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.h:1361
> >>> #14 0x00007fb71197d41e in
> >>>
clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(clang::OMPLoopDirective
> >>> const&, clang::Expr*,
> >>> llvm::function_ref<std::pair<clang::CodeGen::LValue,
> >>> clang::CodeGen::LValue>
(clang::CodeGen::CodeGenFunction&,
> >>> clang::OMPExecutableDirective const&)> const&,
> >>> llvm::function_ref<std::pair<llvm::Value*,
llvm::Value*>
> >>> (clang::CodeGen::CodeGenFunction&,
clang::OMPExecutableDirective
> >>> const&, clang::CodeGen::Address,
clang::CodeGen::Address)> const&)
> >>> (this=0x7fffd4fda9d0, S=...,
> >>>      EUB=0x4cb7250, CodeGenLoopBounds=...,
CGDispatchBounds=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3050
> >>> #15 0x00007fb71197f746 in emitWorksharingDirective (CGF=...,
S=...,
> >>> HasCancel=false)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3335
> >>> #16 0x00007fb71197f7b0 in
> >>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
> >>>
clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
> >>> &, clang::CodeGen::PrePostActionTy &) const
(__closure=0x7fffd4fd8940,
> >>>      CGF=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3346
> >>> #17 0x00007fb71199771f in
> >>>
clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPForDirective(const
> >>>
clang::OMPForDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
> >>> clang::CodeGen::PrePostActionTy&)> >(intptr_t,
> >>> clang::CodeGen::CodeGenFunction &,
clang::CodeGen::PrePostActionTy &)
> >>> (CodeGen=140736766773568, CGF=..., Action=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
> >>> #18 0x00007fb7118a8787 in
clang::CodeGen::RegionCodeGenTy::operator()
> >>> (this=0x4d7e948, CGF=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
> >>> #19 0x00007fb7118ab598 in (anonymous
> >>> namespace)::CGOpenMPRegionInfo::EmitBody (this=0x4d7e8d0,
CGF=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
> >>> #20 0x00007fb7118cce36 in
> >>> clang::CodeGen::CGOpenMPRuntime::emitInlinedDirective
(this=0x4614460,
> >>> CGF=...,
> >>>      InnerKind=llvm::omp::OMPD_for, CodeGen=...,
HasCancel=false)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:6222
> >>> #21 0x00007fb71197f8bd in
> >>> clang::CodeGen::CodeGenFunction::EmitOMPForDirective
> >>> (this=0x7fffd4fda9d0, S=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3353
> >>> #22 0x00007fb711952aeb in
clang::CodeGen::CodeGenFunction::EmitStmt
> >>> (this=0x7fffd4fda9d0, S=0x46c3b18, Attrs=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:202
> >>> #23 0x00007fb7119537d3 in
> >>> clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
> >>> (this=0x7fffd4fda9d0, S=...,
> >>>      GetLast=false, AggSlot=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
> >>> #24 0x00007fb7119534b6 in
> >>> clang::CodeGen::CodeGenFunction::EmitCompoundStmt
> >>> (this=0x7fffd4fda9d0, S=..., GetLast=false,
> >>>      AggSlot=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:404
> >>> #25 0x00007fb7119532b8 in
> >>> clang::CodeGen::CodeGenFunction::EmitSimpleStmt
(this=0x7fffd4fda9d0,
> >>> S=0x46c4750)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:377
> >>> #26 0x00007fb711952547 in
clang::CodeGen::CodeGenFunction::EmitStmt
> >>> (this=0x7fffd4fda9d0, S=0x46c4750, Attrs=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:53
> >>> #27 0x00007fb7119757e9 in
> >>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
> >>>
clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
> >>> &, clang::CodeGen::PrePostActionTy &) const
(__closure=0x7fffd4fdc360,
> >>>      CGF=..., Action=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1733
> >>> #28 0x00007fb711996ecf in
> >>>
clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective(const
> >>>
clang::OMPParallelDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
> >>> clang::CodeGen::PrePostActionTy&)> >(intptr_t,
> >>> clang::CodeGen::CodeGenFunction &,
clang::CodeGen::PrePostActionTy &)
> >>> (CodeGen=140736766788448, CGF=..., Action=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
> >>> #29 0x00007fb7118a8787 in
clang::CodeGen::RegionCodeGenTy::operator()
> >>> (this=0x7fffd4fda988, CGF=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
> >>> #30 0x00007fb7118ab598 in (anonymous
> >>> namespace)::CGOpenMPRegionInfo::EmitBody (this=0x7fffd4fda910,
> >>> CGF=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
> >>> #31 0x00007fb71196e1e1 in
> >>>
clang::CodeGen::CodeGenFunction::GenerateOpenMPCapturedStmtFunction
> >>> (this=0x7fffd4fda9d0, S=...,
> >>>      Loc=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:615
> >>> #32 0x00007fb7118ace11 in emitParallelOrTeamsOutlinedFunction
> >>> (CGM=..., D=..., CS=0x4a693a8, ThreadIDVar=0x49fe398,
> >>>      InnermostKind=llvm::omp::OMPD_parallel,
OutlinedHelperName=..., CodeGen=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1289
> >>> #33 0x00007fb7118acec7 in
> >>> clang::CodeGen::CGOpenMPRuntime::emitParallelOutlinedFunction
> >>> (this=0x4614460, D=...,
> >>>      ThreadIDVar=0x49fe398,
InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1297
> >>> #34 0x00007fb7119745c9 in emitCommonOMPParallelDirective
(CGF=...,
> >>> S=..., InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...,
> >>>      CodeGenBoundParameters=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1530
> >>> #35 0x00007fb711975bd9 in
> >>> clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective
> >>> (this=0x7fffd4fdc7e0, S=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1740
> >>> #36 0x00007fb711952aa5 in
clang::CodeGen::CodeGenFunction::EmitStmt
> >>> (this=0x7fffd4fdc7e0, S=0x4a695a8, Attrs=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:196
> >>> #37 0x00007fb7119537d3 in
> >>> clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
> >>> (this=0x7fffd4fdc7e0, S=...,
> >>>      GetLast=false, AggSlot=...) at
> >>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
> >>> #38 0x00007fb7119d69ec in
> >>> clang::CodeGen::CodeGenFunction::EmitFunctionBody
> >>> (this=0x7fffd4fdc7e0, Body=0x4a695e0)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1147
> >>> #39 0x00007fb7119d75bc in
> >>> clang::CodeGen::CodeGenFunction::GenerateCode
(this=0x7fffd4fdc7e0,
> >>> GD=..., Fn=0x4699718, FnInfo=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1312
> >>> #40 0x00007fb7119facdf in
> >>> clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition
> >>> (this=0x4b103a0, GD=..., GV=0x4699718)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:4528
> >>> #41 0x00007fb7119f3f73 in
> >>> clang::CodeGen::CodeGenModule::EmitGlobalDefinition
(this=0x4b103a0,
> >>> GD=..., GV=0x4699718)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2890
> >>> #42 0x00007fb7119f0ec2 in
clang::CodeGen::CodeGenModule::EmitDeferred
> >>> (this=0x4b103a0)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2208
> >>> #43 0x00007fb7119e8448 in
clang::CodeGen::CodeGenModule::Release
> >>> (this=0x4b103a0)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:399
> >>> ---Type <return> to continue, or q <return> to
quit---
> >>> #44 0x00007fb71166b523 in (anonymous
> >>> namespace)::CodeGeneratorImpl::HandleTranslationUnit
(this=0x4ceb9c0,
> >>> Ctx=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/ModuleBuilder.cpp:267
> >>> #45 0x00007fb71163ff48 in
> >>> clang::BackendConsumer::HandleTranslationUnit (this=0x49fc8d0,
C=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:283
> >>> #46 0x00007fb711dbf5f5 in
> >>> clang::MultiplexConsumer::HandleTranslationUnit
(this=0x2cc6290,
> >>> Ctx=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/MultiplexConsumer.cpp:292
> >>> #47 0x00007fb71203fb15 in clang::ParseAST (S=...,
PrintStats=false,
> >>> SkipFunctionBodies=false)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/Parse/ParseAST.cpp:171
> >>> #48 0x00007fb711d7fee7 in
clang::ASTFrontendAction::ExecuteAction
> >>> (this=0x41bcf20)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1058
> >>> #49 0x00007fb71163b730 in clang::CodeGenAction::ExecuteAction
(this=0x41bcf20)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:1184
> >>> #50 0x00007fb711d80137 in
clang::WrapperFrontendAction::ExecuteAction
> >>> (this=0x4cb8cd0)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1089
> >>> #51 0x00007fb711d7f83f in clang::FrontendAction::Execute
(this=0x4cb8cd0)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:950
> >>> #52 0x00007fb711d0d96f in
clang::CompilerInstance::ExecuteAction
> >>> (this=0x7fffd4fe1ed0, Act=...)
> >>>      at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:984
> >>>
> >>> On Mon, Aug 3, 2020 at 2:28 PM Geoff Levner <glevner at
gmail.com> wrote:
> >>>> Greetings, Lang and other JITters,
> >>>>
> >>>> Last week I moved our Orc v2-based app from top-of-tree to
the new
> >>>> 11.x branch, and it no longer handles C++ code containing
OpenMP
> >>>> directives correctly.
> >>>>
> >>>> More specifically, if I JIT compile a function containing
OpenMP
> >>>> parallel/for pragmas, using a release version of LLVM, the
code
> >>>> compiles and seems to work, but if I try to write the
module's bitcode
> >>>> to file, I get the following error message:
> >>>>
> >>>>      Invalid record (Producer: 'LLVM11.0.0'
Reader: 'LLVM 11.0.0').
> >>>>
> >>>> If I use a debug version of LLVM, compilation fails with a
broken assertion:
> >>>>
> >>>>      .../llvm/lib/IR/Instructions.cpp:455: void
> >>>> llvm::CallInst::init(llvm::FunctionType*, llvm::Va
> >>>> lue*, llvm::ArrayRef<llvm::Value*>,
> >>>>
llvm::ArrayRef<llvm::OperandBundleDefT<llvm::Value*> >, const
> >>>> llvm::Twine&): Assertion `(i >
> >>>> = FTy->getNumParams() || FTy->getParamType(i) ==
Args[i]->getType())
> >>>> && "Calling a function with a bad
signature!"' failed.
> >>>>
> >>>> The same code compiles correctly using clang from the
command line. It
> >>>> also works fine using Orc v2 in the master branch of LLVM
from the end
> >>>> of May.
> >>>>
> >>>> The problems are way too deep in the LLVM code for me to
make any
> >>>> sense of them... Does anybody else have a clue what might
have
> >>>> happened to Orc and/or OpenMP in the past couple months to
trigger
> >>>> this?
> >>>>
> >>>> Geoff
>

Johannes Doerfert via llvm-dev

2020-Aug-10 16:01 UTC

head link

[llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?

I am looking into a "fix", though I'm not 100% sure what this
problem
actually is.


Can you dump the two first arguments, I mean the llvm::Values and the 
llvm::Types? I would

"expect" they are equal structs with different names. If that is the 
case, my "fix" should

make this work. If not, I'm a little lost. We can always go the route of 
Clang, simply type

cast the function pointer, that is not great either though.



On 8/10/20 10:24 AM, Geoff Levner wrote:> Thanks, Joseph and Johannes.
>
> I have not merged in anything, I am using the code from the repository
> as is. What is this -debug-only option, and to whom would I pass it? I
> am running our own JIT application, which uses clang to compile
> modules on the fly via clang::CompilerInstance::ExecuteAction().
>
> Working on the assumption that there is a mismatch in the declared
> type of an OpenMP runtime function, can you give me a hint as to where
> I might find which function it is, and where it would be declared?
> What I am seeing in gdb is that a two-parameter function is indeed
> being passed two arguments, but the first argument is not of the
> expected type. The call originates in
> CGOpenMPRuntime::emitForStaticFinish(), with DKind =>
llvm::omp::Directive::OMPD_for, at this line:
>
>    CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
>                            CGM.getModule(), OMPRTL___kmpc_for_static_fini),
>                        Args);
>
> But I don't know where to go from there... Help?
>
> Geoff
>
> On Mon, Aug 10, 2020 at 4:43 PM Johannes Doerfert
> <johannesdoerfert at gmail.com> wrote:
>> This looks strange but it makes sense that you see the assertion
>> "because of" that patch.
>>
>> We use this code path to generate OpenMP code for a while nowthat is
why
>> I am not convinced there is not another (undiagnosed) problem which is
>> the actual cause.
>>
>> Could you run it with
`-debug-only='openmp-opt,openmp-ir-builder'`
>> please? Is it possible you merged in an old module or some user code
>> with OpenMP declarations?
>>
>>
>> Thanks,
>>
>>     Johannes
>>
>>
>>
>> On 8/10/20 8:46 AM, Huber, Joseph wrote:
>>> Hi,
>>>
>>> That patch was from an ongoing effort to consolidate OpenMP
generation in clang. If memory serves the implementation there is still a little
incomplete. It's supposed to use types from OMPConstants rather than ones it
defined itself and the methods used to create the functions shouldn't need
to be static. However attempting this caused a lot of errors so there might be
an underlying problem related to the issues you're seeing. Another one of
the errors I had to deal with was dealing with conflicting function
declarations, for example declaring omp_get_thread_num extern in the file.
Judging from the assertions you got previously this might be the problem. What
would happen is if the name of an OpenMP runtime function was specified in the
file the types wouldn't match up when it tried to find the declaration and
cause an error. The solution to this was to always attempt to typecast the
function type  found in the module to the one defined in OMPKinds.def. There
should be some debug clauses there that will tell you the types it's finding
if that's the issue.
>>>
>>> Joseph Huber
>>> ________________________________
>>> From: Geoff Levner <glevner at gmail.com>
>>> Sent: Monday, August 10, 2020 4:34 AM
>>> To: Lang Hames <lhames at gmail.com>
>>> Cc: llvm-dev <llvm-dev at lists.llvm.org>; Huber, Joseph
<huberjn at ornl.gov>
>>> Subject: [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?
>>>
>>> Thanks, Lang.
>>>
>>> I think I have identified the commit that breaks our application,
>>> although I haven't quite finished verifying (it takes a while
to
>>> compile and test a debug version of LLVM). It's commit
>>> 1a4fb2edcb908d6c9141036d29b46a347b1b6f18, dated June 1:
>>>
>>> -----
>>> Summary: This changes Clang's generation of OpenMP runtime
functions
>>> to use the types and functions defined in OpenMPKinds and
>>> OpenMPConstants. New OpenMP runtime function information should now
be
>>> added to OMPKinds.def. This patch also changed the definitions of
>>> __kmpc_push_num_teams and __kmpc_copyprivate to match those found
in
>>> the runtime.
>>> -----
>>>
>>> The review of the changes is here: https://reviews.llvm.org/D80222
>>>
>>> I am CCing the author.
>>>
>>> Geoff
>>>
>>>
>>> On Mon, Aug 10, 2020 at 6:54 AM Lang Hames <lhames at
gmail.com> wrote:
>>>> Hi Geoff,
>>>>
>>>> Nothing in that backtrace leaps out at me. Based on the stack
trace and description my first guess would be a clang misconfiguration rather
than a JIT bug.
>>>>
>>>> How is that clang invocation being made? Is it from inside a
callback from ORC, or is it before you add your module to the JIT?
>>>>
>>>> -- Lang.
>>>>
>>>> On Mon, Aug 3, 2020 at 5:41 AM Geoff Levner <glevner at
gmail.com> wrote:
>>>>> Here, by the way, is the stack trace leading to the failed
assertion.
>>>>> Note #9:
clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish().
>>>>>
>>>>> #0  0x00007fb70fa44870 in abort () from
/usr/lib64/libc.so.6
>>>>> #1  0x00007fb70fa3c0e6 in __assert_fail_base () from
/usr/lib64/libc.so.6
>>>>> #2  0x00007fb70fa3c192 in __assert_fail () from
/usr/lib64/libc.so.6
>>>>> #3  0x00007fb715ca1a68 in llvm::CallInst::init
(this=0x51f6e10,
>>>>> FTy=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
>>>>>       NameStr=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/lib/IR/Instructions.cpp:453
>>>>> #4  0x00007fb7116a9e4e in llvm::CallInst::CallInst
(this=0x51f6e10,
>>>>> Ty=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
>>>>>       NameStr=..., InsertBefore=0x0) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1681
>>>>> #5  0x00007fb7116a9cf2 in llvm::CallInst::Create
(Ty=0x51fd280,
>>>>> Func=0x51f6d28, Args=..., Bundles=..., NameStr=...,
>>>>>       InsertBefore=0x0) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1489
>>>>> #6  0x00007fb71171f1e0 in llvm::IRBuilderBase::CreateCall
>>>>> (this=0x7fffd4fdaac8, FTy=0x51fd280, Callee=0x51f6d28,
Args=...,
>>>>>       OpBundles=..., Name=..., FPMathTag=0x0) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2338
>>>>> #7  0x00007fb71171f2d5 in llvm::IRBuilderBase::CreateCall
>>>>> (this=0x7fffd4fdaac8, Callee=..., Args=..., OpBundles=...,
Name=...,
>>>>>       FPMathTag=0x0) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2356
>>>>> #8  0x00007fb7117124eb in
>>>>> clang::CodeGen::CodeGenFunction::EmitRuntimeCall
(this=0x7fffd4fda9d0,
>>>>> callee=..., args=...,
>>>>>       name=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGCall.cpp:4051
>>>>> #9  0x00007fb7118b601f in
>>>>> clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish
(this=0x4614460,
>>>>> CGF=..., Loc=...,
>>>>>       DKind=llvm::omp::OMPD_for) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:2875
>>>>> #10 0x00007fb71197c294 in
>>>>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction
>>>>> &) const (__closure=0x7fffd4fd8340, CGF=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3048
>>>>> #11 0x00007fb71199754d in
>>>>>
llvm::function_ref<void(clang::CodeGen::CodeGenFunction&)>::callback_fn<clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(const
>>>>> clang::OMPLoopDirective&, clang::Expr*, const
CodeGenLoopBoundsTy&,
>>>>> const
CodeGenDispatchBoundsTy&)::<lambda(clang::CodeGen::CodeGenFunction&)>
>>>>>> (intptr_t, clang::CodeGen::CodeGenFunction &)
>>>>> (callable=140736766772032,
>>>>>       params#0=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:186
>>>>> #12 0x00007fb71199ef91 in llvm::function_ref<void
>>>>>
(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction&)
>>>>> const (this=0x7fffd4fd7b40, params#0=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:203
>>>>> #13 0x00007fb71199d79a in
>>>>>
clang::CodeGen::CodeGenFunction::OpenMPCancelExitStack::emitExit(clang::CodeGen::CodeGenFunction&,
>>>>> llvm::omp::Directive, llvm::function_ref<void
>>>>> (clang::CodeGen::CodeGenFunction&)>)
(this=0x7fffd4fdb678, CGF=...,
>>>>>       Kind=llvm::omp::OMPD_for, CodeGen=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.h:1361
>>>>> #14 0x00007fb71197d41e in
>>>>>
clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(clang::OMPLoopDirective
>>>>> const&, clang::Expr*,
>>>>> llvm::function_ref<std::pair<clang::CodeGen::LValue,
>>>>> clang::CodeGen::LValue>
(clang::CodeGen::CodeGenFunction&,
>>>>> clang::OMPExecutableDirective const&)> const&,
>>>>> llvm::function_ref<std::pair<llvm::Value*,
llvm::Value*>
>>>>> (clang::CodeGen::CodeGenFunction&,
clang::OMPExecutableDirective
>>>>> const&, clang::CodeGen::Address,
clang::CodeGen::Address)> const&)
>>>>> (this=0x7fffd4fda9d0, S=...,
>>>>>       EUB=0x4cb7250, CodeGenLoopBounds=...,
CGDispatchBounds=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3050
>>>>> #15 0x00007fb71197f746 in emitWorksharingDirective
(CGF=..., S=...,
>>>>> HasCancel=false)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3335
>>>>> #16 0x00007fb71197f7b0 in
>>>>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
>>>>>
clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
>>>>> &, clang::CodeGen::PrePostActionTy &) const
(__closure=0x7fffd4fd8940,
>>>>>       CGF=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3346
>>>>> #17 0x00007fb71199771f in
>>>>>
clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPForDirective(const
>>>>>
clang::OMPForDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
>>>>> clang::CodeGen::PrePostActionTy&)> >(intptr_t,
>>>>> clang::CodeGen::CodeGenFunction &,
clang::CodeGen::PrePostActionTy &)
>>>>> (CodeGen=140736766773568, CGF=..., Action=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
>>>>> #18 0x00007fb7118a8787 in
clang::CodeGen::RegionCodeGenTy::operator()
>>>>> (this=0x4d7e948, CGF=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
>>>>> #19 0x00007fb7118ab598 in (anonymous
>>>>> namespace)::CGOpenMPRegionInfo::EmitBody (this=0x4d7e8d0,
CGF=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
>>>>> #20 0x00007fb7118cce36 in
>>>>> clang::CodeGen::CGOpenMPRuntime::emitInlinedDirective
(this=0x4614460,
>>>>> CGF=...,
>>>>>       InnerKind=llvm::omp::OMPD_for, CodeGen=...,
HasCancel=false)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:6222
>>>>> #21 0x00007fb71197f8bd in
>>>>> clang::CodeGen::CodeGenFunction::EmitOMPForDirective
>>>>> (this=0x7fffd4fda9d0, S=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3353
>>>>> #22 0x00007fb711952aeb in
clang::CodeGen::CodeGenFunction::EmitStmt
>>>>> (this=0x7fffd4fda9d0, S=0x46c3b18, Attrs=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:202
>>>>> #23 0x00007fb7119537d3 in
>>>>>
clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
>>>>> (this=0x7fffd4fda9d0, S=...,
>>>>>       GetLast=false, AggSlot=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
>>>>> #24 0x00007fb7119534b6 in
>>>>> clang::CodeGen::CodeGenFunction::EmitCompoundStmt
>>>>> (this=0x7fffd4fda9d0, S=..., GetLast=false,
>>>>>       AggSlot=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:404
>>>>> #25 0x00007fb7119532b8 in
>>>>> clang::CodeGen::CodeGenFunction::EmitSimpleStmt
(this=0x7fffd4fda9d0,
>>>>> S=0x46c4750)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:377
>>>>> #26 0x00007fb711952547 in
clang::CodeGen::CodeGenFunction::EmitStmt
>>>>> (this=0x7fffd4fda9d0, S=0x46c4750, Attrs=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:53
>>>>> #27 0x00007fb7119757e9 in
>>>>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
>>>>>
clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
>>>>> &, clang::CodeGen::PrePostActionTy &) const
(__closure=0x7fffd4fdc360,
>>>>>       CGF=..., Action=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1733
>>>>> #28 0x00007fb711996ecf in
>>>>>
clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective(const
>>>>>
clang::OMPParallelDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
>>>>> clang::CodeGen::PrePostActionTy&)> >(intptr_t,
>>>>> clang::CodeGen::CodeGenFunction &,
clang::CodeGen::PrePostActionTy &)
>>>>> (CodeGen=140736766788448, CGF=..., Action=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
>>>>> #29 0x00007fb7118a8787 in
clang::CodeGen::RegionCodeGenTy::operator()
>>>>> (this=0x7fffd4fda988, CGF=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
>>>>> #30 0x00007fb7118ab598 in (anonymous
>>>>> namespace)::CGOpenMPRegionInfo::EmitBody
(this=0x7fffd4fda910,
>>>>> CGF=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
>>>>> #31 0x00007fb71196e1e1 in
>>>>>
clang::CodeGen::CodeGenFunction::GenerateOpenMPCapturedStmtFunction
>>>>> (this=0x7fffd4fda9d0, S=...,
>>>>>       Loc=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:615
>>>>> #32 0x00007fb7118ace11 in
emitParallelOrTeamsOutlinedFunction
>>>>> (CGM=..., D=..., CS=0x4a693a8, ThreadIDVar=0x49fe398,
>>>>>       InnermostKind=llvm::omp::OMPD_parallel,
OutlinedHelperName=..., CodeGen=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1289
>>>>> #33 0x00007fb7118acec7 in
>>>>>
clang::CodeGen::CGOpenMPRuntime::emitParallelOutlinedFunction
>>>>> (this=0x4614460, D=...,
>>>>>       ThreadIDVar=0x49fe398,
InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1297
>>>>> #34 0x00007fb7119745c9 in emitCommonOMPParallelDirective
(CGF=...,
>>>>> S=..., InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...,
>>>>>       CodeGenBoundParameters=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1530
>>>>> #35 0x00007fb711975bd9 in
>>>>> clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective
>>>>> (this=0x7fffd4fdc7e0, S=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1740
>>>>> #36 0x00007fb711952aa5 in
clang::CodeGen::CodeGenFunction::EmitStmt
>>>>> (this=0x7fffd4fdc7e0, S=0x4a695a8, Attrs=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:196
>>>>> #37 0x00007fb7119537d3 in
>>>>>
clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
>>>>> (this=0x7fffd4fdc7e0, S=...,
>>>>>       GetLast=false, AggSlot=...) at
>>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
>>>>> #38 0x00007fb7119d69ec in
>>>>> clang::CodeGen::CodeGenFunction::EmitFunctionBody
>>>>> (this=0x7fffd4fdc7e0, Body=0x4a695e0)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1147
>>>>> #39 0x00007fb7119d75bc in
>>>>> clang::CodeGen::CodeGenFunction::GenerateCode
(this=0x7fffd4fdc7e0,
>>>>> GD=..., Fn=0x4699718, FnInfo=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1312
>>>>> #40 0x00007fb7119facdf in
>>>>> clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition
>>>>> (this=0x4b103a0, GD=..., GV=0x4699718)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:4528
>>>>> #41 0x00007fb7119f3f73 in
>>>>> clang::CodeGen::CodeGenModule::EmitGlobalDefinition
(this=0x4b103a0,
>>>>> GD=..., GV=0x4699718)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2890
>>>>> #42 0x00007fb7119f0ec2 in
clang::CodeGen::CodeGenModule::EmitDeferred
>>>>> (this=0x4b103a0)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2208
>>>>> #43 0x00007fb7119e8448 in
clang::CodeGen::CodeGenModule::Release
>>>>> (this=0x4b103a0)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:399
>>>>> ---Type <return> to continue, or q <return> to
quit---
>>>>> #44 0x00007fb71166b523 in (anonymous
>>>>> namespace)::CodeGeneratorImpl::HandleTranslationUnit
(this=0x4ceb9c0,
>>>>> Ctx=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/ModuleBuilder.cpp:267
>>>>> #45 0x00007fb71163ff48 in
>>>>> clang::BackendConsumer::HandleTranslationUnit
(this=0x49fc8d0, C=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:283
>>>>> #46 0x00007fb711dbf5f5 in
>>>>> clang::MultiplexConsumer::HandleTranslationUnit
(this=0x2cc6290,
>>>>> Ctx=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/MultiplexConsumer.cpp:292
>>>>> #47 0x00007fb71203fb15 in clang::ParseAST (S=...,
PrintStats=false,
>>>>> SkipFunctionBodies=false)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Parse/ParseAST.cpp:171
>>>>> #48 0x00007fb711d7fee7 in
clang::ASTFrontendAction::ExecuteAction
>>>>> (this=0x41bcf20)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1058
>>>>> #49 0x00007fb71163b730 in
clang::CodeGenAction::ExecuteAction (this=0x41bcf20)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:1184
>>>>> #50 0x00007fb711d80137 in
clang::WrapperFrontendAction::ExecuteAction
>>>>> (this=0x4cb8cd0)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1089
>>>>> #51 0x00007fb711d7f83f in clang::FrontendAction::Execute
(this=0x4cb8cd0)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:950
>>>>> #52 0x00007fb711d0d96f in
clang::CompilerInstance::ExecuteAction
>>>>> (this=0x7fffd4fe1ed0, Act=...)
>>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:984
>>>>>
>>>>> On Mon, Aug 3, 2020 at 2:28 PM Geoff Levner <glevner at
gmail.com> wrote:
>>>>>> Greetings, Lang and other JITters,
>>>>>>
>>>>>> Last week I moved our Orc v2-based app from top-of-tree
to the new
>>>>>> 11.x branch, and it no longer handles C++ code
containing OpenMP
>>>>>> directives correctly.
>>>>>>
>>>>>> More specifically, if I JIT compile a function
containing OpenMP
>>>>>> parallel/for pragmas, using a release version of LLVM,
the code
>>>>>> compiles and seems to work, but if I try to write the
module's bitcode
>>>>>> to file, I get the following error message:
>>>>>>
>>>>>>       Invalid record (Producer: 'LLVM11.0.0'
Reader: 'LLVM 11.0.0').
>>>>>>
>>>>>> If I use a debug version of LLVM, compilation fails
with a broken assertion:
>>>>>>
>>>>>>       .../llvm/lib/IR/Instructions.cpp:455: void
>>>>>> llvm::CallInst::init(llvm::FunctionType*, llvm::Va
>>>>>> lue*, llvm::ArrayRef<llvm::Value*>,
>>>>>>
llvm::ArrayRef<llvm::OperandBundleDefT<llvm::Value*> >, const
>>>>>> llvm::Twine&): Assertion `(i >
>>>>>> = FTy->getNumParams() || FTy->getParamType(i) ==
Args[i]->getType())
>>>>>> && "Calling a function with a bad
signature!"' failed.
>>>>>>
>>>>>> The same code compiles correctly using clang from the
command line. It
>>>>>> also works fine using Orc v2 in the master branch of
LLVM from the end
>>>>>> of May.
>>>>>>
>>>>>> The problems are way too deep in the LLVM code for me
to make any
>>>>>> sense of them... Does anybody else have a clue what
might have
>>>>>> happened to Orc and/or OpenMP in the past couple months
to trigger
>>>>>> this?
>>>>>>
>>>>>> Geoff

Geoff Levner via llvm-dev

2020-Aug-10 17:05 UTC

head link

[llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?

Yep, it happens three times, then crashes afterwards, since I removed
the assert...

arg 0: expected %struct.ident_t*
got %struct.ident_t.21*
value @0 = private unnamed_addr global %struct.ident_t.21 { i32 0, i32
514, i32 0, i32 0, i8* getelementptr inbounds ([23 x i8], [23 x i8]*
@.str, i32 0, i32 0) }, align 8

arg 0: expected %struct.ident_t*
got %struct.ident_t.21*
value @1 = private unnamed_addr global %struct.ident_t.21 { i32 0, i32
66, i32 0, i32 0, i8* getelementptr inbounds ([23 x i8], [23 x i8]*
@.str, i32 0, i32 0) }, align 8

arg 0: expected %struct.ident_t*
got %struct.ident_t.21*
value @2 = private unnamed_addr global %struct.ident_t.21 { i32 0, i32
2, i32 0, i32 0, i8* getelementptr inbounds ([23 x i8], [23 x i8]*
@.str, i32 0, i32 0) }, align 8

Call parameter type does not match function signature!
%struct.ident_t.21* @2
 %struct.ident_t*  call void (%struct.ident_t*, i32, void (i32*, i32*,
...)*, ...) @__kmpc_fork_call(%struct.ident_t.21* @2, i32 4, void
(i32*, i32*, ...)* bitcast (void (i32*, i32*, i64*,
%"class.std::vector.22"*, %"class.std::vector.22"*, float*)*
@.omp_outlined. to void (i32*, i32*, ...)*), i64* %_n,
%"class.std::vector.22"* %5, %"class.std::vector.22"* %6,
float*
%b.addr)
in function
_ZN39kp_8d75c6d9_5169_4aa9_9900_7e97ce4e9b897executeERKSt6vectorIfSaIfEEfRS2_PN5kirpi11GraphTimingE
LLVM ERROR: Broken function found, compilation aborted!
Abort (core dumped)

On Mon, Aug 10, 2020 at 6:03 PM Johannes Doerfert
<johannesdoerfert at gmail.com> wrote:>
> I am looking into a "fix", though I'm not 100% sure what this
problem
> actually is.
>
>
> Can you dump the two first arguments, I mean the llvm::Values and the
> llvm::Types? I would
>
> "expect" they are equal structs with different names. If that is
the
> case, my "fix" should
>
> make this work. If not, I'm a little lost. We can always go the route
of
> Clang, simply type
>
> cast the function pointer, that is not great either though.
>
>
>
> On 8/10/20 10:24 AM, Geoff Levner wrote:
> > Thanks, Joseph and Johannes.
> >
> > I have not merged in anything, I am using the code from the repository
> > as is. What is this -debug-only option, and to whom would I pass it? I
> > am running our own JIT application, which uses clang to compile
> > modules on the fly via clang::CompilerInstance::ExecuteAction().
> >
> > Working on the assumption that there is a mismatch in the declared
> > type of an OpenMP runtime function, can you give me a hint as to where
> > I might find which function it is, and where it would be declared?
> > What I am seeing in gdb is that a two-parameter function is indeed
> > being passed two arguments, but the first argument is not of the
> > expected type. The call originates in
> > CGOpenMPRuntime::emitForStaticFinish(), with DKind => >
llvm::omp::Directive::OMPD_for, at this line:
> >
> >    CGF.EmitRuntimeCall(OMPBuilder.getOrCreateRuntimeFunction(
> >                            CGM.getModule(),
OMPRTL___kmpc_for_static_fini),
> >                        Args);
> >
> > But I don't know where to go from there... Help?
> >
> > Geoff
> >
> > On Mon, Aug 10, 2020 at 4:43 PM Johannes Doerfert
> > <johannesdoerfert at gmail.com> wrote:
> >> This looks strange but it makes sense that you see the assertion
> >> "because of" that patch.
> >>
> >> We use this code path to generate OpenMP code for a while nowthat
is why
> >> I am not convinced there is not another (undiagnosed) problem
which is
> >> the actual cause.
> >>
> >> Could you run it with
`-debug-only='openmp-opt,openmp-ir-builder'`
> >> please? Is it possible you merged in an old module or some user
code
> >> with OpenMP declarations?
> >>
> >>
> >> Thanks,
> >>
> >>     Johannes
> >>
> >>
> >>
> >> On 8/10/20 8:46 AM, Huber, Joseph wrote:
> >>> Hi,
> >>>
> >>> That patch was from an ongoing effort to consolidate OpenMP
generation in clang. If memory serves the implementation there is still a little
incomplete. It's supposed to use types from OMPConstants rather than ones it
defined itself and the methods used to create the functions shouldn't need
to be static. However attempting this caused a lot of errors so there might be
an underlying problem related to the issues you're seeing. Another one of
the errors I had to deal with was dealing with conflicting function
declarations, for example declaring omp_get_thread_num extern in the file.
Judging from the assertions you got previously this might be the problem. What
would happen is if the name of an OpenMP runtime function was specified in the
file the types wouldn't match up when it tried to find the declaration and
cause an error. The solution to this was to always attempt to typecast the
function type  found in the module to the one defined in OMPKinds.def. There
should be some debug clauses there that will tell you the types it's finding
if that's the issue.
> >>>
> >>> Joseph Huber
> >>> ________________________________
> >>> From: Geoff Levner <glevner at gmail.com>
> >>> Sent: Monday, August 10, 2020 4:34 AM
> >>> To: Lang Hames <lhames at gmail.com>
> >>> Cc: llvm-dev <llvm-dev at lists.llvm.org>; Huber, Joseph
<huberjn at ornl.gov>
> >>> Subject: [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x
branch?
> >>>
> >>> Thanks, Lang.
> >>>
> >>> I think I have identified the commit that breaks our
application,
> >>> although I haven't quite finished verifying (it takes a
while to
> >>> compile and test a debug version of LLVM). It's commit
> >>> 1a4fb2edcb908d6c9141036d29b46a347b1b6f18, dated June 1:
> >>>
> >>> -----
> >>> Summary: This changes Clang's generation of OpenMP runtime
functions
> >>> to use the types and functions defined in OpenMPKinds and
> >>> OpenMPConstants. New OpenMP runtime function information
should now be
> >>> added to OMPKinds.def. This patch also changed the definitions
of
> >>> __kmpc_push_num_teams and __kmpc_copyprivate to match those
found in
> >>> the runtime.
> >>> -----
> >>>
> >>> The review of the changes is here:
https://reviews.llvm.org/D80222
> >>>
> >>> I am CCing the author.
> >>>
> >>> Geoff
> >>>
> >>>
> >>> On Mon, Aug 10, 2020 at 6:54 AM Lang Hames <lhames at
gmail.com> wrote:
> >>>> Hi Geoff,
> >>>>
> >>>> Nothing in that backtrace leaps out at me. Based on the
stack trace and description my first guess would be a clang misconfiguration
rather than a JIT bug.
> >>>>
> >>>> How is that clang invocation being made? Is it from inside
a callback from ORC, or is it before you add your module to the JIT?
> >>>>
> >>>> -- Lang.
> >>>>
> >>>> On Mon, Aug 3, 2020 at 5:41 AM Geoff Levner <glevner at
gmail.com> wrote:
> >>>>> Here, by the way, is the stack trace leading to the
failed assertion.
> >>>>> Note #9:
clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish().
> >>>>>
> >>>>> #0  0x00007fb70fa44870 in abort () from
/usr/lib64/libc.so.6
> >>>>> #1  0x00007fb70fa3c0e6 in __assert_fail_base () from
/usr/lib64/libc.so.6
> >>>>> #2  0x00007fb70fa3c192 in __assert_fail () from
/usr/lib64/libc.so.6
> >>>>> #3  0x00007fb715ca1a68 in llvm::CallInst::init
(this=0x51f6e10,
> >>>>> FTy=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
> >>>>>       NameStr=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/lib/IR/Instructions.cpp:453
> >>>>> #4  0x00007fb7116a9e4e in llvm::CallInst::CallInst
(this=0x51f6e10,
> >>>>> Ty=0x51fd280, Func=0x51f6d28, Args=..., Bundles=...,
> >>>>>       NameStr=..., InsertBefore=0x0) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1681
> >>>>> #5  0x00007fb7116a9cf2 in llvm::CallInst::Create
(Ty=0x51fd280,
> >>>>> Func=0x51f6d28, Args=..., Bundles=..., NameStr=...,
> >>>>>       InsertBefore=0x0) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/Instructions.h:1489
> >>>>> #6  0x00007fb71171f1e0 in
llvm::IRBuilderBase::CreateCall
> >>>>> (this=0x7fffd4fdaac8, FTy=0x51fd280, Callee=0x51f6d28,
Args=...,
> >>>>>       OpBundles=..., Name=..., FPMathTag=0x0) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2338
> >>>>> #7  0x00007fb71171f2d5 in
llvm::IRBuilderBase::CreateCall
> >>>>> (this=0x7fffd4fdaac8, Callee=..., Args=...,
OpBundles=..., Name=...,
> >>>>>       FPMathTag=0x0) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/IR/IRBuilder.h:2356
> >>>>> #8  0x00007fb7117124eb in
> >>>>> clang::CodeGen::CodeGenFunction::EmitRuntimeCall
(this=0x7fffd4fda9d0,
> >>>>> callee=..., args=...,
> >>>>>       name=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGCall.cpp:4051
> >>>>> #9  0x00007fb7118b601f in
> >>>>> clang::CodeGen::CGOpenMPRuntime::emitForStaticFinish
(this=0x4614460,
> >>>>> CGF=..., Loc=...,
> >>>>>       DKind=llvm::omp::OMPD_for) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:2875
> >>>>> #10 0x00007fb71197c294 in
> >>>>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction
> >>>>> &) const (__closure=0x7fffd4fd8340, CGF=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3048
> >>>>> #11 0x00007fb71199754d in
> >>>>>
llvm::function_ref<void(clang::CodeGen::CodeGenFunction&)>::callback_fn<clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(const
> >>>>> clang::OMPLoopDirective&, clang::Expr*, const
CodeGenLoopBoundsTy&,
> >>>>> const
CodeGenDispatchBoundsTy&)::<lambda(clang::CodeGen::CodeGenFunction&)>
> >>>>>> (intptr_t, clang::CodeGen::CodeGenFunction &)
> >>>>> (callable=140736766772032,
> >>>>>       params#0=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:186
> >>>>> #12 0x00007fb71199ef91 in llvm::function_ref<void
> >>>>>
(clang::CodeGen::CodeGenFunction&)>::operator()(clang::CodeGen::CodeGenFunction&)
> >>>>> const (this=0x7fffd4fd7b40, params#0=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/llvm/include/llvm/ADT/STLExtras.h:203
> >>>>> #13 0x00007fb71199d79a in
> >>>>>
clang::CodeGen::CodeGenFunction::OpenMPCancelExitStack::emitExit(clang::CodeGen::CodeGenFunction&,
> >>>>> llvm::omp::Directive, llvm::function_ref<void
> >>>>> (clang::CodeGen::CodeGenFunction&)>)
(this=0x7fffd4fdb678, CGF=...,
> >>>>>       Kind=llvm::omp::OMPD_for, CodeGen=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.h:1361
> >>>>> #14 0x00007fb71197d41e in
> >>>>>
clang::CodeGen::CodeGenFunction::EmitOMPWorksharingLoop(clang::OMPLoopDirective
> >>>>> const&, clang::Expr*,
> >>>>>
llvm::function_ref<std::pair<clang::CodeGen::LValue,
> >>>>> clang::CodeGen::LValue>
(clang::CodeGen::CodeGenFunction&,
> >>>>> clang::OMPExecutableDirective const&)>
const&,
> >>>>> llvm::function_ref<std::pair<llvm::Value*,
llvm::Value*>
> >>>>> (clang::CodeGen::CodeGenFunction&,
clang::OMPExecutableDirective
> >>>>> const&, clang::CodeGen::Address,
clang::CodeGen::Address)> const&)
> >>>>> (this=0x7fffd4fda9d0, S=...,
> >>>>>       EUB=0x4cb7250, CodeGenLoopBounds=...,
CGDispatchBounds=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3050
> >>>>> #15 0x00007fb71197f746 in emitWorksharingDirective
(CGF=..., S=...,
> >>>>> HasCancel=false)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3335
> >>>>> #16 0x00007fb71197f7b0 in
> >>>>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
> >>>>>
clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
> >>>>> &, clang::CodeGen::PrePostActionTy &) const
(__closure=0x7fffd4fd8940,
> >>>>>       CGF=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3346
> >>>>> #17 0x00007fb71199771f in
> >>>>>
clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPForDirective(const
> >>>>>
clang::OMPForDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
> >>>>> clang::CodeGen::PrePostActionTy&)>
>(intptr_t,
> >>>>> clang::CodeGen::CodeGenFunction &,
clang::CodeGen::PrePostActionTy &)
> >>>>> (CodeGen=140736766773568, CGF=..., Action=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
> >>>>> #18 0x00007fb7118a8787 in
clang::CodeGen::RegionCodeGenTy::operator()
> >>>>> (this=0x4d7e948, CGF=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
> >>>>> #19 0x00007fb7118ab598 in (anonymous
> >>>>> namespace)::CGOpenMPRegionInfo::EmitBody
(this=0x4d7e8d0, CGF=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
> >>>>> #20 0x00007fb7118cce36 in
> >>>>> clang::CodeGen::CGOpenMPRuntime::emitInlinedDirective
(this=0x4614460,
> >>>>> CGF=...,
> >>>>>       InnerKind=llvm::omp::OMPD_for, CodeGen=...,
HasCancel=false)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:6222
> >>>>> #21 0x00007fb71197f8bd in
> >>>>> clang::CodeGen::CodeGenFunction::EmitOMPForDirective
> >>>>> (this=0x7fffd4fda9d0, S=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:3353
> >>>>> #22 0x00007fb711952aeb in
clang::CodeGen::CodeGenFunction::EmitStmt
> >>>>> (this=0x7fffd4fda9d0, S=0x46c3b18, Attrs=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:202
> >>>>> #23 0x00007fb7119537d3 in
> >>>>>
clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
> >>>>> (this=0x7fffd4fda9d0, S=...,
> >>>>>       GetLast=false, AggSlot=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
> >>>>> #24 0x00007fb7119534b6 in
> >>>>> clang::CodeGen::CodeGenFunction::EmitCompoundStmt
> >>>>> (this=0x7fffd4fda9d0, S=..., GetLast=false,
> >>>>>       AggSlot=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:404
> >>>>> #25 0x00007fb7119532b8 in
> >>>>> clang::CodeGen::CodeGenFunction::EmitSimpleStmt
(this=0x7fffd4fda9d0,
> >>>>> S=0x46c4750)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:377
> >>>>> #26 0x00007fb711952547 in
clang::CodeGen::CodeGenFunction::EmitStmt
> >>>>> (this=0x7fffd4fda9d0, S=0x46c4750, Attrs=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:53
> >>>>> #27 0x00007fb7119757e9 in
> >>>>>
clang::CodeGen::CodeGenFunction::<lambda(clang::CodeGen::CodeGenFunction&,
> >>>>>
clang::CodeGen::PrePostActionTy&)>::operator()(clang::CodeGen::CodeGenFunction
> >>>>> &, clang::CodeGen::PrePostActionTy &) const
(__closure=0x7fffd4fdc360,
> >>>>>       CGF=..., Action=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1733
> >>>>> #28 0x00007fb711996ecf in
> >>>>>
clang::CodeGen::RegionCodeGenTy::CallbackFn<clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective(const
> >>>>>
clang::OMPParallelDirective&)::<lambda(clang::CodeGen::CodeGenFunction&,
> >>>>> clang::CodeGen::PrePostActionTy&)>
>(intptr_t,
> >>>>> clang::CodeGen::CodeGenFunction &,
clang::CodeGen::PrePostActionTy &)
> >>>>> (CodeGen=140736766788448, CGF=..., Action=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.h:80
> >>>>> #29 0x00007fb7118a8787 in
clang::CodeGen::RegionCodeGenTy::operator()
> >>>>> (this=0x7fffd4fda988, CGF=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:595
> >>>>> #30 0x00007fb7118ab598 in (anonymous
> >>>>> namespace)::CGOpenMPRegionInfo::EmitBody
(this=0x7fffd4fda910,
> >>>>> CGF=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1041
> >>>>> #31 0x00007fb71196e1e1 in
> >>>>>
clang::CodeGen::CodeGenFunction::GenerateOpenMPCapturedStmtFunction
> >>>>> (this=0x7fffd4fda9d0, S=...,
> >>>>>       Loc=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:615
> >>>>> #32 0x00007fb7118ace11 in
emitParallelOrTeamsOutlinedFunction
> >>>>> (CGM=..., D=..., CS=0x4a693a8, ThreadIDVar=0x49fe398,
> >>>>>       InnermostKind=llvm::omp::OMPD_parallel,
OutlinedHelperName=..., CodeGen=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1289
> >>>>> #33 0x00007fb7118acec7 in
> >>>>>
clang::CodeGen::CGOpenMPRuntime::emitParallelOutlinedFunction
> >>>>> (this=0x4614460, D=...,
> >>>>>       ThreadIDVar=0x49fe398,
InnermostKind=llvm::omp::OMPD_parallel, CodeGen=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGOpenMPRuntime.cpp:1297
> >>>>> #34 0x00007fb7119745c9 in
emitCommonOMPParallelDirective (CGF=...,
> >>>>> S=..., InnermostKind=llvm::omp::OMPD_parallel,
CodeGen=...,
> >>>>>       CodeGenBoundParameters=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1530
> >>>>> #35 0x00007fb711975bd9 in
> >>>>>
clang::CodeGen::CodeGenFunction::EmitOMPParallelDirective
> >>>>> (this=0x7fffd4fdc7e0, S=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmtOpenMP.cpp:1740
> >>>>> #36 0x00007fb711952aa5 in
clang::CodeGen::CodeGenFunction::EmitStmt
> >>>>> (this=0x7fffd4fdc7e0, S=0x4a695a8, Attrs=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:196
> >>>>> #37 0x00007fb7119537d3 in
> >>>>>
clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope
> >>>>> (this=0x7fffd4fdc7e0, S=...,
> >>>>>       GetLast=false, AggSlot=...) at
> >>>>>
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CGStmt.cpp:453
> >>>>> #38 0x00007fb7119d69ec in
> >>>>> clang::CodeGen::CodeGenFunction::EmitFunctionBody
> >>>>> (this=0x7fffd4fdc7e0, Body=0x4a695e0)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1147
> >>>>> #39 0x00007fb7119d75bc in
> >>>>> clang::CodeGen::CodeGenFunction::GenerateCode
(this=0x7fffd4fdc7e0,
> >>>>> GD=..., Fn=0x4699718, FnInfo=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenFunction.cpp:1312
> >>>>> #40 0x00007fb7119facdf in
> >>>>>
clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition
> >>>>> (this=0x4b103a0, GD=..., GV=0x4699718)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:4528
> >>>>> #41 0x00007fb7119f3f73 in
> >>>>> clang::CodeGen::CodeGenModule::EmitGlobalDefinition
(this=0x4b103a0,
> >>>>> GD=..., GV=0x4699718)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2890
> >>>>> #42 0x00007fb7119f0ec2 in
clang::CodeGen::CodeGenModule::EmitDeferred
> >>>>> (this=0x4b103a0)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:2208
> >>>>> #43 0x00007fb7119e8448 in
clang::CodeGen::CodeGenModule::Release
> >>>>> (this=0x4b103a0)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenModule.cpp:399
> >>>>> ---Type <return> to continue, or q
<return> to quit---
> >>>>> #44 0x00007fb71166b523 in (anonymous
> >>>>> namespace)::CodeGeneratorImpl::HandleTranslationUnit
(this=0x4ceb9c0,
> >>>>> Ctx=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/ModuleBuilder.cpp:267
> >>>>> #45 0x00007fb71163ff48 in
> >>>>> clang::BackendConsumer::HandleTranslationUnit
(this=0x49fc8d0, C=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:283
> >>>>> #46 0x00007fb711dbf5f5 in
> >>>>> clang::MultiplexConsumer::HandleTranslationUnit
(this=0x2cc6290,
> >>>>> Ctx=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/MultiplexConsumer.cpp:292
> >>>>> #47 0x00007fb71203fb15 in clang::ParseAST (S=...,
PrintStats=false,
> >>>>> SkipFunctionBodies=false)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Parse/ParseAST.cpp:171
> >>>>> #48 0x00007fb711d7fee7 in
clang::ASTFrontendAction::ExecuteAction
> >>>>> (this=0x41bcf20)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1058
> >>>>> #49 0x00007fb71163b730 in
clang::CodeGenAction::ExecuteAction (this=0x41bcf20)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/CodeGen/CodeGenAction.cpp:1184
> >>>>> #50 0x00007fb711d80137 in
clang::WrapperFrontendAction::ExecuteAction
> >>>>> (this=0x4cb8cd0)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1089
> >>>>> #51 0x00007fb711d7f83f in
clang::FrontendAction::Execute (this=0x4cb8cd0)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/FrontendAction.cpp:950
> >>>>> #52 0x00007fb711d0d96f in
clang::CompilerInstance::ExecuteAction
> >>>>> (this=0x7fffd4fe1ed0, Act=...)
> >>>>>       at
/u/geoff/llvmrepo/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:984
> >>>>>
> >>>>> On Mon, Aug 3, 2020 at 2:28 PM Geoff Levner
<glevner at gmail.com> wrote:
> >>>>>> Greetings, Lang and other JITters,
> >>>>>>
> >>>>>> Last week I moved our Orc v2-based app from
top-of-tree to the new
> >>>>>> 11.x branch, and it no longer handles C++ code
containing OpenMP
> >>>>>> directives correctly.
> >>>>>>
> >>>>>> More specifically, if I JIT compile a function
containing OpenMP
> >>>>>> parallel/for pragmas, using a release version of
LLVM, the code
> >>>>>> compiles and seems to work, but if I try to write
the module's bitcode
> >>>>>> to file, I get the following error message:
> >>>>>>
> >>>>>>       Invalid record (Producer:
'LLVM11.0.0' Reader: 'LLVM 11.0.0').
> >>>>>>
> >>>>>> If I use a debug version of LLVM, compilation
fails with a broken assertion:
> >>>>>>
> >>>>>>       .../llvm/lib/IR/Instructions.cpp:455: void
> >>>>>> llvm::CallInst::init(llvm::FunctionType*, llvm::Va
> >>>>>> lue*, llvm::ArrayRef<llvm::Value*>,
> >>>>>>
llvm::ArrayRef<llvm::OperandBundleDefT<llvm::Value*> >, const
> >>>>>> llvm::Twine&): Assertion `(i >
> >>>>>> = FTy->getNumParams() ||
FTy->getParamType(i) == Args[i]->getType())
> >>>>>> && "Calling a function with a bad
signature!"' failed.
> >>>>>>
> >>>>>> The same code compiles correctly using clang from
the command line. It
> >>>>>> also works fine using Orc v2 in the master branch
of LLVM from the end
> >>>>>> of May.
> >>>>>>
> >>>>>> The problems are way too deep in the LLVM code for
me to make any
> >>>>>> sense of them... Does anybody else have a clue
what might have
> >>>>>> happened to Orc and/or OpenMP in the past couple
months to trigger
> >>>>>> this?
> >>>>>>
> >>>>>> Geoff
>

llvm dev - Aug 2020 - [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?

[llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?

[llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?

[llvm-dev] [EXTERNAL] Re: Orc JIT v2 breaks OpenMP in 11.x branch?