llvm dev - Mar 2014 - [LLVMdev] A bug or a feature?

I am having an issue with MCJIT on the ppc64 machine architecture.
The symptom is that for a particular IR function the target machine
won't emit neither an object nor an assembly file and subsequent
calling the pointer to function results in a segfault.

My application generates on the fly several functions with the builder
and executes them with the MCJIT engine. I came across this particular
function when the segfault happened.

To investigate this I went down the static linking path and compiled the
barfing function with llc targeting ppc64 and linked it statically. This
works fine. Thus, the issue seems to be MCJIT related.

To further rule out user error I isolated an exploit for this issue and
reran it on the x86-64 architecture. That works fine too. Thus, it seems
to be 'ppc64' related.

I attach the exploit including it's Makefile. It builds 'main' and 
'main_static'.
The former goes the JIT path, the latter the static link path. The offending
function is 'func_fails.ll'. When 'main' is called with this IR
function
as its
argument it'll reproduce the segfault on the ppc64 architecture. The
main_static links statically to 'func_fails.ll' and calls the function 
in exactly
the same way as 'main' does. The statically linked version works fine.

The program 'func_works.ll' can be used a cross check that MCJIT on
the ppc64 architecture works and that the program 'main' is in general
well functioning. To confirm the 'main' executable can be invoked with
the argument 'func_works.ll'.

The issue seems to have to do with the number of calls to external
functions or whether a particular external function is called repeatedly.
I am saying this because when replacing all external calls with trivial
additions (fadd double ..), then the offending function works fine too.

Any suggestion how to hunt down the issue are very much welcome.

Thanks,
Frank




-------------- next part --------------
A non-text attachment was scrubbed...
Name: bug_mcjit_ppc64.tgz
Type: application/x-compressed-tar
Size: 3560 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140310/17d9c786/attachment.bin>

Frank Winter <fwinter at jlab.org> wrote:
> The issue seems to have to do with the number of calls to external
> functions or whether a particular external function is called repeatedly.
> I am saying this because when replacing all external calls with trivial
> additions (fadd double ..), then the offending function works fine too.
Just from this symptom, this reminds me a problem I've been seeing
a while back: when the MCJIT dynamic loader resolves a function call
to an external function, it needs to patch in code to restore the
TOC pointer after the external call returns.  Current code does that,
but only for the first call to a particular external function; for
subsequent calls, it is omitted.

Can you check whether this patch fixes the problem you're seeing?
(See attached file: diff-llvm-ppc64-restoretoc)

Bye,
Ulrich
-------------- next part --------------
A non-text attachment was scrubbed...
Name: diff-llvm-ppc64-restoretoc
Type: application/octet-stream
Size: 977 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140310/59959e05/attachment.obj>

Hi Ulrich!

thanks for the patch. I can confirm that with the patch applied
even the IR function that caused the segfault now runs fine.

Is there any reason why the patch shouldn't be commited?
Would it hurt on any other target architectures?

Best wishes,
Frank


On 03/10/2014 02:35 PM, Ulrich Weigand wrote:
> Frank Winter <fwinter at jlab.org> wrote:
>
>> The issue seems to have to do with the number of calls to external
>> functions or whether a particular external function is called
repeatedly.
>> I am saying this because when replacing all external calls with trivial
>> additions (fadd double ..), then the offending function works fine too.
> Just from this symptom, this reminds me a problem I've been seeing
> a while back: when the MCJIT dynamic loader resolves a function call
> to an external function, it needs to patch in code to restore the
> TOC pointer after the external call returns.  Current code does that,
> but only for the first call to a particular external function; for
> subsequent calls, it is omitted.
>
> Can you check whether this patch fixes the problem you're seeing?
> (See attached file: diff-llvm-ppc64-restoretoc)
>
> Bye,
> Ulrich

Hi,

I've run Clang Static Analyzer checker alpha.cplusplus.NewDeleteLeaks 
over LLVM codebase to detect false-positives and at the same time 
eliminate memory leaks. The majority of leaks were detected in 
lib/Target/* and lib/MC/*. In all cases the similar trick was detected 
as a leak (example from 
lib/Target/Sparc/MCTargetDesc/SparcMCTargetDesc.cpp) :

static MCStreamer *createMCStreamer(const Target &T, StringRef TT,
                                     MCContext &Context, MCAsmBackend
&MAB,
                                     raw_ostream &OS, MCCodeEmitter 
*Emitter,
                                     const MCSubtargetInfo &STI, bool 
RelaxAll,
                                     bool NoExecStack) {
    MCStreamer *S     createELFStreamer(Context, MAB, OS, Emitter, RelaxAll,
NoExecStack);
    new SparcTargetELFStreamer(*S);

            1    Memory is allocated    ?

    return S;

            2   ?   Potential memory leak
}

Have not got why is this SparcTargetELFStreamer created dynamically and 
not assigned.
Can anybody please explain is this a trick or a trouble?

The similar pattern was detected in
lib/Target/ARM/MCTargetDesc/ARMELFStreamer.cpp
lib/Target/ARM/MCTargetDesc/ARMMCTargetDesc.cpp
lib/Target/PowerPC/MCTargetDesc/PPCMCTargetDesc.cpp
lib/Target/Mips/MCTargetDesc/MipsMCTargetDesc.cpp
lib/Target/XCore/MCTargetDesc/XCoreMCTargetDesc.cpp
lib/Target/Mips/MCTargetDesc/MipsMCTargetDesc.cpp
lib/Target/Sparc/MCTargetDesc/SparcMCTargetDesc.cpp
lib/MC/MCELFStreamer.cpp
lib/MC/WinCOFFStreamer.cpp
lib/MC/MCMachOStreamer.cpp

-- 
Anton

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140311/61c395d6/attachment.html>