llvm dev - Oct 2016 - MCJit and remove module memory leak?

I'm on llvm 3.8.1 and was wondering if there's a memory leak in the
removeModule impl of mcjit.
In the tutorial http://llvm.org/releases/3.8.1/docs/tutorial/LangImpl4.html
a module is removed from the Jit by invoking removeModule.

According to the tutorial:

"Its API is very simple:: addModule adds an LLVM IR module to the JIT,
making its functions available for execution; removeModule removes a
module, freeing any memory associated with the code in that module;"

But when I look into the implementation of remove module, it only erases
the ptr from AddedModules
//return AddedModules.erase(M) || LoadedModules.erase(M) ||
FinalizedModules.erase(M);

When you add a module, the unique ptr is stripped, and the raw ptr is
stored.
//AddedModules.insert(M.release());

This should imply that the person who is invoking removeModule also should
invoke a delete on the ptr?

Am I missing something here?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161028/2e4e8d7c/attachment.html>

Hi Koffie,

Kaleidoscope is no longer using MCJIT - it has been moved over to the
ORC-based KaleidoscopeJIT class (see
llvm/examples/Kaleidoscope/include/KaleidoscopeJIT.h). The removeModule
method there does not leak memory.

I've added documentation in r286026 to describe MCJIT removeModule's
crazy
ownership contract.

This will be fixed properly when we kill off ExecutionEngine. :)

- Lang.


On Fri, Oct 28, 2016 at 4:10 AM, koffie drinker <gekkekoe at gmail.com>
wrote:
> I'm on llvm 3.8.1 and was wondering if there's a memory leak in the
> removeModule impl of mcjit.
> In the tutorial http://llvm.org/releases/3.8.1/docs/tutorial/
> LangImpl4.html a module is removed from the Jit by invoking removeModule.
>
> According to the tutorial:
>
> "Its API is very simple:: addModule adds an LLVM IR module to the JIT,
> making its functions available for execution; removeModule removes a
> module, freeing any memory associated with the code in that module;"
>
> But when I look into the implementation of remove module, it only erases
> the ptr from AddedModules
> //return AddedModules.erase(M) || LoadedModules.erase(M) ||
> FinalizedModules.erase(M);
>
> When you add a module, the unique ptr is stripped, and the raw ptr is
> stored.
> //AddedModules.insert(M.release());
>
> This should imply that the person who is invoking removeModule also should
> invoke a delete on the ptr?
>
> Am I missing something here?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161104/558ea0a0/attachment.html>

I Lol'ed at your commit msg :)
We will start migrating to ORC for next release, but for now, this release
invoke delete after remove right?

On Sat, Nov 5, 2016 at 12:20 AM, Lang Hames <lhames at gmail.com> wrote:
> Hi Koffie,
>
> Kaleidoscope is no longer using MCJIT - it has been moved over to the
> ORC-based KaleidoscopeJIT class (see
llvm/examples/Kaleidoscope/include/KaleidoscopeJIT.h).
> The removeModule method there does not leak memory.
>
> I've added documentation in r286026 to describe MCJIT
removeModule's crazy
> ownership contract.
>
> This will be fixed properly when we kill off ExecutionEngine. :)
>
> - Lang.
>
>
> On Fri, Oct 28, 2016 at 4:10 AM, koffie drinker <gekkekoe at
gmail.com>
> wrote:
>
>> I'm on llvm 3.8.1 and was wondering if there's a memory leak in
the
>> removeModule impl of mcjit.
>> In the tutorial http://llvm.org/releases/3.8.1/docs/tutorial/LangIm
>> pl4.html a module is removed from the Jit by invoking removeModule.
>>
>> According to the tutorial:
>>
>> "Its API is very simple:: addModule adds an LLVM IR module to the
JIT,
>> making its functions available for execution; removeModule removes a
>> module, freeing any memory associated with the code in that
module;"
>>
>> But when I look into the implementation of remove module, it only
erases
>> the ptr from AddedModules
>> //return AddedModules.erase(M) || LoadedModules.erase(M) ||
>> FinalizedModules.erase(M);
>>
>> When you add a module, the unique ptr is stripped, and the raw ptr is
>> stored.
>> //AddedModules.insert(M.release());
>>
>> This should imply that the person who is invoking removeModule also
>> should invoke a delete on the ptr?
>>
>> Am I missing something here?
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161106/3628d434/attachment.html>

There's a orc mcjit drop in replacement in the source tree.
Am I correct to assume that Orc is used (and emulating mcjit behaviour)
when replacing
LLVMLinkInOrcMCJITReplacement();
//LLVMLinkInMCJIT();
and linking with libOrcJit ?
Does this replacement handle memory better than original mcjit ?


On Mon, Nov 7, 2016 at 8:39 PM, Kevin P. Neal via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> On Fri, Nov 04, 2016 at 04:20:32PM -0700, Lang Hames via llvm-dev wrote:
> >    Hi Koffie,
> >    Kaleidoscope is no longer using MCJIT - it has been moved over to
the
> >    ORC-based KaleidoscopeJIT class (see
> >    llvm/examples/Kaleidoscope/include/KaleidoscopeJIT.h). The
> removeModule
> >    method there does not leak memory.
> >    I've added documentation in r286026 to describe MCJIT
removeModule's
> >    crazy ownership contract.
> >    This will be fixed properly when we kill off ExecutionEngine. :)
>
> Does this mean MCJIT is dead/deprecated and projects using it should start
> migrating away now? If so, what's the time frame?
>
> --
> Kevin P. Neal                                http://www.pobox.com/~kpn/
>
> "It sounded pretty good, but it's hard to tell how it will work
out
> in practice." -- Dennis Ritchie, ~1977, "Summary of a DEC 32-bit
machine"
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161111/888740e4/attachment.html>

Hi Kevin, Koffie,

We will start migrating to ORC for next release, but for now, this
release
> invoke delete after remove right?

MCJIT's removeModule method does not delete the module. You'll need to
do
that manually. OrcMCJITReplacement is a bug-for-bug compatible
implementation of MCJIT using ORC components, so it does not free the
memory either.

Does this mean MCJIT is dead/deprecated and projects using it should
start
> migrating away now? If so, what's the time frame?

The short answer is yes: I expect MCJIT to be deprecated soon and
eventually killed off. I'll be sending an email with details and a
discussion of the timeline to the dev-list in the next few days. That will
contain suggestions on how to transition to the new APIs (which I expect to
be relatively painless for most people). I can CC you on it if that helps?

Cheers,
Lang.


On Fri, Nov 11, 2016 at 12:00 AM, koffie drinker via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> There's a orc mcjit drop in replacement in the source tree.
> Am I correct to assume that Orc is used (and emulating mcjit behaviour)
> when replacing
> LLVMLinkInOrcMCJITReplacement();
> //LLVMLinkInMCJIT();
> and linking with libOrcJit ?
> Does this replacement handle memory better than original mcjit ?
>
>
> On Mon, Nov 7, 2016 at 8:39 PM, Kevin P. Neal via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> On Fri, Nov 04, 2016 at 04:20:32PM -0700, Lang Hames via llvm-dev
wrote:
>> >    Hi Koffie,
>> >    Kaleidoscope is no longer using MCJIT - it has been moved over
to the
>> >    ORC-based KaleidoscopeJIT class (see
>> >    llvm/examples/Kaleidoscope/include/KaleidoscopeJIT.h). The
>> removeModule
>> >    method there does not leak memory.
>> >    I've added documentation in r286026 to describe MCJIT
removeModule's
>> >    crazy ownership contract.
>> >    This will be fixed properly when we kill off ExecutionEngine.
:)
>>
>> Does this mean MCJIT is dead/deprecated and projects using it should
start
>> migrating away now? If so, what's the time frame?
>>
>> --
>> Kevin P. Neal                                http://www.pobox.com/~kpn/
>>
>> "It sounded pretty good, but it's hard to tell how it will
work out
>> in practice." -- Dennis Ritchie, ~1977, "Summary of a DEC
32-bit machine"
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20161116/5571a8ed/attachment.html>