thr3ads.net - llvm dev - [llvm-dev] OrcV1 removal [Oct 2020]

If this information is useful, please help other people find it:
Share via:

Andres Freund via llvm-dev

2020-Oct-01 21:35 UTC

[llvm-dev] OrcV1 removal

Hi,

On 2020-09-30 21:31:33 -0700, Lang Hames wrote:> I've taken a first shot at hooking RTDyldObjectLinkingLayer up to the
> ResourceTracker API in 7436b2ab2428. Could you let me know whether that
> fixes the leak you were seeing?
It did improve the situation significantly, thanks!

There's still a smaller leak, unfortunately. The function comments for
modules say that:

/**
 * Create a ThreadSafeModule wrapper around the given LLVM module. This takes
 * ownership of the M argument which should not be disposed of or referenced
 * after this function returns.
 *
 * Ownership of the ThreadSafeModule is unique: If it is transferred to the JIT
 * (e.g. by LLVMOrcLLJITAddLLVMIRModule), in which case the client is no longer
 * responsible for it. If it is not transferred to the JIT then the client
 * should call LLVMOrcDisposeThreadSafeModule to dispose of it.
 */
LLVMOrcThreadSafeModuleRef
LLVMOrcCreateNewThreadSafeModule(LLVMModuleRef M,
                                 LLVMOrcThreadSafeContextRef TSCtx);

/**
 * Dispose of a ThreadSafeModule. This should only be called if ownership has
 * not been passed to LLJIT (e.g. because some error prevented the client from
 * adding this to the JIT).
 */
void LLVMOrcDisposeThreadSafeModule(LLVMOrcThreadSafeModuleRef TSM);

but when using -fsanitize-leaks I see:

Direct leak of 9192 byte(s) in 383 object(s) allocated from:
    #0 0x7fe249127670 in operator new(unsigned long)
(/lib/x86_64-linux-gnu/liblsan.so.0+0xf670)
    #1 0x7fe1bfc1cbc0 in LLVMOrcCreateNewThreadSafeModule
(/home/andres/build/llvm-project/master/debug/install/lib/libLLVMOrcJIT.so.12git+0x38bbc0)
    #2 0x7fe245f6ea6d in llvm_compile_module
/home/andres/src/postgresql/src/backend/jit/llvm/llvmjit.c:606
    #3 0x7fe245f6e0b0 in llvm_get_function
/home/andres/src/postgresql/src/backend/jit/llvm/llvmjit.c:275
    #4 0x7fe245f809f1 in ExecRunCompiledExpr
/home/andres/src/postgresql/src/backend/jit/llvm/llvmjit_expr.c:2410
    #5 0x5633fbb6c17f in ExecEvalExpr
/home/andres/src/postgresql/src/include/executor/executor.h:294

for modules which are passed to LLVMOrcLLJITAddLLVMIRModule:
		ts_module = LLVMOrcCreateNewThreadSafeModule(context->module,
llvm_ts_context);
		rt = LLVMOrcJITDylibCreateResourceTracker(jd);
		error = LLVMOrcLLJITAddLLVMIRModule(compile_orc, jd, ts_module, rt);

which seems to suggest that somehow there's a reference too many?

Greetings,

Andres Freund

Lang Hames via llvm-dev

2020-Oct-01 22:29 UTC

head link

[llvm-dev] OrcV1 removal

Hi Andres,

There's still a smaller leak, unfortunately.


Thanks very much for catching this!

but when using -fsanitize-leaks I see:> Direct leak of 9192 byte(s) in 383 object(s) allocated from:
<snip/>> which seems to suggest that somehow there's a reference too many?

24bytes / object -- Looks like I managed module ownership correctly but
leaked the ThreadSafeModule container. This should be fixed in 5044196b412f.

-- Lang.

On Thu, Oct 1, 2020 at 2:35 PM Andres Freund <andres at anarazel.de>
wrote:
> Hi,
>
> On 2020-09-30 21:31:33 -0700, Lang Hames wrote:
> > I've taken a first shot at hooking RTDyldObjectLinkingLayer up to
the
> > ResourceTracker API in 7436b2ab2428. Could you let me know whether
that
> > fixes the leak you were seeing?
>
> It did improve the situation significantly, thanks!
>
> There's still a smaller leak, unfortunately. The function comments for
> modules say that:
>
> /**
>  * Create a ThreadSafeModule wrapper around the given LLVM module. This
> takes
>  * ownership of the M argument which should not be disposed of or
> referenced
>  * after this function returns.
>  *
>  * Ownership of the ThreadSafeModule is unique: If it is transferred to
> the JIT
>  * (e.g. by LLVMOrcLLJITAddLLVMIRModule), in which case the client is no
> longer
>  * responsible for it. If it is not transferred to the JIT then the client
>  * should call LLVMOrcDisposeThreadSafeModule to dispose of it.
>  */
> LLVMOrcThreadSafeModuleRef
> LLVMOrcCreateNewThreadSafeModule(LLVMModuleRef M,
>                                  LLVMOrcThreadSafeContextRef TSCtx);
>
> /**
>  * Dispose of a ThreadSafeModule. This should only be called if ownership
> has
>  * not been passed to LLJIT (e.g. because some error prevented the client
> from
>  * adding this to the JIT).
>  */
> void LLVMOrcDisposeThreadSafeModule(LLVMOrcThreadSafeModuleRef TSM);
>
> but when using -fsanitize-leaks I see:
>
> Direct leak of 9192 byte(s) in 383 object(s) allocated from:
>     #0 0x7fe249127670 in operator new(unsigned long)
> (/lib/x86_64-linux-gnu/liblsan.so.0+0xf670)
>     #1 0x7fe1bfc1cbc0 in LLVMOrcCreateNewThreadSafeModule
>
(/home/andres/build/llvm-project/master/debug/install/lib/libLLVMOrcJIT.so.12git+0x38bbc0)
>     #2 0x7fe245f6ea6d in llvm_compile_module
> /home/andres/src/postgresql/src/backend/jit/llvm/llvmjit.c:606
>     #3 0x7fe245f6e0b0 in llvm_get_function
> /home/andres/src/postgresql/src/backend/jit/llvm/llvmjit.c:275
>     #4 0x7fe245f809f1 in ExecRunCompiledExpr
> /home/andres/src/postgresql/src/backend/jit/llvm/llvmjit_expr.c:2410
>     #5 0x5633fbb6c17f in ExecEvalExpr
> /home/andres/src/postgresql/src/include/executor/executor.h:294
>
> for modules which are passed to LLVMOrcLLJITAddLLVMIRModule:
>                 ts_module >
LLVMOrcCreateNewThreadSafeModule(context->module, llvm_ts_context);
>                 rt = LLVMOrcJITDylibCreateResourceTracker(jd);
>                 error = LLVMOrcLLJITAddLLVMIRModule(compile_orc, jd,
> ts_module, rt);
>
> which seems to suggest that somehow there's a reference too many?
>
> Greetings,
>
> Andres Freund
>-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201001/a08b974a/attachment.html>

Andres Freund via llvm-dev

2020-Oct-01 23:42 UTC

head link

[llvm-dev] OrcV1 removal

Hi,

On 2020-10-01 15:29:12 -0700, Lang Hames wrote:> 24bytes / object -- Looks like I managed module ownership correctly but
> leaked the ThreadSafeModule container. This should be fixed in
5044196b412f.
That helped a bit, but not yet fully. Looks like it might be still
reachable memory, so leakcheck isn't that helpful.

Oooh. I think I see. For various reasons the symbol names we generate
are unique over time. But the interned strings aren't cleared ever. Is
that right?

With massif, I see:

--------------------------------------------------------------------------------
  n        time(i)         total(B)   useful-heap(B) extra-heap(B)    stacks(B)
--------------------------------------------------------------------------------
 22 62,022,701,974        2,766,072        2,664,931       101,141            0
...
->16.97% (469,433B) 0xB29E471: llvm::allocate_buffer(unsigned long, unsigned
long) (MemAlloc.cpp:15)
| ->03.54% (97,812B) 0x831FB40: llvm::MallocAllocator::Allocate(unsigned
long, unsigned long) (AllocatorBase.h:85)
| | ->03.54% (97,812B) 0x8330268:
llvm::StringMapEntry<std::atomic<unsigned long> >*
llvm::StringMapEntry<std::atomic<unsigned long>
>::Create<llvm::MallocAllocator, int>(llvm::StringRef,
llvm::MallocAllocator&, int&&) (StringMapEntry.h:101)
| |   ->03.54% (97,812B) 0x8322B26:
std::pair<llvm::StringMapIterator<std::atomic<unsigned long> >,
bool> llvm::StringMap<std::atomic<unsigned long>,
llvm::MallocAllocator>::try_emplace<int>(llvm::StringRef,
int&&) (StringMap.h:322)
| |     ->03.54% (97,812B) 0x831FD62:
llvm::orc::SymbolStringPool::intern(llvm::StringRef) (SymbolStringPool.h:159)
| |       ->03.54% (97,812B) 0x8320AF5:
llvm::orc::ExecutionSession::intern(llvm::StringRef) (Core.h:1225)
| |         ->03.53% (97,542B) 0x83EFBAE:
llvm::orc::MangleAndInterner::operator()(llvm::StringRef) (Mangling.cpp:30)
| |         | ->03.53% (97,542B) 0x83B20B2:
llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| |         |   ->03.53% (97,542B) 0x83B2376: decltype(auto)
llvm::orc::ThreadSafeModule::withModuleDo<llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| |         |     ->03.53% (97,542B) 0x83B24DA:
llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule) (Layer.cpp:41)
| |         |       ->03.53% (97,542B) 0x83B29B3:
llvm::orc::BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(llvm::orc::IRLayer&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule) (Layer.cpp:131)
| |         |         ->03.53% (97,542B) 0x83B357D:
std::_MakeUniq<llvm::orc::BasicIRLayerMaterializationUnit>::__single_object
std::make_unique<llvm::orc::BasicIRLayerMaterializationUnit,
llvm::orc::IRLayer&, llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule>(llvm::orc::IRLayer&, llvm::o>
| |         |           ->03.53% (97,542B) 0x83B1C7D:
llvm::orc::IRLayer::add(llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>,
llvm::orc::ThreadSafeModule) (Layer.cpp:28)
| |         |             ->03.53% (97,542B) 0x83BCBF5:
llvm::orc::LLJIT::addIRModule(llvm::orc::JITDylib&,
llvm::orc::ThreadSafeModule,
llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>) (LLJIT.cpp:990)
| |         |               ->03.52% (97,354B) 0x840C95B:
LLVMOrcLLJITAddLLVMIRModule (OrcV2CBindings.cpp:263)
| |         |               | ->03.52% (97,354B) 0x80639E9:
llvm_compile_module (llvmjit.c:608)
| |         |               |   ->03.52% (97,354B) 0x8063000:
llvm_get_function (llvmjit.c:275)
| |         |               |     ->03.52% (97,354B) 0x80758E1:
ExecRunCompiledExpr (llvmjit_expr.c:2410)


 52 148,049,498,480        3,039,472        2,901,891       137,581            0
...
->20.01% (608,089B) 0xB29E471: llvm::allocate_buffer(unsigned long, unsigned
long) (MemAlloc.cpp:15)
| ->07.78% (236,468B) 0x831FB40: llvm::MallocAllocator::Allocate(unsigned
long, unsigned long) (AllocatorBase.h:85)
| | ->07.78% (236,468B) 0x8330268:
llvm::StringMapEntry<std::atomic<unsigned long> >*
llvm::StringMapEntry<std::atomic<unsigned long>
>::Create<llvm::MallocAllocator, int>(llvm::StringRef,
llvm::MallocAllocator&, int&&) (StringMapEntry.h:101)
| |   ->07.78% (236,468B) 0x8322B26:
std::pair<llvm::StringMapIterator<std::atomic<unsigned long> >,
bool> llvm::StringMap<std::atomic<unsigned long>,
llvm::MallocAllocator>::try_emplace<int>(llvm::StringRef,
int&&) (StringMap.h:322)
| |     ->07.78% (236,468B) 0x831FD62:
llvm::orc::SymbolStringPool::intern(llvm::StringRef) (SymbolStringPool.h:159)
| |       ->07.78% (236,468B) 0x8320AF5:
llvm::orc::ExecutionSession::intern(llvm::StringRef) (Core.h:1225)
| |         ->07.77% (236,198B) 0x83EFBAE:
llvm::orc::MangleAndInterner::operator()(llvm::StringRef) (Mangling.cpp:30)
| |         | ->07.77% (236,198B) 0x83B20B2:
llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| |         |   ->07.77% (236,198B) 0x83B2376: decltype(auto)
llvm::orc::ThreadSafeModule::withModuleDo<llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule)::{lambda(llvm::Module&)
| |         |     ->07.77% (236,198B) 0x83B24DA:
llvm::orc::IRMaterializationUnit::IRMaterializationUnit(llvm::orc::ExecutionSession&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule) (Layer.cpp:41)
| |         |       ->07.77% (236,198B) 0x83B29B3:
llvm::orc::BasicIRLayerMaterializationUnit::BasicIRLayerMaterializationUnit(llvm::orc::IRLayer&,
llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule) (Layer.cpp:131)
| |         |         ->07.77% (236,198B) 0x83B357D:
std::_MakeUniq<llvm::orc::BasicIRLayerMaterializationUnit>::__single_object
std::make_unique<llvm::orc::BasicIRLayerMaterializationUnit,
llvm::orc::IRLayer&, llvm::orc::IRSymbolMapper::ManglingOptions const&,
llvm::orc::ThreadSafeModule>(llvm::orc::IRLayer&, llvm::>
| |         |           ->07.77% (236,198B) 0x83B1C7D:
llvm::orc::IRLayer::add(llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>,
llvm::orc::ThreadSafeModule) (Layer.cpp:28)
| |         |             ->07.77% (236,198B) 0x83BCBF5:
llvm::orc::LLJIT::addIRModule(llvm::orc::JITDylib&,
llvm::orc::ThreadSafeModule,
llvm::IntrusiveRefCntPtr<llvm::orc::ResourceTracker>) (LLJIT.cpp:990)
| |         |               ->07.76% (236,010B) 0x840C95B:
LLVMOrcLLJITAddLLVMIRModule (OrcV2CBindings.cpp:263)
| |         |               | ->07.76% (236,010B) 0x80639E9:
llvm_compile_module (llvmjit.c:608)
| |         |               |   ->07.76% (236,010B) 0x8063000:
llvm_get_function (llvmjit.c:275)
| |         |               |     ->07.76% (236,010B) 0x80758E1:
ExecRunCompiledExpr (llvmjit_expr.c:2410)


i.e. you can see that over time (snapshot 22 vs 52) the percentage of
memory used by string interning increases. And that the total bytes used
for that increases too.

Regards,

Andres

llvm dev - Oct 2020 - OrcV1 removal

[llvm-dev] OrcV1 removal

[llvm-dev] OrcV1 removal

[llvm-dev] OrcV1 removal