llvm dev - Oct 2021 - ORC JIT Weekly #42 -- JITLinkMemoryManager refactor lands, libunwind improvements, and edge visitor patterns

Hi All,

The JITLInkMemoryManager refactor finally landed in [1] (though it required
a revert and a few follow-up patches to finally stick). The commit message
has the details, but the short version is that:

1. JITLinkMemoryManagers now have direct access to the LinkGraph, giving
them much greater control over JIT'd memory layout.

2. The BasicLayout type lets you ignore the LinkGraph and get back to a
segment-based view of the world if you don't care about the details (all
existing allocators were updated to use this utility in the refactor).

3. JITLinkMemoryManager::Allocation has been split into
JTILinkMemoryManager::InFlightAlloc and
JITLinkMemoryManager::FinalizedAlloc. This allows in-flight allocation data
structures to be written naturally, while FinalizedAlloc has been trimmed
down to a single uint64_t-wrapper to make tracking of JIT'd memory as
efficient as possible.

4. JITLinkMemoryManager now allows you to attach "allocation actions"
to a
graph. These are wrapper function calls to be be made in the executor at
finalize and deallocate time. Allocation actions allow for registration and
deregistration of special sections (e.g. eh-frames) without making extra
IPC round-trips. This should make cross-process JITing more efficient, and
the ORC runtime easier to read.

Support for allocation actions has not been added to the ORC runtime yet,
but it's on the to-do list and I expect to get to it within the next few
weeks.

Other notable contributions this week:

Peter Housel posted a review for new dynamic eh-frame registration APIs in
libunwind [2]: __unw_add_dynamic_eh_frame_section, and
__unw_remove_dynamic_eh_frame_section. When these functions are detected
the JIT will be able to use them to register EH frames in a uniform way on
both Linux and Darwin, simplifying use of libunwind on both platforms.

Stephen Fan committed a handy edge-visitor utility for JITLink [3]. This
utility makes one pass over all existing edges in a LinkGraph, applying a
variadic list of visitors to each edge. Together with his new TableManager
visitors, this results in a really succinct expression of GOT, PLT and TLV
lowering for ELF in JITLink:

Error buildTables_ELF_x86_64(LinkGraph &G) {
   GOTTableManager_ELF_x86_64 GOT;
   PLTTableManager_ELF_x86_64 PLT(GOT);
   TLSInfoTableManager_ELF_x86_64 TLSInfo;
   visitExistingEdges(G, GOT, PLT, TLSInfo);
   return Error::success();
}

I hope to update the MachO backends to use this utility soon. :)

Up next on my todo list: merging the TargetProcessControlTypes.h header
into OrcRTBridge. The aim will be to make OrcRTBridge the definitive header
connecting LLVM and generic ORC Runtime symbol names and types.

-- Lang.

[1]
https://github.com/llvm/llvm-project/commit/e50aea58d59c8cfae807a7fee21c4227472c0678
[2] https://reviews.llvm.org/D111863
[3]
https://github.com/llvm/llvm-project/commit/849b36bf6f5273c2539a18b4a9c79ea1e136eb0a
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20211017/aeb89c44/attachment.html>