llvm dev - Jan 2019 - How to correctly use PerfJITEventListener?

Hello,

I'm struggling to get PerfJITEventListener to work. It was already fun
to find out that I need to enable LLVM_USE_PERF in cmake during
compilation, and that the jitdump files are stored in ~/.debug/jit/ by
default. But it's still not working.

The steps I do:

1. Get event listener with JITEventListener::createPerfJITEventListener
2. In NotifyLoadedFunction passed to orc::RTDyldObjectLinkingLayer ctor
   pass object and info to event listener's NotifyObjectEmitted

Then, I profile the application with:
perf record -k 1 ./app

This generates a jit-*.dump file in ~/.debug/jit/llvm-IR-jit-*/.

Inject the events into the perf data
perf inject -j -i perf.data -o perf.data.jitted

This creates jitted-*.so files in ~/.debug/jit/llvm-IR-jit-*/ for each
JIT compiled function.

Finally, I open the profile report:
perf report -i perf.data.jitted

But here is the problem. The JIT compiled functions are not
"resolved".
I cannot "zoom in" to the code and annotate the instructions with
profile information.

Am I missing something here? Is there a small example somewhere which
uses PerfJITEventListener with ORC?

Best regards,
Frank

Hello,
   I encountered the same problem recently. Finally I found it's a bug 
of linux kernel or linux-perf. LLVM uses mmap() and mprotect() to 
generate code. However, mprotect() will merge adjacent VMAs (virtual 
memory area) and report a event of merged VMA, causing pref think prior 
allocated memory has gone. I have reported the bug to Linux Perf Users 
mailing list.
   As a quick & super dirty workaround, I modified 
SectionMemoryManager::allocateSection (in 
llvm/lib/ExecutionEngine/SectionMemoryManager.cpp), let it alloc two 
pieces of memory for each memory request, and use only one of them:

--- a/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp
+++ b/llvm/lib/ExecutionEngine/SectionMemoryManager.cpp
@@ -101,6 +101,9 @@ uint8_t *SectionMemoryManager::allocateSection(
    // FIXME: Initialize the Near member for each memory group to avoid
    // interleaving.
    std::error_code ec;
+  if (1) MMapper.allocateMappedMemory(
+      Purpose, RequiredSize, &MemGroup.Near,
+      sys::Memory::MF_READ | sys::Memory::MF_WRITE, ec); // XXX: SUPER 
DIRTY
    sys::MemoryBlock MB = MMapper.allocateMappedMemory(
        Purpose, RequiredSize, &MemGroup.Near,
        sys::Memory::MF_READ | sys::Memory::MF_WRITE, ec);

   Of course this can't be merged into LLVM, but I think this code may 
help some people who encountered the same problem.

Zhang Boyang

On 2019/1/15 21:09, llvm-dev at lists.llvm.org (Frank Tetzel via 
llvm-dev) wrote:
> Hello,
> 
> I'm struggling to get PerfJITEventListener to work. It was already fun
> to find out that I need to enable LLVM_USE_PERF in cmake during
> compilation, and that the jitdump files are stored in ~/.debug/jit/ by
> default. But it's still not working.
> 
> The steps I do:
> 
> 1. Get event listener with JITEventListener::createPerfJITEventListener
> 2. In NotifyLoadedFunction passed to orc::RTDyldObjectLinkingLayer ctor
>     pass object and info to event listener's NotifyObjectEmitted
> 
> Then, I profile the application with:
> perf record -k 1 ./app
> 
> This generates a jit-*.dump file in ~/.debug/jit/llvm-IR-jit-*/.
> 
> Inject the events into the perf data
> perf inject -j -i perf.data -o perf.data.jitted
> 
> This creates jitted-*.so files in ~/.debug/jit/llvm-IR-jit-*/ for each
> JIT compiled function.
> 
> Finally, I open the profile report:
> perf report -i perf.data.jitted
> 
> But here is the problem. The JIT compiled functions are not
"resolved".
> I cannot "zoom in" to the code and annotate the instructions with
> profile information.
> 
> Am I missing something here? Is there a small example somewhere which
> uses PerfJITEventListener with ORC?
> 
> Best regards,
> Frank
>