llvm dev - Jan 2013 - [LLVMdev] Dynamic Profiling - Instrumentation basic query

Hi,

I am new to LLVM, and would like to write a dynamic profiler, say which prints
out the load address of all the load instructions encountered in a
program.
>From what I could pick up, edge-profiler.cpp increments a counter
dynamically which is somehow dumped onto llvmprof.out by profile.pl
Could anyone explain me how this works? Can I instrument the code to dump out
the load addresses or other such information to a file?

Thanks!

Hi Silky,

Firstly: Do you really need to do this?  You can't just get a memory 
access trace from a simulator?  Or dcache access/miss rates from 
hardware counters?

Assuming this really is required:

On 12/01/13 23:28, Silky Arora wrote:
> I am new to LLVM, and would like to write a dynamic profiler, say which
prints out the load address of all the load instructions encountered in a
program.
>>From what I could pick up, edge-profiler.cpp increments a counter
dynamically which is somehow dumped onto llvmprof.out by profile.pl
profile.pl is just a wrapper.  'EdgeProfiling.cpp' inserts calls to 
instrumentation functions on control flow edges.  libprofile_rt.so 
provides these functions (code is in runtime/libprofile).  It is the 
functions in libprofile_rt.so that dump the counters to llvmprof.out.

Note: libprofile_rt.so be named differently on some platforms.

> Could anyone explain me how this works? Can I instrument the code to dump
out the load addresses or other such information to a file?
Yes, you can do this, though the current profiling code does not profile 
load addresses at all.  The quickest way to get this working is probably to:

A) add a new pass to insert the load profiling instrumentation, 
EdgeProfiling.cpp should provide a good start point to copy from.  You 
just need to modify the IR to insert a call to some function, say 
'llvm_profile_load_address', and in the IR pass this function the load 
address as an argument.

B) Add an implementation of 'llvm_profile_load_address' to 
runtime/libprofile (don't forget to add the symbol to 
libprofile.exports).  Perhaps it just does 'fprintf(file, "%p\n",
addr);'.

C) If your implementation of 'llvm_profile_load_address' requires 
initialisation (such as an fopen) add a call to an 
'llvm_profile_load_start' in the IR for 'main' and 
'llvm_profile_load_end' can be provided to atexit().  EdgeProfiling.cpp 
does this, so you can look at that code to see how it works.

This will slow down your code a lot.  Maybe even by 100x.  Faster 
implementations are of course possible, but loads are very common, so 
any extra work will slow things down a lot.

Also, once all these calls to external functions have been added the 
optimiser will likely be severely hindered.  So to get realistic results 
the load profiling instrumentation pass should probably happen as late 
as possible.

Regards,
Alastair.

There is code that does this for older versions of LLVM.  I believe it is in the
giri project in the LLVM SVN repository.  I can look into more details when I
get back from vacation. Swarup may also be able to provide information on the
giri code.

-- John T.

________________________________________
From: llvmdev-bounces at cs.uiuc.edu [llvmdev-bounces at cs.uiuc.edu] on behalf
of Silky Arora [silkyar at umich.edu]
Sent: Saturday, January 12, 2013 10:28 PM
To: llvmdev at cs.uiuc.edu
Subject: [LLVMdev] Dynamic Profiling - Instrumentation basic query

Hi,

I am new to LLVM, and would like to write a dynamic profiler, say which prints
out the load address of all the load instructions encountered in a
program.
>From what I could pick up, edge-profiler.cpp increments a counter
dynamically which is somehow dumped onto llvmprof.out by profile.pl
Could anyone explain me how this works? Can I instrument the code to dump out
the load addresses or other such information to a file?

Thanks!



_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Hi,

@Alastair: Thanks a bunch for explaining this so well. I was able to write
a simple profiler, and run it.

I need to profile the code for branches (branch mis predicts simulation),
load/store instructions (for cache hits/miss rate), and a couple of other
things and therefore, would need to instrument the code.
However, I would like to know if writing the output to a file would
increase the execution time, or is it the profiling itself? I can probably
use a data structure to store the output instead.

Also, I have heard of Intel's Pin tool which can provide memory trace
information. Could you please explain to me what you meant by hardware
counters for dcache miss/hit rates.

@Criswell: Thank you so much for helping me with this.
I am starting to write my own code, but having a look at the existing code
would definitely help me.

Thanks and Regards,
Silky


On Mon, Jan 14, 2013 at 12:06 AM, Criswell, John T <criswell at
illinois.edu>wrote:
> There is code that does this for older versions of LLVM.  I believe it is
> in the giri project in the LLVM SVN repository.  I can look into more
> details when I get back from vacation. Swarup may also be able to provide
> information on the giri code.
>
> -- John T.
>
> ________________________________________
> From: llvmdev-bounces at cs.uiuc.edu [llvmdev-bounces at cs.uiuc.edu] on
behalf
> of Silky Arora [silkyar at umich.edu]
> Sent: Saturday, January 12, 2013 10:28 PM
> To: llvmdev at cs.uiuc.edu
> Subject: [LLVMdev] Dynamic Profiling - Instrumentation basic query
>
> Hi,
>
> I am new to LLVM, and would like to write a dynamic profiler, say which
> prints out the load address of all the load instructions encountered in a
> program.
> From what I could pick up, edge-profiler.cpp increments a counter
> dynamically which is somehow dumped onto llvmprof.out by profile.pl
>
> Could anyone explain me how this works? Can I instrument the code to dump
> out the load addresses or other such information to a file?
>
> Thanks!
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130114/4f8cdf8b/attachment.html>

On 1/13/13 11:06 PM, Criswell, John T wrote:
> There is code that does this for older versions of LLVM.  I believe it is
in the giri project in the LLVM SVN repository.  I can look into more details
when I get back from vacation. Swarup may also be able to provide information on
the giri code.
I took a quick look, and the dynamic slicing code doesn't appear to be 
checked into the giri project yet like I had originally thought.

We can, however, give you a copy of the code if you would like. However, 
having looked at other emails in the thread, I'm not sure if it's what 
you want.  Our dynamic slicing code only instruments LLVM IR loads and 
stores; it does not instrument memory accesses caused by stack spill 
slots, function argument setup, etc. (these are only visible at the code 
generation IR level).

If instrumenting LLVM IR loads and stores suffices, and if you'd like a 
copy of our code, please let me know.

-- John T.

>
> -- John T.
>
> ________________________________________
> From: llvmdev-bounces at cs.uiuc.edu [llvmdev-bounces at cs.uiuc.edu] on
behalf of Silky Arora [silkyar at umich.edu]
> Sent: Saturday, January 12, 2013 10:28 PM
> To: llvmdev at cs.uiuc.edu
> Subject: [LLVMdev] Dynamic Profiling - Instrumentation basic query
>
> Hi,
>
> I am new to LLVM, and would like to write a dynamic profiler, say which
prints out the load address of all the load instructions encountered in a
program.
>  From what I could pick up, edge-profiler.cpp increments a counter
dynamically which is somehow dumped onto llvmprof.out by profile.pl
>
> Could anyone explain me how this works? Can I instrument the code to dump
out the load addresses or other such information to a file?
>
> Thanks!
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev