llvm dev - Jun 2018 - Programmatically Toggle Specific LLVM Optimizations

Hi everyone!

First time poster here. Apologies if I am breaking some rules and please 
let me know so I will not break it again!

I am a summer research student at the Computer Science Department at the 
University of Toronto and I am working on benchmarking LLVM optimizations.

I tried looking for it online but was not able to find a programmatic 
way to toggle specific LLVM optimizations. For example, I would like to 
compile using the -O3 flag but with LICM turned off to see how much the 
compiled binary degrades in performance. I would like the ability to 
simultaneously toggle a few optimizations together as well. I would like 
to benchmark many optimizations so I would like to program the experiments.

So to be really specific, there are two questions:

1. Are there some command line interfaces that can accomplish this task? 
Is there a way to write a bash script to enumerate through the 
optimizations and turning them off one a time?

2. If not, what is a good guide to PassManager so that I can program 
this functionality?

Thanks so much for your help!

Sincerely,
Qiongsi

UTEA Reserach Student 2018, EcoSystem
Computer Science, University of Toronto

Some passes have supported options to disable them, e.g.
-fno-vectorize and -fno-unroll-loops, but there is no general option.
Since it's not useful in general to disable arbitrary options, some
handywork is required. Using the legacy (=default) pass manager here.

To get the passes that are run:
$ clang -O3 -mllvm -debug-pass=Arguments

To optimize separately:
$ clang -Xclang -disable-llvm-passes -S -emit-llvm -o - | opt <your
list of passes> | llc
(not tested, so don't expect to work straight out of the box)

This is not strictly the same as optimizing using clang (pass options
are missing), so there will be performance differences. You could
instead customize PassManagerBuilder::populateModulePassManager() and
PassManagerBuilder::populateFunctionPassManager().

Michael

Hi Michael! 

Thanks for your reply. I managed to add a small amount of code to PassManager to
have this feature implemented.

One can now turn off a pass by doing

opt -disablepass=licm or clang -mllvm -disablepass=licm

to turn off, for example, loop invariant code motion. 

I programmed the command line option to be really hidden. This maybe a useful
thing to have in the trunk for people who want to fine tune the optimizations.

Should I submit a patch? I am not sure if my code follows all the best practices
of the community, but it is working from my own testing.

Thanks again for your help! 

Qiongsi
> On Jun 1, 2018, at 1:16 PM, Michael Kruse <llvmdev at meinersbur.de>
wrote:
> 
> Some passes have supported options to disable them, e.g.
> -fno-vectorize and -fno-unroll-loops, but there is no general option.
> Since it's not useful in general to disable arbitrary options, some
> handywork is required. Using the legacy (=default) pass manager here.
> 
> To get the passes that are run:
> $ clang -O3 -mllvm -debug-pass=Arguments
> 
> To optimize separately:
> $ clang -Xclang -disable-llvm-passes -S -emit-llvm -o - | opt <your
> list of passes> | llc
> (not tested, so don't expect to work straight out of the box)
> 
> This is not strictly the same as optimizing using clang (pass options
> are missing), so there will be performance differences. You could
> instead customize PassManagerBuilder::populateModulePassManager() and
> PassManagerBuilder::populateFunctionPassManager().
> 
> Michael