llvm dev - Jul 2015 - [LLVMdev] Load MachineFunctionPass plugin from library in llc?

Hi all,

 

I am working on creating a MachineFunctionPass to perform an analysis on X86
code. After a bit of trouble, I was able to get my pass compiling and
running correctly in llc. However, since a machine pass is compiled directly
into the code generator, rerunning "make" across the LLVM build tree
involves re-linking most of the major libraries and executables. This is a
rather intense process, both in terms of CPU usage and memory (some of the
bigger binaries, especially clang, use as much as 5 GB peak memory during
linking). With a powerful multicore machine and plenty of RAM, this is
doable, but still very time-consuming, and seems wastefully repetitive.

 

For a front-end pass, I can load the pass from a dynamic shared library
using "opt -l path_to_library"; thus I only need to rebuild the pass
itself
to test a code change. There does not appear to be such an option in llc for
dynamically linking a machine pass as a plugin. Does anyone know of a way to
achieve something similar?

 

At the very least, it would be nice to be able to just re-compile the parts
of the LLVM tree I need (perhaps just llc). It seems particularly odd that
clang, which is a front-end, would need to be relinked for a code change in
a back-end pass. (clang, and some of its related components, are responsible
for the lion's share of the time-consuming CPU/memory-intensive linking
phase.)

 

FYI, I am using the Autotools (./configure, make) process to build LLVM, but
have also successfully built with CMake and would be willing to try
suggestions involving either.

 

Any advice/suggestions would be greatly appreciated!

 

Thanks,

Ethan Johnson

 

Ethan J. Johnson

Computer Science PhD student, Systems group, University of Rochester

 <mailto:ejohns48 at cs.rochester.edu> ejohns48 at cs.rochester.edu

 <mailto:ethanjohnson at acm.org> ethanjohnson at acm.org

PGP pubkey available from public directory or on request

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150702/8a65b8f9/attachment.html>

> On Jul 2, 2015, at 1:17 PM, Ethan J. Johnson <ejohns48 at
cs.rochester.edu> wrote:
> 
> Hi all,
>  
> I am working on creating a MachineFunctionPass to perform an analysis on
X86 code. After a bit of trouble, I was able to get my pass compiling and
running correctly in llc. However, since a machine pass is compiled directly
into the code generator, rerunning “make” across the LLVM build tree involves
re-linking most of the major libraries and executables. This is a rather intense
process, both in terms of CPU usage and memory (some of the bigger binaries,
especially clang, use as much as 5 GB peak memory during linking). With a
powerful multicore machine and plenty of RAM, this is doable, but still very
time-consuming, and seems wastefully repetitive.
>  
> For a front-end pass, I can load the pass from a dynamic shared library
using “opt –l path_to_library”; thus I only need to rebuild the pass itself to
test a code change. There does not appear to be such an option in llc for
dynamically linking a machine pass as a plugin. Does anyone know of a way to
achieve something similar?
There is unfortunately no way to do that currently.
>  
> At the very least, it would be nice to be able to just re-compile the parts
of the LLVM tree I need (perhaps just llc). It seems particularly odd that
clang, which is a front-end, would need to be relinked for a code change in a
back-end pass. (clang, and some of its related components, are responsible for
the lion’s share of the time-consuming CPU/memory-intensive linking phase.)
>  
clang the project is the front end, but clang the executable program is the full
compiler, including all of optimization, codegen, object emission, etc..
> FYI, I am using the Autotools (./configure, make) process to build LLVM,
but have also successfully built with CMake and would be willing to try
suggestions involving either.
This, however, can be done, at least w/ CMake+ninja (probably other
permutations, too, but that’s the one I use these days). Just do “ninja llc” and
it will build just what’s required for llc. Same for any of the other tools.


>  
> Any advice/suggestions would be greatly appreciated!
>  
> Thanks,
> Ethan Johnson
>  
> Ethan J. Johnson
> Computer Science PhD student, Systems group, University of Rochester
> ejohns48 at cs.rochester.edu <mailto:ejohns48 at cs.rochester.edu>
> ethanjohnson at acm.org <mailto:ethanjohnson at acm.org>
> PGP pubkey available from public directory or on request
>  
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu <mailto:LLVMdev at cs.uiuc.edu>        
http://llvm.cs.uiuc.edu <http://llvm.cs.uiuc.edu/>
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
<http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150702/cd441dfc/attachment.html>

> On Jul 2, 2015, at 2:05 PM, Jim Grosbach <grosbach at apple.com>
wrote:
> 
>> 
>> On Jul 2, 2015, at 1:17 PM, Ethan J. Johnson <ejohns48 at
cs.rochester.edu <mailto:ejohns48 at cs.rochester.edu>> wrote:
>> 
>> Hi all,
>>  
>> I am working on creating a MachineFunctionPass to perform an analysis
on X86 code. After a bit of trouble, I was able to get my pass compiling and
running correctly in llc. However, since a machine pass is compiled directly
into the code generator, rerunning “make” across the LLVM build tree involves
re-linking most of the major libraries and executables. This is a rather intense
process, both in terms of CPU usage and memory (some of the bigger binaries,
especially clang, use as much as 5 GB peak memory during linking). With a
powerful multicore machine and plenty of RAM, this is doable, but still very
time-consuming, and seems wastefully repetitive.
>>  
>> For a front-end pass, I can load the pass from a dynamic shared library
using “opt –l path_to_library”; thus I only need to rebuild the pass itself to
test a code change. There does not appear to be such an option in llc for
dynamically linking a machine pass as a plugin. Does anyone know of a way to
achieve something similar?
> 
> There is unfortunately no way to do that currently.
> 
>>  
>> At the very least, it would be nice to be able to just re-compile the
parts of the LLVM tree I need (perhaps just llc). It seems particularly odd that
clang, which is a front-end, would need to be relinked for a code change in a
back-end pass. (clang, and some of its related components, are responsible for
the lion’s share of the time-consuming CPU/memory-intensive linking phase.)
>>  
> 
> clang the project is the front end, but clang the executable program is the
full compiler, including all of optimization, codegen, object emission, etc..
> 
>> FYI, I am using the Autotools (./configure, make) process to build
LLVM, but have also successfully built with CMake and would be willing to try
suggestions involving either.
> 
> This, however, can be done, at least w/ CMake+ninja (probably other
permutations, too, but that’s the one I use these days). Just do “ninja llc” and
it will build just what’s required for llc. Same for any of the other tools.
Also enabling shared libraries "cmake -DBUILD_SHARED_LIBS=ON" reduces
link times dramatically in my experience.

- Matthias
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150702/963b421b/attachment.html>