Hi, I have a module pass and I hope to use it to optimize a real-world program. I need LTO，and I have got LTO plugin. But How can I add my passes to LTO Pass. I can't find solution. What should I do? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200422/76d2b046/attachment.html>
Teresa Johnson via llvm-dev
2020-Apr-22 15:11 UTC
[llvm-dev] how to add my own passes to LTO pass
On Wed, Apr 22, 2020 at 5:12 AM y liu via llvm-dev <llvm-dev at lists.llvm.org> wrote:> Hi, > I have a module pass and I hope to use it to optimize a real-world program. > I need LTO，and I have got LTO plugin. >Do you mean the LLVM gold plugin (relevant when doing LTO links with gold or gnu ld), or are you trying to load your new pass via a pass plugin?> But How can I add my passes to LTO Pass. I can't find solution. > What should I do? >This depends on whether you are trying to add via a pass plugin dynamically via the command line (I believe the support for this is work in progress), or by manually modifying the pass pipeline within LLVM. Have you implemented as an old pass manager pass (the current default) or the new pass manager or both? Is it an LTO specific pass, or do you also want to run for non-LTO optimizing compiles (most optimizing passes should run in all cases). Assuming the latter, and also assuming you are using the default old pass manager, you would want to look at the following places in PassManagerBuilder.cpp: regular (full) LTO: PassManagerBuilder::addLTOOptimizationPasses ThinLTO: PassManagerBuilder::populateModulePassManager, which is used both by the pre-LTO compile and by the LTO linking ThinLTO compiles. If you want to insert your pass for both, just put it in the relevant place in this file, if you want to only do during the LTO link portion of the ThinLTO compile, then guard it with a check for "PerformThinLTO". For the new pass manager, it would be added to PassBuilder.cpp (that is structured a bit differently so let me know and I can point you in the right direction). If you are planning to send upstream it should be supported by both pass managers. HTH, Teresa _______________________________________________> LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- Teresa Johnson | Software Engineer | tejohnson at google.com | -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200422/91ae6c6c/attachment.html>
Teresa Johnson via llvm-dev
2020-Apr-23 17:16 UTC
[llvm-dev] how to add my own passes to LTO pass
cc-ed llvm-dev back so others can hopefully help. I don't load passes via plugins so this is outside my area of expertise. But I have seen some recent patches fly by that are working on this. I saw from an earlier question you posted that you are using gold. There is a patch under development/review to get pass plugins to work for LTO through gold. See https://reviews.llvm.org/D77704. Eli (added) was working on adding support as well, here are a couple of patches he recently landed for this in LTO and lld: https://reviews.llvm.org/D75879 (lld) https://reviews.llvm.org/D76866 (LTO) I'm not completely sure what the status is at this point though. Eli is probably the best person to answer. Teresa On Wed, Apr 22, 2020 at 7:50 PM y liu <ly820nha at gmail.com> wrote:> I‘m using LLVM 10, and register my pass by following code: > char MyModulePass::ID = 0; > static RegisterPass<MyModulePass> X("MyModule", "MyModule pass"); > static void registerMyModulePass(const PassManagerBuilder &, > legacy::PassManagerBase &PM) { > PM.add(new MyModulePass()); > } > static RegisterStandardPasses > RegisterMyPass(PassManagerBuilder::EP_FullLinkTimeOptimizationLast, > registerMyModulePass); > MyModulePass reles on helper.c. > I want to use it for optimization after the link is compelte. > Now, I compile it manually with opt. > For example: > > Target: fun1.c fun2.c fun3.c main.c > Pass: Mypass.so > Dependency: helper.c > > fun1.c fun2.c fun3.c main.c ---> target.bc > helper.c ---> helper.o > > target.bc helper.o ---> target-link.bc ----> opt Mypass.so ---> > target-opt.bc ---> target > > I want to achieve my goals by LTO. > > Teresa Johnson <tejohnson at google.com>于2020年4月22日 周三23:11写道： > >> >> >> On Wed, Apr 22, 2020 at 5:12 AM y liu via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> Hi, >>> I have a module pass and I hope to use it to optimize a real-world >>> program. >>> I need LTO，and I have got LTO plugin. >>> >> >> Do you mean the LLVM gold plugin (relevant when doing LTO links with gold >> or gnu ld), or are you trying to load your new pass via a pass plugin? >> > Yes, it’s gold plugin, I need it to link .bc and .o files. I want to >> optimize the target after linking. >> > But How can I add my passes to LTO Pass. I can't find solution. >>> What should I do? >>> >> >> This depends on whether you are trying to add via a pass plugin >> dynamically via the command line (I believe the support for this is work in >> progress), or by manually modifying the pass pipeline within LLVM. >> Have you implemented as an old pass manager pass (the current default) or >> the new pass manager or both? >> Is it an LTO specific pass, or do you also want to run for non-LTO >> optimizing compiles (most optimizing passes should run in all cases). >> >> Assuming the latter, and also assuming you are using the default old pass >> manager, you would want to look at the following places in >> PassManagerBuilder.cpp: >> >> regular (full) LTO: PassManagerBuilder::addLTOOptimizationPasses >> >> ThinLTO: PassManagerBuilder::populateModulePassManager, which is used >> both by the pre-LTO compile and by the LTO linking ThinLTO compiles. If you >> want to insert your pass for both, just put it in the relevant place in >> this file, if you want to only do during the LTO link portion of the >> ThinLTO compile, then guard it with a check for "PerformThinLTO". >> >> For the new pass manager, it would be added to PassBuilder.cpp (that is >> structured a bit differently so let me know and I can point you in the >> right direction). If you are planning to send upstream it should be >> supported by both pass managers. >> >> HTH, >> Teresa >> >> _______________________________________________ >> >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> >>> >> >> -- >> Teresa Johnson | Software Engineer | tejohnson at google.com | >> >-- Teresa Johnson | Software Engineer | tejohnson at google.com | -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200423/3e0aa912/attachment.html>
Eli Friedman via llvm-dev
2020-Apr-23 18:59 UTC
[llvm-dev] how to add my own passes to LTO pass
It should be possible to dynamically load legacy pass manager plugins into lld
on trunk. Pass “-mllvm=-load=plugin.so” to lld (“-fuse-ld=lld
-Wl,-mllvm=-load=plugin.so” to clang) for a plugin with an appropriate
RegisterStandardPasses, and it should be called.
Dynamically loading pass plugins into gold is still in progress. (See
https://reviews.llvm.org/D77704.)
It’s also possible to statically link both legacy pass manager and new pass
manager LTO pass plugins into LLVM; that should work for both gold and lld
without any command-line flags at runtime.
llvm/examples/Bye/ is a very simple pass plugin which supports running during
LTO; it should serve as a reasonable template if you’re having trouble getting
started. It support both the legacy pass manager and the new pass manager; new
pass manager isn’t enabled by default, so you can ignore the getByePluginInfo()
etc. if you’re not interested in that.
There’s a little documentation for pass plugins at
http://llvm.org/docs/WritingAnLLVMPass.html#building-pass-plugins ; the
documentation could probably use some work.
Note that all of this is only working on trunk; 10.0 doesn’t have the changes in
question.
-Eli
From: Teresa Johnson <tejohnson at google.com>
Sent: Thursday, April 23, 2020 10:16 AM
To: y liu <ly820nha at gmail.com>; Eli Friedman <efriedma at
quicinc.com>
Cc: llvm-dev <llvm-dev at lists.llvm.org>
Subject: [EXT] Re: [llvm-dev] how to add my own passes to LTO pass
cc-ed llvm-dev back so others can hopefully help. I don't load passes via
plugins so this is outside my area of expertise. But I have seen some recent
patches fly by that are working on this.
I saw from an earlier question you posted that you are using gold. There is a
patch under development/review to get pass plugins to work for LTO through gold.
See https://reviews.llvm.org/D77704.
Eli (added) was working on adding support as well, here are a couple of patches
he recently landed for this in LTO and lld:
https://reviews.llvm.org/D75879 (lld)
https://reviews.llvm.org/D76866 (LTO)
I'm not completely sure what the status is at this point though. Eli is
probably the best person to answer.
Teresa
On Wed, Apr 22, 2020 at 7:50 PM y liu <ly820nha at
gmail.com<mailto:ly820nha at gmail.com>> wrote:
I‘m using LLVM 10, and register my pass by following code:
char MyModulePass::ID = 0;
static RegisterPass<MyModulePass> X("MyModule", "MyModule
pass");
static void registerMyModulePass(const PassManagerBuilder &,
legacy::PassManagerBase &PM) {
PM.add(new MyModulePass());
}
static RegisterStandardPasses
RegisterMyPass(PassManagerBuilder::EP_FullLinkTimeOptimizationLast,
registerMyModulePass);
MyModulePass reles on helper.c.
I want to use it for optimization after the link is compelte.
Now, I compile it manually with opt.
For example:
Target: fun1.c fun2.c fun3.c main.c
Pass: Mypass.so
Dependency: helper.c
fun1.c fun2.c fun3.c main.c ---> target.bc
helper.c ---> helper.o
target.bc helper.o ---> target-link.bc ----> opt Mypass.so --->
target-opt.bc ---> target
I want to achieve my goals by LTO.
Teresa Johnson <tejohnson at google.com<mailto:tejohnson at
google.com>>于2020年4月22日 周三23:11写道：
On Wed, Apr 22, 2020 at 5:12 AM y liu via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hi,
I have a module pass and I hope to use it to optimize a real-world program.
I need LTO，and I have got LTO plugin.
Do you mean the LLVM gold plugin (relevant when doing LTO links with gold or gnu
ld), or are you trying to load your new pass via a pass plugin?
Yes, it’s gold plugin, I need it to link .bc and .o files. I want to optimize
the target after linking.
But How can I add my passes to LTO Pass. I can't find solution.
What should I do?
This depends on whether you are trying to add via a pass plugin dynamically via
the command line (I believe the support for this is work in progress), or by
manually modifying the pass pipeline within LLVM.
Have you implemented as an old pass manager pass (the current default) or the
new pass manager or both?
Is it an LTO specific pass, or do you also want to run for non-LTO optimizing
compiles (most optimizing passes should run in all cases).
Assuming the latter, and also assuming you are using the default old pass
manager, you would want to look at the following places in
PassManagerBuilder.cpp:
regular (full) LTO: PassManagerBuilder::addLTOOptimizationPasses
ThinLTO: PassManagerBuilder::populateModulePassManager, which is used both by
the pre-LTO compile and by the LTO linking ThinLTO compiles. If you want to
insert your pass for both, just put it in the relevant place in this file, if
you want to only do during the LTO link portion of the ThinLTO compile, then
guard it with a check for "PerformThinLTO".
For the new pass manager, it would be added to PassBuilder.cpp (that is
structured a bit differently so let me know and I can point you in the right
direction). If you are planning to send upstream it should be supported by both
pass managers.
HTH,
Teresa
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
--
Teresa Johnson |
Software Engineer |
tejohnson at google.com<mailto:tejohnson at google.com> |
--
Teresa Johnson |
Software Engineer |
tejohnson at google.com<mailto:tejohnson at google.com> |
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200423/84734691/attachment.html>