Hi All, I am using clang to compile Mysql source code. Because I want to do some inter-procedural analysis, hopefully, I want to get a .o(bitcode) file(a module) containing all possible function declarations and definitions. Is it possible to do that ? Or you guys have some suggestions? To be clear, like Mysql, there is a mysqld routine, which is a major routine. I want to mysqld.o(which is generated by clang) to contain all possible function declarations and definitions. So when I do some analysis, I just need load one module, which is mysqld.o, instead of all .o files. Best, Yuxi -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/6c823316/attachment.html>
Maybe use llvm-link to collect the individual bitcode .o files into one master .o file, then your tool can load just the one big .o file. --paulr From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Yuxi Chen via llvm-dev Sent: Thursday, March 10, 2016 7:55 PM To: llvm-dev at lists.llvm.org Cc: llvmdev-bounces at cs.uiuc.edu; llvmdev at cs.uiuc.edu Subject: [llvm-dev] big module for a project Hi All, I am using clang to compile Mysql source code. Because I want to do some inter-procedural analysis, hopefully, I want to get a .o(bitcode) file(a module) containing all possible function declarations and definitions. Is it possible to do that ? Or you guys have some suggestions? To be clear, like Mysql, there is a mysqld routine, which is a major routine. I want to mysqld.o(which is generated by clang) to contain all possible function declarations and definitions. So when I do some analysis, I just need load one module, which is mysqld.o, instead of all .o files. Best, Yuxi -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/aa0065ce/attachment.html>
Take a look at wllvm (https://github.com/travitch/whole-program-llvm) 2016-03-11 10:55 GMT-06:00 Robinson, Paul via llvm-dev < llvm-dev at lists.llvm.org>:> Maybe use llvm-link to collect the individual bitcode .o files into one > master .o file, then your tool can load just the one big .o file. > > --paulr > > > > *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of *Yuxi > Chen via llvm-dev > *Sent:* Thursday, March 10, 2016 7:55 PM > *To:* llvm-dev at lists.llvm.org > *Cc:* llvmdev-bounces at cs.uiuc.edu; llvmdev at cs.uiuc.edu > *Subject:* [llvm-dev] big module for a project > > > > Hi All, > > I am using clang to compile Mysql source code. Because I want to do some > inter-procedural analysis, hopefully, I want to get a .o(bitcode) file(a > module) containing all possible function declarations and definitions. Is > it possible to do that ? Or you guys have some suggestions? > > To be clear, like Mysql, there is a mysqld routine, which is a major > routine. I want to mysqld.o(which is generated by clang) to contain all > possible function declarations and definitions. So when I do some analysis, > I just need load one module, which is mysqld.o, instead of all .o files. > > > Best, > Yuxi > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/f7109f4c/attachment.html>
Dear Yuxi, There are three general ways to do this (two of which have already been mentioned). First, you can change the Makefiles to emit bitcode and then use llvm-link to link all the bitcode files together into a big bitcode file. You can then run the opt program on this large bitcode file. Second, you can use some third-party tools (for lack of a better term) that will get you a single bitcode file with fewer changes to the Makefiles. I'm not very familiar with these, but I think others have commented on them. Third, you can add your analysis pass to the libLTO library and use LLVM's link-time optimization framework to run your analysis. Documentation libLTO can be found at http://llvm.org/docs/LinkTimeOptimization.html and http://llvm.org/docs/GoldPlugin.html. LLVM's LTO infrastructure is designed to be transparent (i.e., you shouldn't have to modify Makefiles to use it), though some programs' Makefiles are more...complaint than others. I've used the libLTO approach for large programs with reasonable success. However, some of the other third-party approaches have been reported to work well, so you might try looking at those. Regards, John Criswell On 3/10/16 9:54 PM, Yuxi Chen via llvm-dev wrote:> Hi All, > > I am using clang to compile Mysql source code. Because I want to do > some inter-procedural analysis, hopefully, I want to get a .o(bitcode) > file(a module) containing all possible function declarations and > definitions. Is it possible to do that ? Or you guys have some > suggestions? > > To be clear, like Mysql, there is a mysqld routine, which is a major > routine. I want to mysqld.o(which is generated by clang) to contain > all possible function declarations and definitions. So when I do some > analysis, I just need load one module, which is mysqld.o, instead of > all .o files. > > > Best, > Yuxi > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-- John Criswell Assistant Professor Department of Computer Science, University of Rochester http://www.cs.rochester.edu/u/criswell -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160311/30852946/attachment-0001.html>
Thanks for your reply. I tested it on a toy program. It works. Best, Yuxi ________________________________ From: Robinson, Paul [Paul_Robinson at playstation.sony.com] Sent: Friday, March 11, 2016 10:55 AM To: Yuxi Chen; llvm-dev at lists.llvm.org Subject: RE: [llvm-dev] big module for a project Maybe use llvm-link to collect the individual bitcode .o files into one master .o file, then your tool can load just the one big .o file. --paulr From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Yuxi Chen via llvm-dev Sent: Thursday, March 10, 2016 7:55 PM To: llvm-dev at lists.llvm.org Cc: llvmdev-bounces at cs.uiuc.edu; llvmdev at cs.uiuc.edu Subject: [llvm-dev] big module for a project Hi All, I am using clang to compile Mysql source code. Because I want to do some inter-procedural analysis, hopefully, I want to get a .o(bitcode) file(a module) containing all possible function declarations and definitions. Is it possible to do that ? Or you guys have some suggestions? To be clear, like Mysql, there is a mysqld routine, which is a major routine. I want to mysqld.o(which is generated by clang) to contain all possible function declarations and definitions. So when I do some analysis, I just need load one module, which is mysqld.o, instead of all .o files. Best, Yuxi -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160312/698029e2/attachment.html>
Hi John, Really appreciate your detailed information. The first one works for me. I will take a look at the third solution. Thanks again, Yuxi ________________________________ From: John Criswell [jtcriswel at gmail.com] Sent: Friday, March 11, 2016 11:26 AM To: Yuxi Chen; llvm-dev at lists.llvm.org Cc: llvmdev-bounces at cs.uiuc.edu; llvmdev at cs.uiuc.edu Subject: Re: [llvm-dev] big module for a project Dear Yuxi, There are three general ways to do this (two of which have already been mentioned). First, you can change the Makefiles to emit bitcode and then use llvm-link to link all the bitcode files together into a big bitcode file. You can then run the opt program on this large bitcode file. Second, you can use some third-party tools (for lack of a better term) that will get you a single bitcode file with fewer changes to the Makefiles. I'm not very familiar with these, but I think others have commented on them. Third, you can add your analysis pass to the libLTO library and use LLVM's link-time optimization framework to run your analysis. Documentation libLTO can be found at http://llvm.org/docs/LinkTimeOptimization.html and http://llvm.org/docs/GoldPlugin.html. LLVM's LTO infrastructure is designed to be transparent (i.e., you shouldn't have to modify Makefiles to use it), though some programs' Makefiles are more...complaint than others. I've used the libLTO approach for large programs with reasonable success. However, some of the other third-party approaches have been reported to work well, so you might try looking at those. Regards, John Criswell On 3/10/16 9:54 PM, Yuxi Chen via llvm-dev wrote: Hi All, I am using clang to compile Mysql source code. Because I want to do some inter-procedural analysis, hopefully, I want to get a .o(bitcode) file(a module) containing all possible function declarations and definitions. Is it possible to do that ? Or you guys have some suggestions? To be clear, like Mysql, there is a mysqld routine, which is a major routine. I want to mysqld.o(which is generated by clang) to contain all possible function declarations and definitions. So when I do some analysis, I just need load one module, which is mysqld.o, instead of all .o files. Best, Yuxi _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -- John Criswell Assistant Professor Department of Computer Science, University of Rochester http://www.cs.rochester.edu/u/criswell -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160312/ba1adc6f/attachment.html>
Hi John, One more question, I try to use libLTO, and try example ON http://llvm.org/docs/GoldPlugin.html. For the example, it directly generates executable, what I want is a .o file(which I can use llvm-dis to see the bitcode). Maybe I misinterpret your solution. Or you want to say if I follow the instruction like http://llvm.org/docs/GoldPlugin.html. After ./configure&&make&&make ckeck, I can get one module, containing as many functions as possible. Best, Yuxi ________________________________ From: John Criswell [jtcriswel at gmail.com] Sent: Friday, March 11, 2016 11:26 AM To: Yuxi Chen; llvm-dev at lists.llvm.org Cc: llvmdev-bounces at cs.uiuc.edu; llvmdev at cs.uiuc.edu Subject: Re: [llvm-dev] big module for a project Dear Yuxi, There are three general ways to do this (two of which have already been mentioned). First, you can change the Makefiles to emit bitcode and then use llvm-link to link all the bitcode files together into a big bitcode file. You can then run the opt program on this large bitcode file. Second, you can use some third-party tools (for lack of a better term) that will get you a single bitcode file with fewer changes to the Makefiles. I'm not very familiar with these, but I think others have commented on them. Third, you can add your analysis pass to the libLTO library and use LLVM's link-time optimization framework to run your analysis. Documentation libLTO can be found at http://llvm.org/docs/LinkTimeOptimization.html and http://llvm.org/docs/GoldPlugin.html. LLVM's LTO infrastructure is designed to be transparent (i.e., you shouldn't have to modify Makefiles to use it), though some programs' Makefiles are more...complaint than others. I've used the libLTO approach for large programs with reasonable success. However, some of the other third-party approaches have been reported to work well, so you might try looking at those. Regards, John Criswell On 3/10/16 9:54 PM, Yuxi Chen via llvm-dev wrote: Hi All, I am using clang to compile Mysql source code. Because I want to do some inter-procedural analysis, hopefully, I want to get a .o(bitcode) file(a module) containing all possible function declarations and definitions. Is it possible to do that ? Or you guys have some suggestions? To be clear, like Mysql, there is a mysqld routine, which is a major routine. I want to mysqld.o(which is generated by clang) to contain all possible function declarations and definitions. So when I do some analysis, I just need load one module, which is mysqld.o, instead of all .o files. Best, Yuxi _______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev -- John Criswell Assistant Professor Department of Computer Science, University of Rochester http://www.cs.rochester.edu/u/criswell -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160312/d51bcffd/attachment-0001.html>