Dear John, Thanks for your reply. I want to do whole program analysis, all what I need are just the separate .bc files. What instructions should I follow? I run the configure on apache in this way: CC=clang CFLAGS=-emit-llvm ./configure while I get the error message like: checking for gcc... clang checking for C compiler default output file name... configure: error: C compiler cannot create executables Did I miss something? Thanks. -Wink John Criswell wrote:> Wink zhang wrote: >> Hi, >> >> How to compile a large project (such as Apache) by using llvm-gcc? >> I tried to replace CC with llvm-gcc and CFLAGS with -emit-llvm while >> running configure, but it didn't work. >> > If you just want to compile the program with llvm-gcc, then Diego's > instructions are fine. > > If, however, you want to do inter-procedural optimization or whole > program analysis, then there's additional things you need to do. > > If you just want to use LLVM's link-time optimizations, you will need > to use the libLTO library with your linker. Your linker, in turn, > must be able to support plugins (like the gold linker, for example). > For directions on how to set up libLTO, see > http://llvm.org/docs/LinkTimeOptimization.html and > http://llvm.org/docs/GoldPlugin.html. > > If you want to write and use your own link-time passes, then you have > several options: > > 1) Modify the program's Makefiles to use llvm-ld to link bitcode files > together. This will allow you to generate a single bitcode image. > > 2) Modify libLTO to run your analysis and transform passes. If you > plan to work with several large applications, this is probably the way > to go. SAFECode has a version of libLTO that does whole-program > analysis; you can find this tool in the safecode/tools/LTO directory > in the SAFECode source code (directions for getting SAFECode are at > http://safecode.cs.illinois.edu/downloads.html). This approach can > sometimes alleviate the need for changing the Makefiles. > > -- John T. >> Thank you for your help. >> >> -Wink >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > >
Wink zhang wrote:> Dear John, > > Thanks for your reply. I want to do whole program analysis, all what I > need are just the separate .bc files. What instructions should I follow? >I'm a little confused. Whole-program analysis requires that you link all of the individual bitcode files from different compilation unit (i.e., individual .c files) together into a single bitcode file. Are you wanting to do whole program analysis, or are you just analyzing individual compilation units?> > I run the configure on apache in this way: > > CC=clang CFLAGS=-emit-llvm ./configure > > while I get the error message like: > checking for gcc... clang > checking for C compiler default output file name... configure: error: C > compiler cannot create executables > > > > Did I miss something? Thanks. >The problem is that your linker does not know how to link LLVM bitcode files together. When the compiler tries to link two "object files" (in reality, LLVM bitcode files) together, it executes the linker to do linking, but the linker fails because the object files are LLVM bitcode files. This is obviously a problem for link-time optimization, so what the LLVM Developers decided to do was to teach the linker how to link LLVM bitcode files together. They built a dynamic library called libLTO that links LLVM bitcode files together, can do interprocedural optimization on them, and generate native code from them. The gold linker on Linux and the XCode linker on Mac OS X can load this dynamic library; once it does, it can link and optimize LLVM bitcode files, allowing link-time optimization to occur transparently. So, if you are going to do whole program analysis, you have one of two options: 1) Modify the program's build scripts and Makefiles to use llvm-ld to link LLVM bitcode files together. For Apache, this is not too difficult, but it has to be done for every program you compile. 2) You can modify libLTO to run your LLVM passes when creating a final executable (libLTO is told whether it is doing incremental linking or final linking). This approach, in theory, does not require you to change a program's build scripts or Makefiles. I have tried out this approach with SAFECode (which does whole-program analysis). While I have not used it thoroughly yet, I believe it is a promising approach. Note that the libLTO approach can be used for analysis/optimization that does not require whole program analysis; you can have your LLVM passes run whenever libLTO is used or just during final linking. It's up to you. My recommendation is to first determine how many large programs you want to analyze/transform. If it's only 1 or 2, it may be easier to modify the build scripts/Makefiles. If you plan to try your LLVM passes on lots of large programs, then investigating how well the libLTO option works may be worth your time. If you decide to go with option #1, I have modified Apache's build system once to generate a single bitcode file. I can probably provide some guidance in that process if you need it. -- John T.> > -Wink > > John Criswell wrote: > >> Wink zhang wrote: >> >>> Hi, >>> >>> How to compile a large project (such as Apache) by using llvm-gcc? >>> I tried to replace CC with llvm-gcc and CFLAGS with -emit-llvm while >>> running configure, but it didn't work. >>> >>> >> If you just want to compile the program with llvm-gcc, then Diego's >> instructions are fine. >> >> If, however, you want to do inter-procedural optimization or whole >> program analysis, then there's additional things you need to do. >> >> If you just want to use LLVM's link-time optimizations, you will need >> to use the libLTO library with your linker. Your linker, in turn, >> must be able to support plugins (like the gold linker, for example). >> For directions on how to set up libLTO, see >> http://llvm.org/docs/LinkTimeOptimization.html and >> http://llvm.org/docs/GoldPlugin.html. >> >> If you want to write and use your own link-time passes, then you have >> several options: >> >> 1) Modify the program's Makefiles to use llvm-ld to link bitcode files >> together. This will allow you to generate a single bitcode image. >> >> 2) Modify libLTO to run your analysis and transform passes. If you >> plan to work with several large applications, this is probably the way >> to go. SAFECode has a version of libLTO that does whole-program >> analysis; you can find this tool in the safecode/tools/LTO directory >> in the SAFECode source code (directions for getting SAFECode are at >> http://safecode.cs.illinois.edu/downloads.html). This approach can >> sometimes alleviate the need for changing the Makefiles. >> >> -- John T. >> >>> Thank you for your help. >>> >>> -Wink >>> _______________________________________________ >>> LLVM Developers mailing list >>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>> >>> >> > >
Dear John, Thank you very much for your help! I will try the libLTO's approach. I believe the situation can fit me well. Again, thank you for your time. -Wink John Criswell wrote:> Wink zhang wrote: >> Dear John, >> >> Thanks for your reply. I want to do whole program analysis, all what >> I need are just the separate .bc files. What instructions should I >> follow? >> > I'm a little confused. Whole-program analysis requires that you link > all of the individual bitcode files from different compilation unit > (i.e., individual .c files) together into a single bitcode file. Are > you wanting to do whole program analysis, or are you just analyzing > individual compilation units? >> >> I run the configure on apache in this way: >> >> CC=clang CFLAGS=-emit-llvm ./configure >> >> while I get the error message like: >> checking for gcc... clang >> checking for C compiler default output file name... configure: error: >> C compiler cannot create executables >> >> >> >> Did I miss something? Thanks. >> > > The problem is that your linker does not know how to link LLVM bitcode > files together. When the compiler tries to link two "object files" > (in reality, LLVM bitcode files) together, it executes the linker to > do linking, but the linker fails because the object files are LLVM > bitcode files. > > This is obviously a problem for link-time optimization, so what the > LLVM Developers decided to do was to teach the linker how to link LLVM > bitcode files together. They built a dynamic library called libLTO > that links LLVM bitcode files together, can do interprocedural > optimization on them, and generate native code from them. The gold > linker on Linux and the XCode linker on Mac OS X can load this dynamic > library; once it does, it can link and optimize LLVM bitcode files, > allowing link-time optimization to occur transparently. > > So, if you are going to do whole program analysis, you have one of two > options: > > 1) Modify the program's build scripts and Makefiles to use llvm-ld to > link LLVM bitcode files together. For Apache, this is not too > difficult, but it has to be done for every program you compile. > > 2) You can modify libLTO to run your LLVM passes when creating a final > executable (libLTO is told whether it is doing incremental linking or > final linking). This approach, in theory, does not require you to > change a program's build scripts or Makefiles. I have tried out this > approach with SAFECode (which does whole-program analysis). While I > have not used it thoroughly yet, I believe it is a promising approach. > > Note that the libLTO approach can be used for analysis/optimization > that does not require whole program analysis; you can have your LLVM > passes run whenever libLTO is used or just during final linking. It's > up to you. > > My recommendation is to first determine how many large programs you > want to analyze/transform. If it's only 1 or 2, it may be easier to > modify the build scripts/Makefiles. If you plan to try your LLVM > passes on lots of large programs, then investigating how well the > libLTO option works may be worth your time. > > If you decide to go with option #1, I have modified Apache's build > system once to generate a single bitcode file. I can probably provide > some guidance in that process if you need it. > > -- John T. >> >> -Wink >> >> John Criswell wrote: >> >>> Wink zhang wrote: >>> >>>> Hi, >>>> >>>> How to compile a large project (such as Apache) by using llvm-gcc? >>>> I tried to replace CC with llvm-gcc and CFLAGS with -emit-llvm >>>> while running configure, but it didn't work. >>>> >>> If you just want to compile the program with llvm-gcc, then Diego's >>> instructions are fine. >>> >>> If, however, you want to do inter-procedural optimization or whole >>> program analysis, then there's additional things you need to do. >>> >>> If you just want to use LLVM's link-time optimizations, you will >>> need to use the libLTO library with your linker. Your linker, in >>> turn, must be able to support plugins (like the gold linker, for >>> example). For directions on how to set up libLTO, see >>> http://llvm.org/docs/LinkTimeOptimization.html and >>> http://llvm.org/docs/GoldPlugin.html. >>> >>> If you want to write and use your own link-time passes, then you >>> have several options: >>> >>> 1) Modify the program's Makefiles to use llvm-ld to link bitcode >>> files together. This will allow you to generate a single bitcode >>> image. >>> >>> 2) Modify libLTO to run your analysis and transform passes. If you >>> plan to work with several large applications, this is probably the >>> way to go. SAFECode has a version of libLTO that does whole-program >>> analysis; you can find this tool in the safecode/tools/LTO directory >>> in the SAFECode source code (directions for getting SAFECode are at >>> http://safecode.cs.illinois.edu/downloads.html). This approach can >>> sometimes alleviate the need for changing the Makefiles. >>> >>> -- John T. >>> >>>> Thank you for your help. >>>> >>>> -Wink >>>> _______________________________________________ >>>> LLVM Developers mailing list >>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >>>> >>> >> >> > >