On Wed, Feb 26, 2014 at 9:58 PM, Brad King <brad.king at kitware.com> wrote:> On 02/26/2014 12:43 PM, Alexey Samsonov wrote: > > Do you think it makes sense to land my ExternalProject_Add patch > > so that others can experiment with it? I can add quit with a > > fatal_error/warning if the build tree rules are generated with Ninja. > > Since it is conditional on LLVM_BUILD_EXTERNAL_COMPILER_RT, yes. >Submitted as r202367.> > > parallelism doesn't work when I run "make check-compiler-rt -j8" > > in the original build tree, presumably because we call > > "cd /path/to/compiler-rt/build/tree && make check-all" there. > > Right. The ExternalProject module has a special case for the > Makefile generators to make with $(MAKE) instead of "make": > > > http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v2.8.12.2#l846 > > so that flags like -j propagate automatically. You could do > that too: > > if(CMAKE_GENERATOR MATCHES "Make") > set(check_compiler_rt "$(MAKE)" "check-all") > else() > set(check_compiler_rt ${CMAKE_COMMAND} --build . > --target check-all --config $<CONFIGURATION>) > endif() > > ExternalProject_Get_Property(compiler-rt BINARY_DIR) > add_custom_target(check-compiler-rt > COMMAND ${check_compiler_rt} > DEPENDS compiler-rt > WORKING_DIRECTORY ${BINARY_DIR} > VERBATIM > ) > >This worked, thanks! Currently I also print fatal_error message if I detect Ninja as a CMAKE_GENERATOR. -- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140227/cce3482d/attachment.html>
> ExternalProject_Add(compiler-rt ...)So that was quite the experiment. Looking at clang/runtime/CMakeLists.txt, I'm not seeing a lot of bang for buck here, and it looks like this file is prone to bit rot. I think we should consider punting on this one and looking for a more incremental strategy. For instance, how about starting by moving the call 'add_llvm_external_project(compiler-rt)' from llvm/projects/CMakesLists.txt to clang/runtime? We can tweak this macro (or a clang variant) to optionally do something like: include(AddCompilerRt) That way if CMAKE_PREFIX_PATH includes a path to the compiler-rt install directory, it can import all the same build targets that would be created by add_subdirectory(compiler-rt). If I were to use CMake's ExternalProject feature, it would be from a top-level superproject. I would use it to manage dependencies and to populate CMAKE_PREFIX_PATH for each dependency. CMake, Make or a package manager, it wouldn't matter which you used for this part (I've used Make for this). Ultimately, you want to end up with a top-level build that performs the following: mkdir -p llvm/out && cd llvm/out cmake .. -DCMAKE_INSTALL_PREFIX=ship ninja check-all install cd - mkdir -p compiler-rt/out && cd compiler-rt/out cmake .. -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship -DCMAKE_INSTALL_PREFIX=ship ninja check-all install cd - mkdir -p clang/out && cd clang/out cmake .. -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship:`pwd`/../../compiler-rt/out/ship -DCMAKE_INSTALL_PREFIX=ship ninja check-all install cd - In this scenario, the compiler-rt build creates a cmake module that clang includes. It uses that cmake module to find the compiler-rt install targets and copy them into "lib/clang/<version>". The clang build then calls 'add_subdirectory' on the compiler-rt 'test' directory, pointing it to the just-built-clang. So the dependency tree for clang is: sanitizer-tests -> clang -> compiler-rt -> c-compiler Thoughts? Thanks, Greg On Thu, Feb 27, 2014 at 1:19 AM, Alexey Samsonov <samsonov at google.com> wrote:> > On Wed, Feb 26, 2014 at 9:58 PM, Brad King <brad.king at kitware.com> wrote: >> >> On 02/26/2014 12:43 PM, Alexey Samsonov wrote: >> > Do you think it makes sense to land my ExternalProject_Add patch >> > so that others can experiment with it? I can add quit with a >> > fatal_error/warning if the build tree rules are generated with Ninja. >> >> Since it is conditional on LLVM_BUILD_EXTERNAL_COMPILER_RT, yes. > > > Submitted as r202367. > >> >> >> > parallelism doesn't work when I run "make check-compiler-rt -j8" >> > in the original build tree, presumably because we call >> > "cd /path/to/compiler-rt/build/tree && make check-all" there. >> >> Right. The ExternalProject module has a special case for the >> Makefile generators to make with $(MAKE) instead of "make": >> >> >> http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v2.8.12.2#l846 >> >> so that flags like -j propagate automatically. You could do >> that too: >> >> if(CMAKE_GENERATOR MATCHES "Make") >> set(check_compiler_rt "$(MAKE)" "check-all") >> else() >> set(check_compiler_rt ${CMAKE_COMMAND} --build . >> --target check-all --config $<CONFIGURATION>) >> endif() >> >> ExternalProject_Get_Property(compiler-rt BINARY_DIR) >> add_custom_target(check-compiler-rt >> COMMAND ${check_compiler_rt} >> DEPENDS compiler-rt >> WORKING_DIRECTORY ${BINARY_DIR} >> VERBATIM >> ) >> > > This worked, thanks! Currently I also print fatal_error message if I detect > Ninja as a CMAKE_GENERATOR. > > > -- > Alexey Samsonov, MSK > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >
On Thu, Mar 20, 2014 at 10:12 PM, Greg Fitzgerald <garious at gmail.com> wrote:> > ExternalProject_Add(compiler-rt ...) > > So that was quite the experiment. Looking at > clang/runtime/CMakeLists.txt, I'm not seeing a lot of bang for buck > here, and it looks like this file is prone to bit rot.Could you please elaborate on this? In fact, I don't plan to give up on this experiment. On the contrary, I wanted to move LLVM_BUILD_EXTERNAL_COMPILER_RT to the bots, migrate the workflow of our team to use this mode, and eventually make it the default. The problem is that we want top-level targets from compiler-rt build tree be visible in the llvm/clang top-level build tree. I thought this could be achieved by ugly propagation of top-level compiler-rt targets with add_custom_target() command. (like I did this for check-compiler-rt command).> I think we > should consider punting on this one and looking for a more incremental > strategy. For instance, how about starting by moving the call > 'add_llvm_external_project(compiler-rt)' from > llvm/projects/CMakesLists.txt to clang/runtime? We can tweak this > macro (or a clang variant) to optionally do something like: > > include(AddCompilerRt) > > That way if CMAKE_PREFIX_PATH includes a path to the compiler-rt > install directory, it can import all the same build targets that would > be created by add_subdirectory(compiler-rt). >Probably I don't understand how CMAKE_PREFIX_PATH works. How can top-level targets from one build tree be visible in another build tree?> > If I were to use CMake's ExternalProject feature, it would be from a > top-level superproject. I would use it to manage dependencies and to > populate CMAKE_PREFIX_PATH for each dependency. CMake, Make or a > package manager, it wouldn't matter which you used for this part (I've > used Make for this). Ultimately, you want to end up with a top-level > build that performs the following: > > mkdir -p llvm/out && cd llvm/out > cmake .. -DCMAKE_INSTALL_PREFIX=ship > ninja check-all install > cd - > > mkdir -p compiler-rt/out && cd compiler-rt/out > cmake .. -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship > -DCMAKE_INSTALL_PREFIX=ship > ninja check-all install > cd - > > mkdir -p clang/out && cd clang/out > cmake .. > -DCMAKE_PREFIX_PATH=`pwd`/../../llvm/out/ship:`pwd`/../../compiler-rt/out/ship > -DCMAKE_INSTALL_PREFIX=ship > ninja check-all install > cd - > > In this scenario, the compiler-rt build creates a cmake module that > clang includes. It uses that cmake module to find the compiler-rt > install targets and copy them into "lib/clang/<version>". The clang > build then calls 'add_subdirectory' on the compiler-rt 'test' > directory, pointing it to the just-built-clang. So the dependency > tree for clang is: > > sanitizer-tests -> clang -> compiler-rt -> c-compiler >... But we want to build compiler-rt libraries with just-built Clang as well. This is what Makefile-based build does, and what we can benefit from.> > Thoughts? > > Thanks, > Greg > > > On Thu, Feb 27, 2014 at 1:19 AM, Alexey Samsonov <samsonov at google.com> > wrote: > > > > On Wed, Feb 26, 2014 at 9:58 PM, Brad King <brad.king at kitware.com> > wrote: > >> > >> On 02/26/2014 12:43 PM, Alexey Samsonov wrote: > >> > Do you think it makes sense to land my ExternalProject_Add patch > >> > so that others can experiment with it? I can add quit with a > >> > fatal_error/warning if the build tree rules are generated with Ninja. > >> > >> Since it is conditional on LLVM_BUILD_EXTERNAL_COMPILER_RT, yes. > > > > > > Submitted as r202367. > > > >> > >> > >> > parallelism doesn't work when I run "make check-compiler-rt -j8" > >> > in the original build tree, presumably because we call > >> > "cd /path/to/compiler-rt/build/tree && make check-all" there. > >> > >> Right. The ExternalProject module has a special case for the > >> Makefile generators to make with $(MAKE) instead of "make": > >> > >> > >> > http://cmake.org/gitweb?p=cmake.git;a=blob;f=Modules/ExternalProject.cmake;hb=v2.8.12.2#l846 > >> > >> so that flags like -j propagate automatically. You could do > >> that too: > >> > >> if(CMAKE_GENERATOR MATCHES "Make") > >> set(check_compiler_rt "$(MAKE)" "check-all") > >> else() > >> set(check_compiler_rt ${CMAKE_COMMAND} --build . > >> --target check-all --config $<CONFIGURATION>) > >> endif() > >> > >> ExternalProject_Get_Property(compiler-rt BINARY_DIR) > >> add_custom_target(check-compiler-rt > >> COMMAND ${check_compiler_rt} > >> DEPENDS compiler-rt > >> WORKING_DIRECTORY ${BINARY_DIR} > >> VERBATIM > >> ) > >> > > > > This worked, thanks! Currently I also print fatal_error message if I > detect > > Ninja as a CMAKE_GENERATOR. > > > > > > -- > > Alexey Samsonov, MSK > > > > _______________________________________________ > > LLVM Developers mailing list > > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > > >-- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140321/23fafe01/attachment.html>