Hi Brad,
I have a few questions regarding ExternalProject_Add. For me it doesn't
really work as expected.
I add the following code to the tools/clang/runtime/CMakeLists.txt to
configure compiler-rt as external project:
ExternalProject_Add(compiler-rt
#DEPENDS clang clang++ llvm-config
PREFIX ${CMAKE_BINARY_DIR}/projects/compiler-rt
SOURCE_DIR ${COMPILER_RT_SRC_ROOT}
CMAKE_ARGS -DCMAKE_C_COMPILER=${LLVM_RUNTIME_OUTPUT_INTDIR}/clang
-DCMAKE_CXX_COMPILER=${LLVM_RUNTIME_OUTPUT_INTDIR}/clang++
-DCMAKE_BUILD_TYPE=Release
-DLLVM_CONFIG_PATH=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-config
-DCOMPILER_RT_OUTPUT_DIR=${LLVM_LIBRARY_OUTPUT_INTDIR}/clang/${CLANG_VERSION}
-DCOMPILER_RT_INSTALL_PATH=lib${LLVM_LIBDIR_SUFFIX}/clang/${CLANG_VERSION}
# -DCOMPILER_RT_INCLUDE_TESTS=ON
INSTALL_COMMAND ""
)
add_dependencies(compiler-rt clang clang++ llvm-config)
1) Looks like "DEPENDS" option is just broken - docs here (
http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html)
state that you can pass CMake targets there, but if I uncomment that line,
I get errors like:
CMake Error at /usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:720
(message):
External project "clang" has no stamp_dir
Call Stack (most recent call first):
/usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:932
(ExternalProject_Get_Property)
/usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:1488
(_ep_get_step_stampfile)
/usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:1702
(_ep_add_configure_command)
tools/clang/runtime/CMakeLists.txt:18 (ExternalProject_Add)
as if "clang" was supposed to be another external project. FTR, I use
cmake
2.8.10.2.
2) The dependencies don't act as expected: if I run "make
compiler-rt", it
builds Clang, uses it to configure
compiler-rt and builds compiler-rt. But if I then change the Clang sources,
and re-run "make compiler-rt",
Clang is re-built, but compiler-rt is *not* re-configured or re-built,
while I definitely want this.
3) The same doc (
http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html)
states that
"One drawback, however, is ExternalProject’s lack of full dependency
analysis. Changes in header files of an external project may
not cause an incremental rebuild of the affected sources that depend on
those headers."
Looks like even if I modify *sources* under projects/compiler-rt, and
re-run "make compiler-rt" from the build
directory, it doesn't re-build the compiler-rt libraries. This makes
incremental development simply impossible.
What am I doing wrong?
On Fri, Feb 7, 2014 at 1:59 AM, Brad King <brad.king at kitware.com>
wrote:
> On 02/06/2014 01:02 PM, Alexey Samsonov wrote:
> > OK, I feel like I need to learn more about ExternalProject_Add magic.
>
> It is the CMake equivalent to a configure+make build target that
> runs configure+make for another project.
>
> > I have a few quick questions for people with knowledge - currently:
> > 1) "compiler-rt"'s CMake needs to know about targets in
the Clang build
> tree ("clang") and
> > in LLVM build tree ("FileCheck", llvm-* command-line tools,
googletest
> etc.), and uses common
> > macro from LLVM's CMake modules like
"configute_lit_site_cfg",
> "parse_arguments".
>
> This will be possible once LLVM and Clang are taught to provide proper
> content in LLVMConfig.cmake and ClangConfig.cmake package configuration
> files. The patch series I proposed in the thread Greg linked at the
> start of this thread is the first step to do that.
>
> > 2) top-level targets from compiler-rt's CMake files are visible at
the
> root of LLVM's build tree,
> > so that I can run "make check-asan" or even "make
clang_rt.asan-x86_64"
> from the root of
> > the build tree.
>
> This would not work with ExternalProject_Add because the CMake
> running on LLVM or Clang would not see the targets in compiler-rt
> since the project will not even be processed until build (make) time.
> In return the compiler-rt would be able to build using the just-built
> Clang because it will now be available when CMake runs on compiler-rt.
> (IIUC this is currently the case for configure+make.) Also compiler-rt
> could now be built outside of a LLVM/Clang that was built unaware of
> compiler-rt.
>
> It is also possible to make project/compiler-rt build optionally
> with add_subdirectory instead of ExternalProject_Add, as it does now,
> with the cost of not using the just-built Clang.
>
--
Alexey Samsonov, MSK
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20140221/e77c08c8/attachment.html>
Hm, this thread: http://cmake.3232098.n2.nabble.com/ExternalProject-Add-Automatic-Incremental-Rebuilds-Add-Sources-to-IDE-td7585544.html suggests that ExternalProject is not good at all for tracking source file changes, and suggests to use add_subdirectory approach and pass information around in variables. Looks like we just can't express rules like "use this just-built compiler to build those libraries" in terms of CMake *sigh* We can try to configure new build trees with CMake manually, though, similar to the way it's done in add_tablegen() macro. On Fri, Feb 21, 2014 at 6:49 PM, Alexey Samsonov <samsonov at google.com>wrote:> Hi Brad, > > I have a few questions regarding ExternalProject_Add. For me it doesn't > really work as expected. > I add the following code to the tools/clang/runtime/CMakeLists.txt to > configure compiler-rt as external project: > > ExternalProject_Add(compiler-rt > > #DEPENDS clang clang++ llvm-config > > PREFIX ${CMAKE_BINARY_DIR}/projects/compiler-rt > SOURCE_DIR ${COMPILER_RT_SRC_ROOT} > > CMAKE_ARGS -DCMAKE_C_COMPILER=${LLVM_RUNTIME_OUTPUT_INTDIR}/clang > > -DCMAKE_CXX_COMPILER=${LLVM_RUNTIME_OUTPUT_INTDIR}/clang++ > > -DCMAKE_BUILD_TYPE=Release > > > -DLLVM_CONFIG_PATH=${LLVM_RUNTIME_OUTPUT_INTDIR}/llvm-config > > > -DCOMPILER_RT_OUTPUT_DIR=${LLVM_LIBRARY_OUTPUT_INTDIR}/clang/${CLANG_VERSION} > > > -DCOMPILER_RT_INSTALL_PATH=lib${LLVM_LIBDIR_SUFFIX}/clang/${CLANG_VERSION} > > # -DCOMPILER_RT_INCLUDE_TESTS=ON > > INSTALL_COMMAND "" > > ) > add_dependencies(compiler-rt clang clang++ llvm-config) > > 1) Looks like "DEPENDS" option is just broken - docs here ( > http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html > ) > state that you can pass CMake targets there, but if I uncomment that line, > I get errors like: > > CMake Error at > /usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:720 (message): > External project "clang" has no stamp_dir > Call Stack (most recent call first): > /usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:932 > (ExternalProject_Get_Property) > /usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:1488 > (_ep_get_step_stampfile) > /usr/local/share/cmake-2.8/Modules/ExternalProject.cmake:1702 > (_ep_add_configure_command) > tools/clang/runtime/CMakeLists.txt:18 (ExternalProject_Add) > > as if "clang" was supposed to be another external project. FTR, I use > cmake 2.8.10.2. > > 2) The dependencies don't act as expected: if I run "make compiler-rt", it > builds Clang, uses it to configure > compiler-rt and builds compiler-rt. But if I then change the Clang > sources, and re-run "make compiler-rt", > Clang is re-built, but compiler-rt is *not* re-configured or re-built, > while I definitely want this. > > 3) The same doc ( > http://www.kitware.com/media/html/BuildingExternalProjectsWithCMake2.8.html) > states that > "One drawback, however, is ExternalProject’s lack of full dependency > analysis. Changes in header files of an external project may > not cause an incremental rebuild of the affected sources that depend on > those headers." > Looks like even if I modify *sources* under projects/compiler-rt, and > re-run "make compiler-rt" from the build > directory, it doesn't re-build the compiler-rt libraries. This makes > incremental development simply impossible. > > What am I doing wrong? > > On Fri, Feb 7, 2014 at 1:59 AM, Brad King <brad.king at kitware.com> wrote: > >> On 02/06/2014 01:02 PM, Alexey Samsonov wrote: >> > OK, I feel like I need to learn more about ExternalProject_Add magic. >> >> It is the CMake equivalent to a configure+make build target that >> runs configure+make for another project. >> >> > I have a few quick questions for people with knowledge - currently: >> > 1) "compiler-rt"'s CMake needs to know about targets in the Clang build >> tree ("clang") and >> > in LLVM build tree ("FileCheck", llvm-* command-line tools, googletest >> etc.), and uses common >> > macro from LLVM's CMake modules like "configute_lit_site_cfg", >> "parse_arguments". >> >> This will be possible once LLVM and Clang are taught to provide proper >> content in LLVMConfig.cmake and ClangConfig.cmake package configuration >> files. The patch series I proposed in the thread Greg linked at the >> start of this thread is the first step to do that. >> >> > 2) top-level targets from compiler-rt's CMake files are visible at the >> root of LLVM's build tree, >> > so that I can run "make check-asan" or even "make clang_rt.asan-x86_64" >> from the root of >> > the build tree. >> >> This would not work with ExternalProject_Add because the CMake >> running on LLVM or Clang would not see the targets in compiler-rt >> since the project will not even be processed until build (make) time. >> In return the compiler-rt would be able to build using the just-built >> Clang because it will now be available when CMake runs on compiler-rt. >> (IIUC this is currently the case for configure+make.) Also compiler-rt >> could now be built outside of a LLVM/Clang that was built unaware of >> compiler-rt. >> >> It is also possible to make project/compiler-rt build optionally >> with add_subdirectory instead of ExternalProject_Add, as it does now, >> with the cost of not using the just-built Clang. >> > > -- > Alexey Samsonov, MSK >-- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140221/3ad3a1fd/attachment.html>
On 02/21/2014 09:49 AM, Alexey Samsonov wrote:> 1) Looks like "DEPENDS" option is just broken > I use cmake 2.8.10.2.Dependencies on normal targets were accidentally broken in 2.8.9 and later fixed in 2.8.11 (along with a test case to keep it working): http://www.cmake.org/Bug/view.php?id=13849 http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c0f1af92> if I then change the Clang sources, and re-run "make compiler-rt", > Clang is re-built, but compiler-rt is *not* re-configured or re-built[snip]> even if I modify *sources* under projects/compiler-rt > it doesn't re-build the compiler-rt libraries.ExternalProject was originally designed for two purposes: * Build third-party dependences on which local development will not be done * Manage a "superbuild" project that has no binaries of its own but has a bunch of ExternalProject_Add calls to drive set up of a bunch of build trees. Local development then occurs in each tree manually. Using ExternalProject for a recursive-make-like structure was only later proposed as a use case. In order to make the build step always run you need to make it depend on an extra step that is always out of date: ExternalProject_Add_Step(compiler-rt always-build COMMENT "Always building MyCOnly..." DEPENDERS build ALWAYS 1 ) -Brad
On 02/21/2014 10:31 AM, Brad King wrote:> Using ExternalProject for a recursive-make-like structure was > only later proposed as a use case. In order to make the build > step always run you need to make it depend on an extra step that > is always out of date: > > ExternalProject_Add_Step(compiler-rt always-build > COMMENT "Always building MyCOnly..."s/MyCOnly/compiler-rt/> DEPENDERS build > ALWAYS 1 > )In order to simplify this in the future, I've looked at adding a BUILD_ALWAYS option to ExternalProject_Add: ExternalProject: Add option to always run the build step http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=73e5c6ae This would allow one to request that the external build step run on every build of the main project without adding an extra step manually. -Brad
On Fri, Feb 21, 2014 at 7:31 PM, Brad King <brad.king at kitware.com> wrote:> On 02/21/2014 09:49 AM, Alexey Samsonov wrote: > > 1) Looks like "DEPENDS" option is just broken > > I use cmake 2.8.10.2. > > Dependencies on normal targets were accidentally broken in 2.8.9 and > later fixed in 2.8.11 (along with a test case to keep it working): > > http://www.cmake.org/Bug/view.php?id=13849 > http://cmake.org/gitweb?p=cmake.git;a=commitdiff;h=c0f1af92 > > > if I then change the Clang sources, and re-run "make compiler-rt", > > Clang is re-built, but compiler-rt is *not* re-configured or re-built > [snip] > > even if I modify *sources* under projects/compiler-rt > > it doesn't re-build the compiler-rt libraries. > > ExternalProject was originally designed for two purposes: > > * Build third-party dependences on which local development will > not be done > > * Manage a "superbuild" project that has no binaries of its own > but has a bunch of ExternalProject_Add calls to drive set up > of a bunch of build trees. Local development then occurs in > each tree manually. > > Using ExternalProject for a recursive-make-like structure was > only later proposed as a use case. In order to make the build > step always run you need to make it depend on an extra step that > is always out of date: > > ExternalProject_Add_Step(compiler-rt always-build > COMMENT "Always building MyCOnly..." > DEPENDERS build > ALWAYS 1 > ) >Great, thanks for suggestion! We can even use "DEPENDERS configure" instead of "DEPENDERS build" here. And we'd better "clobber" the compiler-rt build tree if any of the dependencies change (we need to rebuild every compiler-rt source from scratch if clang has changed). I will also try to add support for running tests in compiler-rt build tree. I think this can be achieved by smth. like: add_custom_target(check-compiler-rt COMMAND ${CMAKE_BUILD_TOOL} check-all DEPENDS compiler-rt WORKING_DIRECTORY ${COMPILER_RT_BUILD_DIR})> -Brad > >-- Alexey Samsonov, MSK -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140221/319edf9d/attachment.html>