llvm dev - Jun 2018 - Unit Tests CMake configuration

Hi llvm-dev,

Does anybody know why we're not using the `add_test` feature in CMake
[0] for unit tests? In particular, compiler-rt (sanitizers and xray)
has a number of unit tests which could really just be built as normal
binaries and invoked appropriately.

If we're avoiding ctest [1], then for the unit tests in compiler-rt
I'd like to see whether just using normal binaries can make the build
faster. The 'Generating TEST_OBJECTS.*' messages seem to indicate that
these are executed sequentially, or cannot be parallelized by
ninja-build [2] and take a really long time to complete.

Thoughts?

[0] https://cmake.org/cmake/help/v3.4/command/add_test.html
[1] https://cmake.org/cmake/help/v3.4/manual/ctest.1.html#manual:ctest(1)
[2] https://ninja-build.org/

-- 
Dean

On Tue, Jun 5, 2018 at 6:02 PM Dean Michael Berris
<dean.berris at gmail.com> wrote:
>
> Hi llvm-dev,
>
> Does anybody know why we're not using the `add_test` feature in CMake
> [0] for unit tests? In particular, compiler-rt (sanitizers and xray)
> has a number of unit tests which could really just be built as normal
> binaries and invoked appropriately.
>
> If we're avoiding ctest [1], then for the unit tests in compiler-rt
> I'd like to see whether just using normal binaries can make the build
> faster. The 'Generating TEST_OBJECTS.*' messages seem to indicate
that
> these are executed sequentially, or cannot be parallelized by
> ninja-build [2] and take a really long time to complete.
>
Okay, it's not sequential -- but it seems like the sanitizer builds
are taking a long time.

Anybody aware of changes recently that would cause the sanitizer
builds to take longer? Maybe changes in the compiler?

Adding in Kostya and Evgeniy.

In particular, I'm seeing really bad build times with
msan-instrumented unit tests when doing in-tree builds.

-- 
Dean

2018-06-05 3:02 GMT-05:00 Dean Michael Berris via llvm-dev
<llvm-dev at lists.llvm.org>:
> Does anybody know why we're not using the `add_test` feature in CMake
> [0] for unit tests? In particular, compiler-rt (sanitizers and xray)
> has a number of unit tests which could really just be built as normal
> binaries and invoked appropriately.
I assume this is mostly historical when LLVM was using GNU autotools,
i.e. there was no ctest back then, and LLVM introduced llvm-lit
(2009-09-08; could also have used dejagnu). The first CMakeLists.txt
was introduced in 2008-09-21, but the autotools build was still
present until 2016.

Michael

On Wed, Jun 6, 2018 at 5:26 AM Michael Kruse <llvmdev at meinersbur.de>
wrote:
>
> 2018-06-05 3:02 GMT-05:00 Dean Michael Berris via llvm-dev
> <llvm-dev at lists.llvm.org>:
> > Does anybody know why we're not using the `add_test` feature in
CMake
> > [0] for unit tests? In particular, compiler-rt (sanitizers and xray)
> > has a number of unit tests which could really just be built as normal
> > binaries and invoked appropriately.
>
> I assume this is mostly historical when LLVM was using GNU autotools,
> i.e. there was no ctest back then, and LLVM introduced llvm-lit
> (2009-09-08; could also have used dejagnu). The first CMakeLists.txt
> was introduced in 2008-09-21, but the autotools build was still
> present until 2016.
>
Thanks, that's interesting.

Any thoughts from others on whether it's worth changing this?

-- 
Dean