In a recent conversation with Chris it was brought up that the LLVM
project doesn't have a way to incorporate fine-grained unit tests (in
other words, tests that exersize the members of a single C++ class). I
was wondering what is the general feeling about adding this to LLVM.
Let me discuss some of the details:
First, there is the question of a test framework. One possibility is to
create a custom framework based on the support classes already defined
within LLVM; This has the benefit of not adding an extra dependency,
minimizes potential license incompatibilities, and may perhaps lead to a
more integrated testing experience. The downside is that writing a
decent framework is a lot of work, and you aren't going to get something
as full-featured or mature as some of the various open-source C++
testing frameworks out there.
The other alternative is to adopt one of those frameworks. Based on my
research, I would go with either UnitTest++ or GoogleTest, as these are
the most modern and supported test frameworks available. (There are many
others, but the two I mentioned were written with the full knowledge of
all of the others and with the benefit of hindsight.)
I've been using GoogleTest (http://code.google.com/p/googletest) in my
own project and I am pretty happy with it. Tests are easy to write
without a lot of boilerplate, and it supports automatic test discovery,
test fixtures, customized assertion predicates, customizable
stringification of application-specific types, debugger integration, and
so on. I use the svn 'external' feature to incorporate a link to
GoogleTest, so that it gets checked out automatically as a result of
checking out my project's sources. I didn't bother with trying to get
autoconf's embedded project support to work, I just included the source
files directly in my own library's makefile and that worked fine.
A third possibility is to write an LLVM-specific framework that is
effectively clone of one of the existing frameworks, but using LLVM's
infrastructure as a basis. Its unlikely that LLVM will need more than
50% of the features of the open-source solutions, so it would not be
necessary to clone the entire framework, merely a simplified version of it.
In any case, I would imagine that a new directory, 'unit' be added to
the top level of the project, and appropriate makefile targets written
so that 'make check' (or whatever target name you like) would build and
run the unit tests. Only the executables defined in the 'unit' directory
would have any dependency on the test framework.
It may be that the existing test infrastructure is adequate for LLVM's
needs. However, one of my interests is in improving various container
classes in ADT, and the current test infrastructure doesn't serve that
use case very well IMHO.
-- Talin