search for: localtests_scripts

For v6: This is mainly just a rebase, but I have also added tests in the ocaml/ language bindings directory, and for all the OCaml-written virt tools. Rich.

v7: The only changes since v6 are those suggested by Pino in the review of v5.

I should probably start by saying this patch series isn't ready for review yet. This patch series adds a test harness to libguestfs. It allows us to run the tests outside the ordinary 'make check' path in the build tree. In particular, you can use this to run tests when libguestfs has been installed. 'make check' and the other 'make check-*' rules still work. The

This converts more of the tests (basically everything under tests/) to use the test harness, revealing some problems which have subsequently been fixed. Rich.

Since v4: - More tests have been converted. - Testing local guests fixed. - Drop no-exec-stack test.

Since v2: - Add perl tests. - Reworked and fixed the tests for virt-builder. - Some further minor bug fixes.

This is a more complete patch to add the test harness. The only parts missing now are the language bindings (except OCaml). The language bindings need a bit more thought. At the moment most language binding tests are done through some sort of shell script like perl/run-perl-tests which either runs each test itself or uses some language-specific machinary to run each test. The problem with that

Since v3: - A large number of fixes, especially for running the tests on installed libguestfs. - Fixed EXTRA_DIST rules throughout. - Extra patch 17/17 which is a tidy-up of the generated XML listing guests. Rich.

On Sunday 05 October 2014 14:08:35 Richard W.M. Jones wrote: > +# Install the test harness. > +localtestsdir = $(alltestsdir) > +localtests_SCRIPTS = ../test-harness Shouldn't it rather go to libexec? > +../test-harness: > + rm -f $@ $@-t > + echo 'echo Warning: Install OCaml compiler in order to rebuild the test-harness.' > $@-t > + chmod +x $@-t > + mv $@-t $@ > + > endif I gues...

On Tue, Oct 21, 2014 at 05:56:14PM +0200, Pino Toscano wrote: > On Sunday 05 October 2014 14:08:35 Richard W.M. Jones wrote: > > +# Install the test harness. > > +localtestsdir = $(alltestsdir) > > +localtests_SCRIPTS = ../test-harness > > Shouldn't it rather go to libexec? No, it's meant to be run by the end user. > > +../test-harness: > > + rm -f $@ $@-t > > + echo 'echo Warning: Install OCaml compiler in order to rebuild the test-harness.' > $@-t >...

...exit 1; - } - - next FILES; - } - - die "Didn't find GNU_STACK entry"; -} diff --git a/tests/regressions/tests.mk b/tests/regressions/tests.mk index 4fa0414..3175d56 100644 --- a/tests/regressions/tests.mk +++ b/tests/regressions/tests.mk @@ -44,8 +44,7 @@ localtests_SCRIPTS = \ rhbz895904.sh \ rhbz909624.sh \ rhbz957772.sh \ - rhbz975797.sh \ - test-noexec-stack.pl + rhbz975797.sh localtests_PROGRAMS = \ rhbz1055452 \ -- 2.0.4

...exit 1; - } - - next FILES; - } - - die "Didn't find GNU_STACK entry"; -} diff --git a/tests/regressions/tests.mk b/tests/regressions/tests.mk index d582b91..4d9217b 100644 --- a/tests/regressions/tests.mk +++ b/tests/regressions/tests.mk @@ -46,8 +46,7 @@ localtests_SCRIPTS = \ rhbz895904.sh \ rhbz909624.sh \ rhbz957772.sh \ - rhbz975797.sh \ - test-noexec-stack.pl + rhbz975797.sh localtests_PROGRAMS = \ rhbz1055452 \ -- 2.5.0

...ness_objects = \ + types.cmo \ + utils.cmo \ + tests.cmo \ + test_harness.cmo + EXTRA_DIST = $(sources) files-generated.txt OCAMLCFLAGS = -warn-error CDEFLMPSUVYZX-3 -I $(srcdir) -I . -package unix,str noinst_PROGRAM = generator +# Install the test harness. +localtestsdir = $(alltestsdir) +localtests_SCRIPTS = ../test-harness + if HAVE_OCAML generator: $(objects) $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -linkpkg $^ -o $@ +../test-harness: $(test_harness_objects) + $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -linkpkg $^ -o $@ + # Dependencies. %.cmi: %.mli $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -c $< -o $@...

...test_harness_objects = \ + types.cmo \ + utils.cmo \ + tests.cmo \ + test_harness.cmo + EXTRA_DIST = $(sources) files-generated.txt OCAMLCFLAGS = $(OCAML_WARN_ERROR) -I $(srcdir) -I . -package unix,str noinst_PROGRAM = generator +# Install the test harness. +localtestsdir = $(alltestsdir) +localtests_SCRIPTS = ../test-harness + if HAVE_OCAML generator: $(objects) $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -linkpkg $^ -o $@ +../test-harness: $(test_harness_objects) + $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -linkpkg $^ -o $@ + # Dependencies. %.cmi: %.mli $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -c $< -o $@...

This has got to the stage where it actually works, both for running the tests in-tree and installed. The 'test-harness' script has become rather over-complex in the process however. Rich.

...ness_objects = \ + types.cmo \ + utils.cmo \ + tests.cmo \ + test_harness.cmo + EXTRA_DIST = $(sources) files-generated.txt OCAMLCFLAGS = -warn-error CDEFLMPSUVYZX-3 -I $(srcdir) -I . -package unix,str noinst_PROGRAM = generator +# Install the test harness. +localtestsdir = $(alltestsdir) +localtests_SCRIPTS = ../test-harness + if HAVE_OCAML generator: $(objects) $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -linkpkg $^ -o $@ +../test-harness: $(test_harness_objects) + $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -linkpkg $^ -o $@ + # Dependencies. %.cmi: %.mli $(OCAMLFIND) ocamlc $(OCAMLCFLAGS) -c $< -o $@...

These are my thoughts on adding a test harness to run tests instead of using automake. The aim of this exercise is to allow us to run the full test suite on an installed copy of libguestfs. Another aim is to allow us to work around all the limitations and problems of automake. The first patch makes an observation that since the ./run script sets up $PATH to contain all the directories