Libguestfs - Sep 2018 - [PATCH v2 nbdkit 0/5] tests: Move common functions into tests/functions.sh

v1 was here:
https://www.redhat.com/archives/libguestfs/2018-September/msg00057.html

v2:

 - Fix tab vs spaces in configure.ac.

 - To generate list of plugins, use printf instead of xargs.

 - Use 'source ./functions.sh' instead of 'source functions'.

 - functions.sh: Consistent quoting in foreach_plugin function.

 - functions.sh: Change the contract of start_nbdkit so it requires
   (and checks that) the first two parameters are '-P pidfile'.  This
   avoids needing to iterate over the arguments.

 - functions.sh: Use {1..10} instead of seq.

 - functions.sh: Add a common mechanism for handling traps / cleanups
   (this is extra patch 4/5).

 - functions.sh: start_nbdkit manages its own cleanup function for
   killing nbdkit on exit

It's easier to get it to other places if it starts out in the
configure script.

Also split the list into language and non-language plugins.
---
 common-rules.mk | 44 ----------------------------------------
 configure.ac    | 53 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 53 insertions(+), 44 deletions(-)

diff --git a/common-rules.mk b/common-rules.mk
index 2c5609f..03e01e6 100644
--- a/common-rules.mk
+++ b/common-rules.mk
@@ -30,50 +30,6 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
-plugins = \
-	curl \
-	data \
-	example1 \
-	example2 \
-	example3 \
-	example4 \
-	ext2 \
-	file \
-	guestfs \
-	gzip \
-	libvirt \
-	lua \
-	memory \
-	nbd \
-	null \
-	ocaml \
-	pattern \
-	perl \
-	python \
-	random \
-	ruby \
-	sh \
-	split \
-	streaming \
-	tar \
-	tcl \
-	vddk \
-	xz \
-	zero
-
-filters = \
-	blocksize \
-	cache \
-	cow \
-	delay \
-	error \
-	fua \
-	log \
-	nozero \
-	offset \
-	partition \
-	truncate
-
 plugindir = $(libdir)/nbdkit/plugins
 filterdir = $(libdir)/nbdkit/filters
 
diff --git a/configure.ac b/configure.ac
index 4d340d4..ccb3262 100644
--- a/configure.ac
+++ b/configure.ac
@@ -542,6 +542,59 @@ AC_ARG_ENABLE([vddk],[
     [enable_vddk=yes])
 AM_CONDITIONAL([HAVE_VDDK], [test "x$enable_vddk" =
"xyes"])
 
+dnl List of plugins and filters.
+lang_plugins="\
+        lua \
+        ocaml \
+        perl \
+        python \
+        ruby \
+        sh \
+        tcl \
+        "
+non_lang_plugins="\
+        curl \
+        data \
+        example1 \
+        example2 \
+        example3 \
+        example4 \
+        ext2 \
+        file \
+        guestfs \
+        gzip \
+        libvirt \
+        memory \
+        nbd \
+        null \
+        pattern \
+        random \
+        split \
+        streaming \
+        tar \
+        vddk \
+        xz \
+        zero \
+        "
+plugins="$(echo $(printf %s\\n $lang_plugins $non_lang_plugins | sort
-u))"
+filters="\
+        blocksize \
+        cache \
+        cow \
+        delay \
+        error \
+        fua \
+        log \
+        nozero \
+        offset \
+        partition \
+        truncate \
+        "
+AC_SUBST([plugins])
+AC_SUBST([lang_plugins])
+AC_SUBST([non_lang_plugins])
+AC_SUBST([filters])
+
 dnl Produce output files.
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_FILES([nbdkit],
-- 
2.19.0.rc0

---
 docs/Makefile.am | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/docs/Makefile.am b/docs/Makefile.am
index ccf0a79..cbe7033 100644
--- a/docs/Makefile.am
+++ b/docs/Makefile.am
@@ -104,10 +104,11 @@ nbdkit-filter.3: nbdkit-filter.pod filter-links.pod
 
 endif HAVE_POD
 
-# XXX Language links are broken - they link to section 1 instead of section 3.
 plugin-links.pod: $(top_srcdir)/common-rules.mk
 	rm -f $@ $@-t
-	$(srcdir)/make-links.sh plugin 1 $(plugins) > $@-t
+	$(srcdir)/make-links.sh plugin 1 $(non_lang_plugins) > $@-t
+	echo \; >> $@-t
+	$(srcdir)/make-links.sh plugin 3 $(lang_plugins) >> $@-t
 	mv $@-t $@
 
 filter-links.pod: $(top_srcdir)/common-rules.mk
-- 
2.19.0.rc0

This resurrects the unused tests/functions.sh file (although now we
need to generate it from tests/functions.sh.in).  Put the common code
for running a test against every plugin here.

Because of the previous commits we can now use the plugins list
directly from configure.ac instead of needing to use weird shell
script, although we still need to preserve the test that the plugin
was built so that the tests continue to work if a plugin is disabled
or not built through missing dependencies.
---
 .gitignore                              |  1 +
 configure.ac                            |  1 +
 tests/Makefile.am                       |  1 -
 tests/{functions.sh => functions.sh.in} | 23 +++++++++++++++++++----
 tests/test-dump-plugin.sh               | 19 ++++++++-----------
 tests/test-help.sh                      | 18 ++++++++----------
 tests/test-version.sh                   | 18 ++++++++----------
 7 files changed, 45 insertions(+), 36 deletions(-)

diff --git a/.gitignore b/.gitignore
index a21484e..b8396f1 100644
--- a/.gitignore
+++ b/.gitignore
@@ -55,6 +55,7 @@ Makefile.in
 /tests/disk.xz
 /tests/ext2.img
 /tests/file-data
+/tests/functions.sh
 /tests/keys.psk
 /tests/offset-data
 /tests/partition-disk
diff --git a/configure.ac b/configure.ac
index ccb3262..54c4289 100644
--- a/configure.ac
+++ b/configure.ac
@@ -651,6 +651,7 @@ AC_CONFIG_FILES([Makefile
                  filters/truncate/Makefile
                  src/Makefile
                  src/nbdkit.pc
+                 tests/functions.sh
                  tests/Makefile])
 
 AC_OUTPUT
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a7d0c2d..23316ea 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -35,7 +35,6 @@ include $(top_srcdir)/common-rules.mk
 MAINTAINERCLEANFILES  
 EXTRA_DIST = \
-	functions.sh \
 	make-pki.sh \
 	make-psk.sh \
 	python-exception.py \
diff --git a/tests/functions.sh b/tests/functions.sh.in
similarity index 74%
rename from tests/functions.sh
rename to tests/functions.sh.in
index e54bb9d..62fdfe5 100644
--- a/tests/functions.sh
+++ b/tests/functions.sh.in
@@ -1,5 +1,7 @@
 # nbdkit
-# Copyright (C) 2017 Red Hat Inc.
+# Common functions used by the tests.
+# @configure_input@
+# Copyright (C) 2017-2018 Red Hat Inc.
 # All rights reserved.
 #
 # Redistribution and use in source and binary forms, with or without
@@ -30,6 +32,19 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
-# Some functions used by the tests.
-
-# (Currently empty)
+# foreach_plugin f
+#
+# For each plugin that was built, call test function f with the plugin
+# name as argument.
+foreach_plugin ()
+{
+    for p in @plugins@; do
+        # Was the plugin built?
+        d="@top_builddir@/plugins/$p"
+        if [ -f "$d/.libs/nbdkit-$p-plugin.so" ] ||
+           [ -f "$d/nbdkit-$p-plugin" ]; then
+            # Yes so run the test.
+            "$1" "$p"
+        fi
+    done
+}
diff --git a/tests/test-dump-plugin.sh b/tests/test-dump-plugin.sh
index e08db84..dc52742 100755
--- a/tests/test-dump-plugin.sh
+++ b/tests/test-dump-plugin.sh
@@ -31,9 +31,9 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
-source ./functions.sh
 
 # Basic check that the name field is present.
 output="$(nbdkit example1 --dump-plugin)"
@@ -51,15 +51,11 @@ if [[ ! ( "$output" =~ example2_extra\=hello ) ]];
then
     exit 1
 fi
 
-# Get a list of all plugins and run --dump-plugin on all of them.
+# Run nbdkit plugin --dump-plugin for each plugin.
 # However some of these tests are expected to fail.
-plugins="$(
-    cd ..;
-    ls -1 plugins/*/.libs/nbdkit-*-plugin.so plugins/*/nbdkit-*-plugin |
-        sed 's,^plugins/\([^/]*\)/.*,\1,'
-)"
-for p in $plugins; do
-    case "$p${NBDKIT_VALGRIND:+-valgrind}" in
+test ()
+{
+    case "$1${NBDKIT_VALGRIND:+-valgrind}" in
         vddk | vddk-valgrind)
             # VDDK won't run without special environment variables
             # being set, so ignore it.
@@ -70,7 +66,8 @@ for p in $plugins; do
             # Plugins written in scripting languages can't run under
valgrind.
             ;;
         *)
-            nbdkit $p --dump-plugin
+            nbdkit $1 --dump-plugin
             ;;
     esac
-done
+}
+foreach_plugin test
diff --git a/tests/test-help.sh b/tests/test-help.sh
index f1a988c..68e0bfc 100755
--- a/tests/test-help.sh
+++ b/tests/test-help.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 
 output="$(nbdkit --help)"
@@ -40,15 +41,11 @@ if [[ ! ( "$output" =~ dump-config ) ]]; then
     exit 1
 fi
 
-# Get a list of all plugins and run --help on all of them.
+# Run nbdkit plugin --help for each plugin.
 # However some of these tests are expected to fail.
-plugins="$(
-    cd ..;
-    ls -1 plugins/*/.libs/nbdkit-*-plugin.so plugins/*/nbdkit-*-plugin |
-        sed 's,^plugins/\([^/]*\)/.*,\1,'
-)"
-for p in $plugins; do
-    case "$p${NBDKIT_VALGRIND:+-valgrind}" in
+test ()
+{
+    case "$1${NBDKIT_VALGRIND:+-valgrind}" in
         vddk | vddk-valgrind)
             # VDDK won't run without special environment variables
             # being set, so ignore it.
@@ -59,7 +56,8 @@ for p in $plugins; do
             # Plugins written in scripting languages can't run under
valgrind.
             ;;
         *)
-            nbdkit $p --help
+            nbdkit $1 --help
             ;;
     esac
-done
+}
+foreach_plugin test
diff --git a/tests/test-version.sh b/tests/test-version.sh
index 39fe74a..869d88c 100755
--- a/tests/test-version.sh
+++ b/tests/test-version.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 
 output="$(nbdkit --version)"
@@ -40,15 +41,11 @@ if [[ ! ( "$output" =~ ^nbdkit\ 1\. ) ]]; then
     exit 1
 fi
 
-# Get a list of all plugins and run --version on all of them.
+# Run nbdkit plugin --version for each plugin.
 # However some of these tests are expected to fail.
-plugins="$(
-    cd ..;
-    ls -1 plugins/*/.libs/nbdkit-*-plugin.so plugins/*/nbdkit-*-plugin |
-        sed 's,^plugins/\([^/]*\)/.*,\1,'
-)"
-for p in $plugins; do
-    case "$p${NBDKIT_VALGRIND:+-valgrind}" in
+test ()
+{
+    case "$1${NBDKIT_VALGRIND:+-valgrind}" in
         vddk | vddk-valgrind)
             # VDDK won't run without special environment variables
             # being set, so ignore it.
@@ -59,7 +56,8 @@ for p in $plugins; do
             # Plugins written in scripting languages can't run under
valgrind.
             ;;
         *)
-            nbdkit $p --version
+            nbdkit $1 --version
             ;;
     esac
-done
+}
+foreach_plugin test
-- 
2.19.0.rc0

Thanks: Eric Blake for the suggestion here:
https://www.redhat.com/archives/libguestfs/2018-September/msg00069.html
---
 tests/functions.sh.in                  | 25 +++++++++++++++++++++++++
 tests/test-blocksize.sh                |  9 ++-------
 tests/test-cache.sh                    |  9 ++-------
 tests/test-cow.sh                      |  9 ++-------
 tests/test-data-7E.sh                  |  9 ++-------
 tests/test-data-base64.sh              |  9 ++-------
 tests/test-data-raw.sh                 |  9 ++-------
 tests/test-fua.sh                      |  9 ++-------
 tests/test-log.sh                      |  9 ++-------
 tests/test-memory-largest-for-qemu.sh  |  9 ++-------
 tests/test-memory-largest.sh           |  9 ++-------
 tests/test-nozero.sh                   |  9 ++-------
 tests/test-offset2.sh                  |  9 ++-------
 tests/test-parallel-file.sh            |  4 +++-
 tests/test-parallel-nbd.sh             |  4 +++-
 tests/test-pattern-largest-for-qemu.sh |  9 ++-------
 tests/test-pattern-largest.sh          |  9 ++-------
 tests/test-pattern.sh                  |  9 ++-------
 tests/test-single.sh                   | 10 ++--------
 tests/test-tls-psk.sh                  | 10 ++--------
 tests/test-tls.sh                      | 10 ++--------
 tests/test-truncate1.sh                |  9 ++-------
 tests/test-truncate2.sh                |  9 ++-------
 tests/test-zero.sh                     | 14 ++------------
 24 files changed, 73 insertions(+), 157 deletions(-)

diff --git a/tests/functions.sh.in b/tests/functions.sh.in
index 62fdfe5..42e3925 100644
--- a/tests/functions.sh.in
+++ b/tests/functions.sh.in
@@ -32,6 +32,31 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+# cleanup_fn f [args]
+#
+# A generic trap handling function.  This runs the function f + args
+# when the script exits for any reason.
+declare -a _cleanup_hook
+cleanup_fn ()
+{
+    _cleanup_hook[${#_cleanup_hook[@]}]="$@"
+}
+
+_run_cleanup_hooks ()
+{
+    status=$?
+    set +e
+    trap '' INT QUIT TERM EXIT ERR
+    echo $0: run cleanup hooks: exit code $status
+
+    for (( i = 0; i < ${#_cleanup_hook[@]}; ++i )); do
+        ${_cleanup_hook[i]}
+    done
+
+    exit $status
+}
+trap _run_cleanup_hooks INT QUIT TERM EXIT ERR
+
 # foreach_plugin f
 #
 # For each plugin that was built, call test function f with the plugin
diff --git a/tests/test-blocksize.sh b/tests/test-blocksize.sh
index cb9b09f..21b55b9 100755
--- a/tests/test-blocksize.sh
+++ b/tests/test-blocksize.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 
 files="blocksize1.img blocksize1.log blocksize1.sock blocksize1.pid
@@ -54,10 +55,6 @@ pid1= pid2 # Kill any nbdkit processes on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     test "$pid1" && kill $pid1
     test "$pid2" && kill $pid2
     # For easier debugging, dump the final log files before removing them.
@@ -66,10 +63,8 @@ cleanup ()
     echo "Log 2 file contents:"
     cat blocksize2.log || :
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Run two parallel nbdkit; to compare the logs and see what changes.
 nbdkit -P blocksize1.pid -U blocksize1.sock \
diff --git a/tests/test-cache.sh b/tests/test-cache.sh
index 215bab1..495469d 100755
--- a/tests/test-cache.sh
+++ b/tests/test-cache.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -60,16 +61,10 @@ pid="$(cat cache.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Open the overlay and perform some operations.
 guestfish --format=raw -a "nbd://?socket=$PWD/cache.sock"
<<'EOF'
diff --git a/tests/test-cow.sh b/tests/test-cow.sh
index f9b0649..b5c8f63 100755
--- a/tests/test-cow.sh
+++ b/tests/test-cow.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -61,16 +62,10 @@ pid="$(cat cow.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Write some data into the overlay.
 guestfish --format=raw -a "nbd://?socket=$PWD/cow.sock" -m /dev/sda1
<<EOF
diff --git a/tests/test-data-7E.sh b/tests/test-data-7E.sh
index 8327511..adccd86 100755
--- a/tests/test-data-7E.sh
+++ b/tests/test-data-7E.sh
@@ -34,6 +34,7 @@
 # Test the data plugin creating a 7 EB partitioned disk, and
 # the partition filter on top.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -104,16 +105,10 @@ pid="$(cat data-7E.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Since we're reading the empty first partition, any read returns zeroes.
 qemu-io -r -f raw 'nbd+unix://?socket=data-7E.sock' \
diff --git a/tests/test-data-base64.sh b/tests/test-data-base64.sh
index 6ff324b..72cf743 100755
--- a/tests/test-data-base64.sh
+++ b/tests/test-data-base64.sh
@@ -33,6 +33,7 @@
 
 # Test the data plugin with base64= parameter.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -72,16 +73,10 @@ pid="$(cat data-base64.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=data-base64.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
data-base64.out
diff --git a/tests/test-data-raw.sh b/tests/test-data-raw.sh
index c6652b7..00d4fb1 100755
--- a/tests/test-data-raw.sh
+++ b/tests/test-data-raw.sh
@@ -33,6 +33,7 @@
 
 # Test the data plugin with raw= parameter.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -66,16 +67,10 @@ pid="$(cat data-raw.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=data-raw.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
data-raw.out
diff --git a/tests/test-fua.sh b/tests/test-fua.sh
index f95aa18..f7c4571 100755
--- a/tests/test-fua.sh
+++ b/tests/test-fua.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -55,10 +56,6 @@ pid1= pid2= pid3= pid4 # Kill any nbdkit processes on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     test "$pid1" && kill $pid1
     test "$pid2" && kill $pid2
     test "$pid3" && kill $pid3
@@ -73,10 +70,8 @@ cleanup ()
     echo "Log 4 file contents:"
     cat fua4.log || :
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Run four parallel nbdkit; to compare the logs and see what changes.
 # 1: fuamode=none (default): client should send flush instead
diff --git a/tests/test-log.sh b/tests/test-log.sh
index f811de4..5ad5f22 100755
--- a/tests/test-log.sh
+++ b/tests/test-log.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 
 files="log.img log.log log.sock log.pid"
@@ -63,19 +64,13 @@ pid="$(cat log.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     # For easier debugging, dump the final log file before removing it.
     echo "Log file contents:"
     cat log.log
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Write, then read some data in the file.
 qemu-io -f raw -c 'w -P 11 1M 2M' 'nbd+unix://?socket=log.sock'
diff --git a/tests/test-memory-largest-for-qemu.sh
b/tests/test-memory-largest-for-qemu.sh
index 9bc498d..65f69b6 100755
--- a/tests/test-memory-largest-for-qemu.sh
+++ b/tests/test-memory-largest-for-qemu.sh
@@ -34,6 +34,7 @@
 # Test the memory plugin with the largest possible size supported
 # by qemu and nbdkit.
 
+source ./functions.sh
 set -e
 
 files="memory-largest-for-qemu.out memory-largest-for-qemu.pid
memory-largest-for-qemu.sock"
@@ -68,16 +69,10 @@ pid="$(cat memory-largest-for-qemu.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Write some stuff to the beginning, middle and end.
 qemu-io -f raw 'nbd+unix://?socket=memory-largest-for-qemu.sock' \
diff --git a/tests/test-memory-largest.sh b/tests/test-memory-largest.sh
index d0a6c77..d2cad1c 100755
--- a/tests/test-memory-largest.sh
+++ b/tests/test-memory-largest.sh
@@ -34,6 +34,7 @@
 # Test the memory plugin with the largest possible size supported
 # by nbdkit.
 
+source ./functions.sh
 set -e
 
 files="memory-largest.out memory-largest.pid memory-largest.sock"
@@ -69,16 +70,10 @@ pid="$(cat memory-largest.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # qemu cannot open this image!
 #
diff --git a/tests/test-nozero.sh b/tests/test-nozero.sh
index 904d822..239d4bd 100755
--- a/tests/test-nozero.sh
+++ b/tests/test-nozero.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 
 files="nozero1.img nozero1.log nozero1.sock nozero1.pid
@@ -65,10 +66,6 @@ pid1= pid2= pid3= pid4= pid5a= pid5b # Kill any nbdkit
processes on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     test "$pid1" && kill $pid1
     test "$pid2" && kill $pid2
     test "$pid3" && kill $pid3
@@ -89,10 +86,8 @@ cleanup ()
     echo "Log 5b file contents:"
     cat nozero5b.log || :
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Run four parallel nbdkit; to compare the logs and see what changes.
 # 1: unfiltered, to check that qemu-io sends ZERO request and plugin trims
diff --git a/tests/test-offset2.sh b/tests/test-offset2.sh
index c37e060..2376301 100755
--- a/tests/test-offset2.sh
+++ b/tests/test-offset2.sh
@@ -33,6 +33,7 @@
 
 # Additional test of the offset filter using the pattern plugin.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -69,16 +70,10 @@ pid="$(cat offset2.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=offset2.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
offset2.out
diff --git a/tests/test-parallel-file.sh b/tests/test-parallel-file.sh
index fd88faa..1d27f93 100755
--- a/tests/test-parallel-file.sh
+++ b/tests/test-parallel-file.sh
@@ -31,6 +31,8 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
+
 # Check file-data was created by Makefile and qemu-io exists.
 if ! test -f file-data; then
     echo "$0: missing file-data"
@@ -41,7 +43,7 @@ if ! qemu-io --version; then
     exit 77
 fi
 
-trap 'rm -f test-parallel-file.data test-parallel-file.out' 0 1 2 3 15
+cleanup_fn rm -f test-parallel-file.data test-parallel-file.out
 
 # Populate file, and sanity check that qemu-io can issue parallel requests
 printf '%1024s' . > test-parallel-file.data
diff --git a/tests/test-parallel-nbd.sh b/tests/test-parallel-nbd.sh
index cb70f46..7aa8a9f 100755
--- a/tests/test-parallel-nbd.sh
+++ b/tests/test-parallel-nbd.sh
@@ -31,6 +31,8 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
+
 # Check file-data was created by Makefile and qemu-io exists.
 if ! test -f file-data; then
     echo "$0: missing file-data"
@@ -47,7 +49,7 @@ fi
 
 files="test-parallel-nbd.out test-parallel-nbd.sock test-parallel-nbd.data
test-parallel-nbd.pid"
 rm -f $files
-trap "rm -f $files" INT QUIT TERM EXIT ERR
+cleanup_fn rm -f $files
 
 # Populate file, and sanity check that qemu-io can issue parallel requests
 printf '%1024s' . > test-parallel-nbd.data
diff --git a/tests/test-pattern-largest-for-qemu.sh
b/tests/test-pattern-largest-for-qemu.sh
index cf70354..c2fa324 100755
--- a/tests/test-pattern-largest-for-qemu.sh
+++ b/tests/test-pattern-largest-for-qemu.sh
@@ -34,6 +34,7 @@
 # Test the pattern plugin with the largest possible size supported
 # by qemu and nbdkit.
 
+source ./functions.sh
 set -e
 
 files="pattern-largest-for-qemu.out pattern-largest-for-qemu.pid
pattern-largest-for-qemu.sock"
@@ -67,16 +68,10 @@ pid="$(cat pattern-largest-for-qemu.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=pattern-largest-for-qemu.sock' \
         -c 'r -v 9223372036854774784 512' | grep -E
'^[[:xdigit:]]+:' > pattern-largest-for-qemu.out
diff --git a/tests/test-pattern-largest.sh b/tests/test-pattern-largest.sh
index 2a903bf..60a77b3 100755
--- a/tests/test-pattern-largest.sh
+++ b/tests/test-pattern-largest.sh
@@ -34,6 +34,7 @@
 # Test the pattern plugin with the largest possible size supported
 # by nbdkit.
 
+source ./functions.sh
 set -e
 
 files="pattern-largest.out pattern-largest.pid pattern-largest.sock"
@@ -67,16 +68,10 @@ pid="$(cat pattern-largest.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # qemu cannot open this image!
 #
diff --git a/tests/test-pattern.sh b/tests/test-pattern.sh
index 68022c9..98c5139 100755
--- a/tests/test-pattern.sh
+++ b/tests/test-pattern.sh
@@ -38,6 +38,7 @@
 # what read parameters we give it.  Hence these tests are rather
 # limited. (XXX)
 
+source ./functions.sh
 set -e
 
 files="pattern.out pattern.pid pattern.sock"
@@ -69,16 +70,10 @@ pid="$(cat pattern.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=pattern.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
pattern.out
diff --git a/tests/test-single.sh b/tests/test-single.sh
index 35017c8..7c7edfc 100755
--- a/tests/test-single.sh
+++ b/tests/test-single.sh
@@ -31,9 +31,9 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
-source ./functions.sh
 
 if ! socat -h; then
     echo "$0: 'socat' command not available"
@@ -56,16 +56,10 @@ pid=$!
 
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Wait for socat to start up and create the socket.
 for i in `seq 1 10`; do
diff --git a/tests/test-tls-psk.sh b/tests/test-tls-psk.sh
index 99c5945..ac59a4c 100755
--- a/tests/test-tls-psk.sh
+++ b/tests/test-tls-psk.sh
@@ -31,9 +31,9 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
-source ./functions.sh
 
 # Don't fail if certain commands aren't available.
 if ! ss --version; then
@@ -97,16 +97,10 @@ pid="$(cat tls-psk.pid)"
 # Kill the process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f tls-psk.pid tls-psk.out
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Run qemu-img against the server.
 LANG=C \
diff --git a/tests/test-tls.sh b/tests/test-tls.sh
index 71bd8a4..1ba262e 100755
--- a/tests/test-tls.sh
+++ b/tests/test-tls.sh
@@ -31,9 +31,9 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
-source ./functions.sh
 
 # Don't fail if certain commands aren't available.
 if ! ss --version; then
@@ -91,16 +91,10 @@ pid="$(cat tls.pid)"
 # Kill the process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f tls.pid tls.out
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 # Run qemu-img against the server.
 LANG=C \
diff --git a/tests/test-truncate1.sh b/tests/test-truncate1.sh
index 719f63b..c4f3b63 100755
--- a/tests/test-truncate1.sh
+++ b/tests/test-truncate1.sh
@@ -33,6 +33,7 @@
 
 # Test the truncate filter using the pattern plugin.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -68,16 +69,10 @@ pid="$(cat truncate1.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=truncate1.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
truncate1.out
diff --git a/tests/test-truncate2.sh b/tests/test-truncate2.sh
index 0295b96..8a04661 100755
--- a/tests/test-truncate2.sh
+++ b/tests/test-truncate2.sh
@@ -33,6 +33,7 @@
 
 # Test the truncate filter using the pattern plugin.
 
+source ./functions.sh
 set -e
 set -x
 
@@ -68,16 +69,10 @@ pid="$(cat truncate2.pid)"
 # Kill the nbdkit process on exit.
 cleanup ()
 {
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
     kill $pid
     rm -f $files
-
-    exit $status
 }
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn cleanup
 
 qemu-io -r -f raw 'nbd+unix://?socket=truncate2.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
truncate2.out
diff --git a/tests/test-zero.sh b/tests/test-zero.sh
index 2c5cece..636e735 100755
--- a/tests/test-zero.sh
+++ b/tests/test-zero.sh
@@ -31,6 +31,7 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -x
 set -e
 
@@ -42,18 +43,7 @@ fi
 
 files="test-zero.out"
 rm -f $files
-
-cleanup ()
-{
-    status=$?
-    trap '' INT QUIT TERM EXIT ERR
-    echo $0: cleanup: exit code $status
-
-    rm $files
-
-    exit $status
-}
-trap cleanup INT QUIT TERM EXIT ERR
+cleanup_fn rm $files
 
 nbdkit -U - zero --run 'qemu-img convert $nbd test-zero.out'
 
-- 
2.19.0.rc0

This assumes bashisms, but bash is required to run the tests.

This is mostly refactoring.  However the changes (simplifications) are
quite substantial:

 - Since the new start_nbdkit helper function cleans up nbdkit on
   exit, most scripts no longer need to deal with the pid or kill the
   pid in the cleanup function.

 - As a result, cleanup functions are radically simpler, and often
   disappear completely.

 - Also removed the comment "# The cleanup() function is called
   implicitly on exit" passim partly because the cleanup functions
   have mostly been removed and partly because it should be clear from
   looking at functions.sh.

There is one additional change: In the test-memory*.sh tests, nbdkit
used to run in the foreground, but that seems to be a consequence of
some left over debugging.  I changed it so they work like the other
tests.
---
 tests/functions.sh.in                  | 33 ++++++++++++++++
 tests/test-blocksize.sh                | 30 ++-------------
 tests/test-cache.sh                    | 27 +------------
 tests/test-cow.sh                      | 27 +------------
 tests/test-data-7E.sh                  | 27 +------------
 tests/test-data-base64.sh              | 27 +------------
 tests/test-data-raw.sh                 | 27 +------------
 tests/test-fua.sh                      | 52 +++++++-------------------
 tests/test-ip.sh                       | 23 ++----------
 tests/test-log.sh                      | 23 ++----------
 tests/test-memory-largest-for-qemu.sh  | 30 ++-------------
 tests/test-memory-largest.sh           | 31 ++-------------
 tests/test-nozero.sh                   | 48 ++++--------------------
 tests/test-offset2.sh                  | 27 +------------
 tests/test-parallel-nbd.sh             | 19 ++--------
 tests/test-pattern-largest-for-qemu.sh | 27 +------------
 tests/test-pattern-largest.sh          | 27 +------------
 tests/test-pattern.sh                  | 27 +------------
 tests/test-start.sh                    | 17 +--------
 tests/test-tls-psk.sh                  | 26 ++-----------
 tests/test-tls.sh                      | 25 +------------
 tests/test-truncate1.sh                | 27 +------------
 tests/test-truncate2.sh                | 27 +------------
 tests/test-zero.sh                     |  3 --
 24 files changed, 104 insertions(+), 553 deletions(-)

diff --git a/tests/functions.sh.in b/tests/functions.sh.in
index 42e3925..5c1f34c 100644
--- a/tests/functions.sh.in
+++ b/tests/functions.sh.in
@@ -57,6 +57,39 @@ _run_cleanup_hooks ()
 }
 trap _run_cleanup_hooks INT QUIT TERM EXIT ERR
 
+# start_nbdkit -P pidfile args...
+#
+# Run nbdkit with args and wait for it to start up.  If it fails to
+# start up, exit with an error message.  Also a cleanup handler is
+# installed automatically which kills nbdkit on exit.
+start_nbdkit ()
+{
+    # -P <pidfile> must be the first two parameters.
+    if [ "$1" != "-P" ]; then
+       echo "$0: start_nbdkit: -P <pidfile> option must be
first"
+       exit 1
+    fi
+    pidfile="$2"
+
+    # Run nbdkit.
+    nbdkit "$@"
+
+    # Wait for the pidfile to appear.
+    for i in {1..10}; do
+        if test -f "$pidfile"; then
+            break
+        fi
+        sleep 1
+    done
+    if ! test -f "$pidfile"; then
+        echo "$0: start_nbdkit: PID file $pidfile was not created"
+        exit 1
+    fi
+
+    # Kill nbdkit on exit.
+    cleanup_fn kill "$(cat $pidfile)"
+}
+
 # foreach_plugin f
 #
 # For each plugin that was built, call test function f with the plugin
diff --git a/tests/test-blocksize.sh b/tests/test-blocksize.sh
index 21b55b9..aa3cc13 100755
--- a/tests/test-blocksize.sh
+++ b/tests/test-blocksize.sh
@@ -50,14 +50,10 @@ if ! qemu-io -f raw -c 'r 0 1' -c 'w -z 1000
2000' \
 fi
 truncate -s 10M blocksize2.img
 
-pid1= pid2-
-# Kill any nbdkit processes on exit.
+# For easier debugging, dump the final log files before removing them
+# on exit.
 cleanup ()
 {
-    test "$pid1" && kill $pid1
-    test "$pid2" && kill $pid2
-    # For easier debugging, dump the final log files before removing them.
     echo "Log 1 file contents:"
     cat blocksize1.log || :
     echo "Log 2 file contents:"
@@ -67,28 +63,12 @@ cleanup ()
 cleanup_fn cleanup
 
 # Run two parallel nbdkit; to compare the logs and see what changes.
-nbdkit -P blocksize1.pid -U blocksize1.sock \
+start_nbdkit -P blocksize1.pid -U blocksize1.sock \
        --filter=log file logfile=blocksize1.log blocksize1.img
-nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \
+start_nbdkit -P blocksize2.pid -U blocksize2.sock --filter=blocksize \
        --filter=log file logfile=blocksize2.log blocksize2.img \
        minblock=1024 maxdata=512k maxlen=1M
 
-# We may have to wait a short time for the pid files to appear.
-for i in `seq 1 10`; do
-    if test -f blocksize1.pid && test -f blocksize2.pid; then
-        break
-    fi
-    sleep 1
-done
-
-pid1="$(cat blocksize1.pid)" || :
-pid2="$(cat blocksize2.pid)" || :
-
-if ! test -f blocksize1.pid || ! test -f blocksize2.pid; then
-    echo "$0: PID files were not created"
-    exit 1
-fi
-
 # Test behavior on short accesses.
 qemu-io -f raw -c 'r 1 1' -c 'w 10001 1' -c 'w -z 20001
1' \
          -c 'discard 30001 1'
'nbd+unix://?socket=blocksize1.sock'
@@ -147,5 +127,3 @@ if grep
'count=0x[0-9a-f]*\([1235679abdef]00\|[0-9a-f]\(.[^0]\|[^0].\)\) ' \
     exit 1;
 fi
 diff -u blocksize1.img blocksize2.img
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-cache.sh b/tests/test-cache.sh
index 495469d..7b1fccb 100755
--- a/tests/test-cache.sh
+++ b/tests/test-cache.sh
@@ -37,34 +37,13 @@ set -x
 
 files="cache.img cache.sock cache.pid"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Create an empty base image.
 truncate -s 1G cache.img
 
 # Run nbdkit with the caching filter.
-nbdkit -P cache.pid -U cache.sock --filter=cache file cache.img
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f cache.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f cache.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat cache.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
+start_nbdkit -P cache.pid -U cache.sock --filter=cache file cache.img
 
 # Open the overlay and perform some operations.
 guestfish --format=raw -a "nbd://?socket=$PWD/cache.sock"
<<'EOF'
@@ -82,5 +61,3 @@ guestfish --ro -a cache.img -m /dev/sda1 <<'EOF'
   cat /hello
   cat /large | cat >/dev/null
 EOF
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-cow.sh b/tests/test-cow.sh
index b5c8f63..09fd16e 100755
--- a/tests/test-cow.sh
+++ b/tests/test-cow.sh
@@ -37,35 +37,14 @@ set -x
 
 files="cow-base.img cow-diff.qcow2 cow.sock cow.pid"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Create a base image which is partitioned with an empty filesystem.
 guestfish -N cow-base.img=fs exit
 lastmod="$(stat -c "%y" cow-base.img)"
 
 # Run nbdkit with a COW overlay.
-nbdkit -P cow.pid -U cow.sock --filter=cow file cow-base.img
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f cow.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f cow.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat cow.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
+start_nbdkit -P cow.pid -U cow.sock --filter=cow file cow-base.img
 
 # Write some data into the overlay.
 guestfish --format=raw -a "nbd://?socket=$PWD/cow.sock" -m /dev/sda1
<<EOF
@@ -92,5 +71,3 @@ if qemu-img --version >/dev/null 2>&1; then
     # This checks the file we created exists.
     guestfish --ro -a cow-diff.qcow2 -m /dev/sda1 cat /hello
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-data-7E.sh b/tests/test-data-7E.sh
index adccd86..eae40de 100755
--- a/tests/test-data-7E.sh
+++ b/tests/test-data-7E.sh
@@ -40,6 +40,7 @@ set -x
 
 files="data-7E.out data-7E.pid data-7E.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -48,7 +49,7 @@ if ! qemu-io --help >/dev/null; then
 fi
 
 # Run nbdkit.
-nbdkit -P data-7E.pid -U data-7E.sock \
+start_nbdkit -P data-7E.pid -U data-7E.sock \
        --filter=partition \
        data size=7E partition=1 \
        data="
@@ -88,28 +89,6 @@ nbdkit -P data-7E.pid -U data-7E.sock \
           0x80 0 0 0 0x80 0 0 0  0x79 0x8a 0xd0 0x7e 0 0 0 0
    "
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f data-7E.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f data-7E.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat data-7E.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 # Since we're reading the empty first partition, any read returns zeroes.
 qemu-io -r -f raw 'nbd+unix://?socket=data-7E.sock' \
         -c 'r -v 498 16' | grep -E '^[[:xdigit:]]+:' >
data-7E.out
@@ -119,5 +98,3 @@ then
     cat data-7E.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-data-base64.sh b/tests/test-data-base64.sh
index 72cf743..cc53edf 100755
--- a/tests/test-data-base64.sh
+++ b/tests/test-data-base64.sh
@@ -39,6 +39,7 @@ set -x
 
 files="data-base64.out data-base64.pid data-base64.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test if the base64 parameter is supported in this build.
 if ! nbdkit data --dump-plugin | grep -sq "data_base64=yes"; then
@@ -53,31 +54,9 @@ if ! qemu-io --help >/dev/null; then
 fi
 
 # Run nbdkit.
-nbdkit -P data-base64.pid -U data-base64.sock \
+start_nbdkit -P data-base64.pid -U data-base64.sock \
        data base64=MTIz size=512
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f data-base64.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f data-base64.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat data-base64.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 qemu-io -r -f raw 'nbd+unix://?socket=data-base64.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
data-base64.out
 if [ "$(cat data-base64.out)" != "00000000:  31 32 33 00 00 00
00 00 00 00 00 00 00 00 00 00  123.............
@@ -117,5 +96,3 @@ then
     cat data-base64.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-data-raw.sh b/tests/test-data-raw.sh
index 00d4fb1..f4af2ec 100755
--- a/tests/test-data-raw.sh
+++ b/tests/test-data-raw.sh
@@ -39,6 +39,7 @@ set -x
 
 files="data-raw.out data-raw.pid data-raw.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -47,31 +48,9 @@ if ! qemu-io --help >/dev/null; then
 fi
 
 # Run nbdkit.
-nbdkit -P data-raw.pid -U data-raw.sock \
+start_nbdkit -P data-raw.pid -U data-raw.sock \
        data raw=123 size=512
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f data-raw.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f data-raw.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat data-raw.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 qemu-io -r -f raw 'nbd+unix://?socket=data-raw.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
data-raw.out
 if [ "$(cat data-raw.out)" != "00000000:  31 32 33 00 00 00 00
00 00 00 00 00 00 00 00 00  123.............
@@ -111,5 +90,3 @@ then
     cat data-raw.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-fua.sh b/tests/test-fua.sh
index f7c4571..20ba16d 100755
--- a/tests/test-fua.sh
+++ b/tests/test-fua.sh
@@ -51,16 +51,10 @@ if ! qemu-io -f raw -t none -c flush -c 'w -f -z 0
64k' fua.img; then
     exit 77
 fi
 
-pid1= pid2= pid3= pid4-
-# Kill any nbdkit processes on exit.
+# For easier debugging, dump the final log files before removing them
+# on exit.
 cleanup ()
 {
-    test "$pid1" && kill $pid1
-    test "$pid2" && kill $pid2
-    test "$pid3" && kill $pid3
-    test "$pid4" && kill $pid4
-    # For easier debugging, dump the final log files before removing them.
     echo "Log 1 file contents:"
     cat fua1.log || :
     echo "Log 2 file contents:"
@@ -78,34 +72,18 @@ cleanup_fn cleanup
 # 2: fuamode=emulate: log shows that blocksize optimizes fua to flush
 # 3: fuamode=native: log shows that blocksize preserves fua
 # 4: fuamode=force: log shows that fua is always enabled
-nbdkit -P fua1.pid -U fua1.sock --filter=log --filter=fua \
-       file logfile=fua1.log fua.img
-nbdkit -P fua2.pid -U fua2.sock --filter=blocksize --filter=log --filter=fua \
-       file logfile=fua2.log fua.img fuamode=emulate maxdata=4k maxlen=4k
-nbdkit -P fua3.pid -U fua3.sock --filter=blocksize --filter=log --filter=fua \
-       file logfile=fua3.log fua.img fuamode=native maxdata=4k maxlen=4k
-nbdkit -P fua4.pid -U fua4.sock --filter=fua --filter=log \
-       file logfile=fua4.log fua.img fuamode=force
-
-# We may have to wait a short time for the pid files to appear.
-for i in `seq 1 10`; do
-    if test -f fua1.pid && test -f fua2.pid && test -f fua3.pid
&&
-       test -f fua4.pid; then
-        break
-    fi
-    sleep 1
-done
-
-pid1="$(cat fua1.pid)" || :
-pid2="$(cat fua2.pid)" || :
-pid3="$(cat fua3.pid)" || :
-pid4="$(cat fua4.pid)" || :
-
-if ! test -f fua1.pid || ! test -f fua2.pid || ! test -f fua3.pid ||
-   ! test -f fua4.pid; then
-    echo "$0: PID files were not created"
-    exit 1
-fi
+start_nbdkit -P fua1.pid -U fua1.sock \
+             --filter=log --filter=fua \
+             file logfile=fua1.log fua.img
+start_nbdkit -P fua2.pid -U fua2.sock \
+             --filter=blocksize --filter=log --filter=fua \
+             file logfile=fua2.log fua.img fuamode=emulate maxdata=4k maxlen=4k
+start_nbdkit -P fua3.pid -U fua3.sock \
+             --filter=blocksize --filter=log --filter=fua \
+             file logfile=fua3.log fua.img fuamode=native maxdata=4k maxlen=4k
+start_nbdkit -P fua4.pid -U fua4.sock \
+             --filter=fua --filter=log \
+             file logfile=fua4.log fua.img fuamode=force
 
 # Perform a flush, write, and zero, first without then with FUA
 for f in '' -f; do
@@ -145,5 +123,3 @@ if grep 'Flush' fua4.log; then
     echo "filter should have elided flush"
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-ip.sh b/tests/test-ip.sh
index 12448c4..edf0b6a 100755
--- a/tests/test-ip.sh
+++ b/tests/test-ip.sh
@@ -34,10 +34,11 @@
 # Every other test uses a Unix domain socket.  This tests nbdkit over
 # IPv4 and IPv6 localhost connections.
 
-set -e
 source ./functions.sh
+set -e
 
 rm -f ip.pid ipv4.out ipv6.out
+cleanup_fn rm -f ip.pid ipv4.out ipv6.out
 
 # Don't fail if certain commands aren't available.
 if ! ss --version; then
@@ -65,20 +66,7 @@ echo picked unused port $port
 
 # By default nbdkit will listen on all available interfaces, ie.
 # IPv4 and IPv6.
-nbdkit -P ip.pid -p $port example1
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f ip.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f ip.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
+start_nbdkit -P ip.pid -p $port example1
 pid="$(cat ip.pid)"
 
 # Check the process exists.
@@ -99,8 +87,3 @@ if test -n "$ipv6_lo"; then
     cat ipv6.out
     grep -sq "^virtual size: 100M" ipv6.out
 fi
-
-# Kill the process.
-kill $pid
-rm ip.pid
-rm -f ipv4.out ipv6.out
diff --git a/tests/test-log.sh b/tests/test-log.sh
index 5ad5f22..f0eacb7 100755
--- a/tests/test-log.sh
+++ b/tests/test-log.sh
@@ -45,27 +45,12 @@ if ! qemu-io -f raw -c 'w 1M 2M' log.img; then
 fi
 
 # Run nbdkit with logging enabled to file.
-nbdkit -P log.pid -U log.sock --filter=log file log.img logfile=log.log
+start_nbdkit -P log.pid -U log.sock --filter=log file log.img logfile=log.log
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f log.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f log.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat log.pid)"
-
-# Kill the nbdkit process on exit.
+# For easier debugging, dump the final log files before removing them
+# on exit.
 cleanup ()
 {
-    kill $pid
-    # For easier debugging, dump the final log file before removing it.
     echo "Log file contents:"
     cat log.log
     rm -f $files
@@ -79,5 +64,3 @@ qemu-io -r -f raw -c 'r -P 11 2M 1M'
'nbd+unix://?socket=log.sock'
 # The log should show a write on connection 1, and read on connection 2.
 grep 'connection=1 Write id=1 offset=0x100000 count=0x200000 ' log.log
 grep 'connection=2 Read id=1 offset=0x200000 count=0x100000 ' log.log
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-memory-largest-for-qemu.sh
b/tests/test-memory-largest-for-qemu.sh
index 65f69b6..2b56f30 100755
--- a/tests/test-memory-largest-for-qemu.sh
+++ b/tests/test-memory-largest-for-qemu.sh
@@ -39,6 +39,7 @@ set -e
 
 files="memory-largest-for-qemu.out memory-largest-for-qemu.pid
memory-largest-for-qemu.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -48,31 +49,8 @@ fi
 
 # Run nbdkit with memory plugin.
 # size = (2^63-1) & ~511 which is the largest supported by qemu.
-nbdkit -f -v -D memory.dir=1 \
-       -P memory-largest-for-qemu.pid -U memory-largest-for-qemu.sock \
-       memory size=9223372036854775296 &
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f memory-largest-for-qemu.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f memory-largest-for-qemu.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat memory-largest-for-qemu.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
+start_nbdkit -P memory-largest-for-qemu.pid -U memory-largest-for-qemu.sock \
+       memory size=9223372036854775296
 
 # Write some stuff to the beginning, middle and end.
 qemu-io -f raw 'nbd+unix://?socket=memory-largest-for-qemu.sock' \
@@ -240,5 +218,3 @@ then
     cat memory-largest-for-qemu.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-memory-largest.sh b/tests/test-memory-largest.sh
index d2cad1c..a6c269e 100755
--- a/tests/test-memory-largest.sh
+++ b/tests/test-memory-largest.sh
@@ -39,6 +39,7 @@ set -e
 
 files="memory-largest.out memory-largest.pid memory-largest.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -48,32 +49,8 @@ fi
 
 # Run nbdkit with memory plugin.
 # size = 2^63-1
-nbdkit -f -v \
-       -D memory.dir=1 \
-       -P memory-largest.pid -U memory-largest.sock \
-       memory size=9223372036854775807 &
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f memory-largest.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f memory-largest.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat memory-largest.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
+start_nbdkit -P memory-largest.pid -U memory-largest.sock \
+       memory size=9223372036854775807
 
 # qemu cannot open this image!
 #
@@ -82,5 +59,3 @@ cleanup_fn cleanup
 # Therefore we skip the remainder of this test (in effect, testing
 # only that nbdkit can create the file).
 exit 77
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-nozero.sh b/tests/test-nozero.sh
index 239d4bd..c22363b 100755
--- a/tests/test-nozero.sh
+++ b/tests/test-nozero.sh
@@ -61,18 +61,10 @@ if test "$(stat -c %b nozero1.img)" = "$(stat
-c %b nozero2.img)"; then
 fi
 cp nozero2.img nozero1.img
 
-pid1= pid2= pid3= pid4= pid5a= pid5b-
-# Kill any nbdkit processes on exit.
+# For easier debugging, dump the final log files before removing them
+# on exit.
 cleanup ()
 {
-    test "$pid1" && kill $pid1
-    test "$pid2" && kill $pid2
-    test "$pid3" && kill $pid3
-    test "$pid4" && kill $pid4
-    test "$pid5a" && kill $pid5a
-    test "$pid5b" && kill $pid5b
-    # For easier debugging, dump the final log files before removing them.
     echo "Log 1 file contents:"
     cat nozero1.log || :
     echo "Log 2 file contents:"
@@ -96,41 +88,19 @@ cleanup_fn cleanup
 # 4: log after filter with zeromode=emulate, to ensure no ZERO to plugin
 # 5a/b: both sides of nbd plugin: even though server side does not advertise
 # ZERO, the client side still exposes it, and just skips calling nbd's
.zero
-nbdkit -P nozero1.pid -U nozero1.sock --filter=log \
+start_nbdkit -P nozero1.pid -U nozero1.sock --filter=log \
        file logfile=nozero1.log nozero1.img
-nbdkit -P nozero2.pid -U nozero2.sock --filter=log --filter=nozero \
+start_nbdkit -P nozero2.pid -U nozero2.sock --filter=log --filter=nozero \
        file logfile=nozero2.log nozero2.img
-nbdkit -P nozero3.pid -U nozero3.sock --filter=log --filter=nozero \
+start_nbdkit -P nozero3.pid -U nozero3.sock --filter=log --filter=nozero \
        file logfile=nozero3.log nozero3.img zeromode=emulate
-nbdkit -P nozero4.pid -U nozero4.sock --filter=nozero --filter=log \
+start_nbdkit -P nozero4.pid -U nozero4.sock --filter=nozero --filter=log \
        file logfile=nozero4.log nozero4.img zeromode=emulate
-nbdkit -P nozero5a.pid -U nozero5a.sock --filter=log --filter=nozero \
+start_nbdkit -P nozero5a.pid -U nozero5a.sock --filter=log --filter=nozero \
        file logfile=nozero5a.log nozero5.img
-nbdkit -P nozero5b.pid -U nozero5b.sock --filter=log \
+start_nbdkit -P nozero5b.pid -U nozero5b.sock --filter=log \
        nbd logfile=nozero5b.log socket=nozero5a.sock
 
-# We may have to wait a short time for the pid files to appear.
-for i in `seq 1 10`; do
-    if test -f nozero1.pid && test -f nozero2.pid && test -f
nozero3.pid &&
-       test -f nozero4.pid && test -f nozero5a.pid && test -f
nozero5b.pid; then
-        break
-    fi
-    sleep 1
-done
-
-pid1="$(cat nozero1.pid)" || :
-pid2="$(cat nozero2.pid)" || :
-pid3="$(cat nozero3.pid)" || :
-pid4="$(cat nozero4.pid)" || :
-pid5a="$(cat nozero5a.pid)" || :
-pid5b="$(cat nozero5b.pid)" || :
-
-if ! test -f nozero1.pid || ! test -f nozero2.pid || ! test -f nozero3.pid ||
-   ! test -f nozero4.pid || ! test -f nozero5a.pid || ! test -f nozero5b.pid;
then
-    echo "$0: PID files were not created"
-    exit 1
-fi
-
 # Perform the zero write.
 qemu-io -f raw -c 'w -z -u 0 1M'
'nbd+unix://?socket=nozero1.sock'
 qemu-io -f raw -c 'w -z -u 0 1M'
'nbd+unix://?socket=nozero2.sock'
@@ -160,5 +130,3 @@ cmp nozero1.img nozero2.img
 cmp nozero2.img nozero3.img
 cmp nozero3.img nozero4.img
 cmp nozero4.img nozero5.img
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-offset2.sh b/tests/test-offset2.sh
index 2376301..4e1ef2b 100755
--- a/tests/test-offset2.sh
+++ b/tests/test-offset2.sh
@@ -39,6 +39,7 @@ set -x
 
 files="offset2.out offset2.pid offset2.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -48,33 +49,11 @@ fi
 
 # Run nbdkit with pattern plugin and offset filter in front.
 # 8070450532247927809 = 7E - 1023
-nbdkit -P offset2.pid -U offset2.sock \
+start_nbdkit -P offset2.pid -U offset2.sock \
        --filter=offset \
        pattern size=7E \
        offset=8070450532247927809 range=512
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f offset2.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f offset2.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat offset2.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 qemu-io -r -f raw 'nbd+unix://?socket=offset2.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
offset2.out
 if [ "$(cat offset2.out)" != "00000000:  ff ff ff ff ff fc 00 6f
ff ff ff ff ff fc 08 6f  .......o.......o
@@ -114,5 +93,3 @@ then
     cat offset2.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-parallel-nbd.sh b/tests/test-parallel-nbd.sh
index 7aa8a9f..e4a2352 100755
--- a/tests/test-parallel-nbd.sh
+++ b/tests/test-parallel-nbd.sh
@@ -63,21 +63,10 @@ qemu-io -f raw -c "aio_write -P 1 0 512" -c
"aio_write -P 2 512 512" \
 # may have spurious failures under heavy loads on the test machine, where
 # tuning the delays may help.
 
-nbdkit --exit-with-parent -v \
-    -U test-parallel-nbd.sock -P test-parallel-nbd.pid \
-    --filter=delay \
-    file test-parallel-nbd.data wdelay=2 rdelay=1 &
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f test-parallel-nbd.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f test-parallel-nbd.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
+start_nbdkit -P test-parallel-nbd.pid \
+             -U test-parallel-nbd.sock \
+             --filter=delay \
+             file test-parallel-nbd.data wdelay=2 rdelay=1
 
 # With --threads=1, the write should complete first because it was issued first
 nbdkit -v -t 1 -U - nbd socket=test-parallel-nbd.sock --run '
diff --git a/tests/test-pattern-largest-for-qemu.sh
b/tests/test-pattern-largest-for-qemu.sh
index c2fa324..1883fe6 100755
--- a/tests/test-pattern-largest-for-qemu.sh
+++ b/tests/test-pattern-largest-for-qemu.sh
@@ -39,6 +39,7 @@ set -e
 
 files="pattern-largest-for-qemu.out pattern-largest-for-qemu.pid
pattern-largest-for-qemu.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -48,31 +49,9 @@ fi
 
 # Run nbdkit with pattern plugin.
 # size = (2^63-1) & ~511 which is the largest supported by qemu.
-nbdkit -P pattern-largest-for-qemu.pid -U pattern-largest-for-qemu.sock \
+start_nbdkit -P pattern-largest-for-qemu.pid -U pattern-largest-for-qemu.sock \
        pattern size=9223372036854775296
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f pattern-largest-for-qemu.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f pattern-largest-for-qemu.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat pattern-largest-for-qemu.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 qemu-io -r -f raw 'nbd+unix://?socket=pattern-largest-for-qemu.sock' \
         -c 'r -v 9223372036854774784 512' | grep -E
'^[[:xdigit:]]+:' > pattern-largest-for-qemu.out
 if [ "$(cat pattern-largest-for-qemu.out)" != "7ffffffffffffc00:
7f ff ff ff ff ff fc 00 7f ff ff ff ff ff fc 08  ................
@@ -112,5 +91,3 @@ then
     cat pattern-largest-for-qemu.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-pattern-largest.sh b/tests/test-pattern-largest.sh
index 60a77b3..de5e1d7 100755
--- a/tests/test-pattern-largest.sh
+++ b/tests/test-pattern-largest.sh
@@ -39,6 +39,7 @@ set -e
 
 files="pattern-largest.out pattern-largest.pid pattern-largest.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -48,31 +49,9 @@ fi
 
 # Run nbdkit with pattern plugin.
 # size = 2^63-1
-nbdkit -P pattern-largest.pid -U pattern-largest.sock \
+start_nbdkit -P pattern-largest.pid -U pattern-largest.sock \
        pattern size=9223372036854775807
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f pattern-largest.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f pattern-largest.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat pattern-largest.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 # qemu cannot open this image!
 #
 #   can't open device nbd+unix://?socket=pattern-largest.sock: Could not
get image size: File too large
@@ -92,5 +71,3 @@ then
     cat pattern-largest.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-pattern.sh b/tests/test-pattern.sh
index 98c5139..93dca94 100755
--- a/tests/test-pattern.sh
+++ b/tests/test-pattern.sh
@@ -43,6 +43,7 @@ set -e
 
 files="pattern.out pattern.pid pattern.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -51,29 +52,7 @@ if ! qemu-io --help >/dev/null; then
 fi
 
 # Run nbdkit with pattern plugin.
-nbdkit -P pattern.pid -U pattern.sock pattern size=1G
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f pattern.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f pattern.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat pattern.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
+start_nbdkit -P pattern.pid -U pattern.sock pattern size=1G
 
 qemu-io -r -f raw 'nbd+unix://?socket=pattern.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
pattern.out
@@ -114,5 +93,3 @@ then
     cat pattern.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-start.sh b/tests/test-start.sh
index dc578e7..4bea0ef 100755
--- a/tests/test-start.sh
+++ b/tests/test-start.sh
@@ -31,29 +31,16 @@
 # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
 # SUCH DAMAGE.
 
+source ./functions.sh
 set -e
 set -x
-source ./functions.sh
 
 # Test nbdkit starts up, forks in the background, writes a PID file,
 # and can be killed.
 
 rm -f start.pid start.sock
 
-nbdkit -P start.pid -U start.sock example1
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f start.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f start.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
+start_nbdkit -P start.pid -U start.sock example1
 pid="$(cat start.pid)"
 
 # Check the process exists.
diff --git a/tests/test-tls-psk.sh b/tests/test-tls-psk.sh
index ac59a4c..9e88782 100755
--- a/tests/test-tls-psk.sh
+++ b/tests/test-tls-psk.sh
@@ -78,29 +78,9 @@ for port in `seq 51000 65535`; do
 done
 echo picked unused port $port
 
-nbdkit -P tls-psk.pid -p $port -n --tls=require --tls-psk=keys.psk example1
-
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f tls-psk.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f tls-psk.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat tls-psk.pid)"
-
-# Kill the process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f tls-psk.pid tls-psk.out
-}
-cleanup_fn cleanup
+cleanup_fn rm -f tls-psk.pid tls-psk.out
+start_nbdkit -P tls-psk.pid -p $port -n \
+             --tls=require --tls-psk=keys.psk example1
 
 # Run qemu-img against the server.
 LANG=C \
diff --git a/tests/test-tls.sh b/tests/test-tls.sh
index 1ba262e..e41c1d5 100755
--- a/tests/test-tls.sh
+++ b/tests/test-tls.sh
@@ -71,31 +71,10 @@ for port in `seq 50000 65535`; do
 done
 echo picked unused port $port
 
-nbdkit -P tls.pid -p $port -n --tls=require \
+cleanup_fn rm -f tls.pid tls.out
+start_nbdkit -P tls.pid -p $port -n --tls=require \
        --tls-certificates="$pkidir" example1
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f tls.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f tls.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat tls.pid)"
-
-# Kill the process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f tls.pid tls.out
-}
-cleanup_fn cleanup
-
 # Run qemu-img against the server.
 LANG=C \
 qemu-img info \
diff --git a/tests/test-truncate1.sh b/tests/test-truncate1.sh
index c4f3b63..eb02a69 100755
--- a/tests/test-truncate1.sh
+++ b/tests/test-truncate1.sh
@@ -39,6 +39,7 @@ set -x
 
 files="truncate1.out truncate1.pid truncate1.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -47,33 +48,11 @@ if ! qemu-io --help >/dev/null; then
 fi
 
 # Run nbdkit with pattern plugin and truncate filter in front.
-nbdkit -P truncate1.pid -U truncate1.sock \
+start_nbdkit -P truncate1.pid -U truncate1.sock \
        --filter=truncate \
        pattern size=503 \
        truncate=512
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f truncate1.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f truncate1.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat truncate1.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 qemu-io -r -f raw 'nbd+unix://?socket=truncate1.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
truncate1.out
 if [ "$(cat truncate1.out)" != "00000000:  00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 08  ................
@@ -113,5 +92,3 @@ then
     cat truncate1.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-truncate2.sh b/tests/test-truncate2.sh
index 8a04661..4a3a89e 100755
--- a/tests/test-truncate2.sh
+++ b/tests/test-truncate2.sh
@@ -39,6 +39,7 @@ set -x
 
 files="truncate2.out truncate2.pid truncate2.sock"
 rm -f $files
+cleanup_fn rm -f $files
 
 # Test that qemu-io works
 if ! qemu-io --help >/dev/null; then
@@ -47,33 +48,11 @@ if ! qemu-io --help >/dev/null; then
 fi
 
 # Run nbdkit with pattern plugin and truncate filter in front.
-nbdkit -P truncate2.pid -U truncate2.sock \
+start_nbdkit -P truncate2.pid -U truncate2.sock \
        --filter=truncate \
        pattern size=503 \
        round-up=512
 
-# We may have to wait a short time for the pid file to appear.
-for i in `seq 1 10`; do
-    if test -f truncate2.pid; then
-        break
-    fi
-    sleep 1
-done
-if ! test -f truncate2.pid; then
-    echo "$0: PID file was not created"
-    exit 1
-fi
-
-pid="$(cat truncate2.pid)"
-
-# Kill the nbdkit process on exit.
-cleanup ()
-{
-    kill $pid
-    rm -f $files
-}
-cleanup_fn cleanup
-
 qemu-io -r -f raw 'nbd+unix://?socket=truncate2.sock' \
         -c 'r -v 0 512' | grep -E '^[[:xdigit:]]+:' >
truncate2.out
 if [ "$(cat truncate2.out)" != "00000000:  00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 08  ................
@@ -113,5 +92,3 @@ then
     cat truncate2.out
     exit 1
 fi
-
-# The cleanup() function is called implicitly on exit.
diff --git a/tests/test-zero.sh b/tests/test-zero.sh
index 636e735..3fba24d 100755
--- a/tests/test-zero.sh
+++ b/tests/test-zero.sh
@@ -50,6 +50,3 @@ nbdkit -U - zero --run 'qemu-img convert $nbd
test-zero.out'
 # Resulting file should be zero-sized.
 test -f test-zero.out
 ! test -s test-zero.out
-
-# Existing implicitly calls the trap function above to clean up.
-exit 0
-- 
2.19.0.rc0

On 9/13/18 11:09 AM, Richard W.M. Jones wrote:
> This resurrects the unused tests/functions.sh file (although now we
> need to generate it from tests/functions.sh.in).  Put the common code
> for running a test against every plugin here.
> 
> Because of the previous commits we can now use the plugins list
> directly from configure.ac instead of needing to use weird shell
> script, although we still need to preserve the test that the plugin
> was built so that the tests continue to work if a plugin is disabled
> or not built through missing dependencies.
> ---
> -# (Currently empty)
> +# foreach_plugin f
> +#
> +# For each plugin that was built, call test function f with the plugin
> +# name as argument.
Possible improvement - allow callers to pass parameters, as in:

foreach_plugin f arg1 arg2
> +foreach_plugin ()
> +{
f=$1; shift
> +    for p in @plugins@; do
> +        # Was the plugin built?
> +        d="@top_builddir@/plugins/$p"
> +        if [ -f "$d/.libs/nbdkit-$p-plugin.so" ] ||
> +           [ -f "$d/nbdkit-$p-plugin" ]; then
> +            # Yes so run the test.
> +            "$1" "$p"
"$f" "$p" "$@"

Up to you if you want that (if you even have a use where additional 
arguments would make sense); I'm also fine with this patch going in as-is.
> +++ b/tests/test-help.sh
> +test ()
> +{
> +    case "$1${NBDKIT_VALGRIND:+-valgrind}" in
>           vddk | vddk-valgrind)
>               # VDDK won't run without special environment variables
>               # being set, so ignore it.
Not for this patch, but is it worth making 'nbdkit vddk --help' display 
something useful even when env-vars are not set?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

On 9/13/18 11:09 AM, Richard W.M. Jones wrote:
> Thanks: Eric Blake for the suggestion here:
> https://www.redhat.com/archives/libguestfs/2018-September/msg00069.html
> ---
I'm glad the idea worked!

> +# cleanup_fn f [args]
> +#
> +# A generic trap handling function.  This runs the function f + args
Not just functions, but any command name...
> +# when the script exits for any reason.
> +declare -a _cleanup_hook
> +cleanup_fn ()
> +{
> +    _cleanup_hook[${#_cleanup_hook[@]}]="$@"
> +}
> +++ b/tests/test-zero.sh
> @@ -31,6 +31,7 @@
>   # OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
>   # SUCH DAMAGE.
>   
> +source ./functions.sh
>   set -x
>   set -e
>   
> @@ -42,18 +43,7 @@ fi
>   
>   files="test-zero.out"
>   rm -f $files
> -
> -cleanup ()
> -{
> -    status=$?
> -    trap '' INT QUIT TERM EXIT ERR
> -    echo $0: cleanup: exit code $status
> -
> -    rm $files
> -
> -    exit $status
> -}
> -trap cleanup INT QUIT TERM EXIT ERR
> +cleanup_fn rm $files
...as evidenced by this conversion.  Pre-existing, but you could fix it 
while here: why is this not using 'rm -f' like all the other cleanups?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org

On 9/13/18 11:09 AM, Richard W.M. Jones wrote:
> This assumes bashisms, but bash is required to run the tests.
> 
> This is mostly refactoring.  However the changes (simplifications) are
> quite substantial:
> 
>   - Since the new start_nbdkit helper function cleans up nbdkit on
>     exit, most scripts no longer need to deal with the pid or kill the
>     pid in the cleanup function.
> 
>   - As a result, cleanup functions are radically simpler, and often
>     disappear completely.
> 
>   - Also removed the comment "# The cleanup() function is called
>     implicitly on exit" passim partly because the cleanup functions
>     have mostly been removed and partly because it should be clear from
>     looking at functions.sh.
> 
> There is one additional change: In the test-memory*.sh tests, nbdkit
> used to run in the foreground, but that seems to be a consequence of
> some left over debugging.  I changed it so they work like the other
> tests.
> ---
> +# start_nbdkit -P pidfile args...
> +#
> +# Run nbdkit with args and wait for it to start up.  If it fails to
> +# start up, exit with an error message.  Also a cleanup handler is
> +# installed automatically which kills nbdkit on exit.
> +start_nbdkit ()
> +{
> +    # -P <pidfile> must be the first two parameters.
> +    if [ "$1" != "-P" ]; then
> +       echo "$0: start_nbdkit: -P <pidfile> option must be
first"
> +       exit 1
> +    fi
> +    pidfile="$2"
Although none of the tests passes a space within $2, it's better to 
consistently quote...
> +
> +    # Run nbdkit.
> +    nbdkit "$@"
> +
> +    # Wait for the pidfile to appear.
> +    for i in {1..10}; do
> +        if test -f "$pidfile"; then
> +            break
> +        fi
> +        sleep 1
> +    done
> +    if ! test -f "$pidfile"; then
> +        echo "$0: start_nbdkit: PID file $pidfile was not
created"
> +        exit 1
> +    fi
> +
> +    # Kill nbdkit on exit.
> +    cleanup_fn kill "$(cat $pidfile)"
...and thus this should be "$(cat "$pidfile")"
> +++ b/tests/test-ip.sh
> -if ! test -f ip.pid; then
> -    echo "$0: PID file was not created"
> -    exit 1
> -fi
> -
> +start_nbdkit -P ip.pid -p $port example1
>   pid="$(cat ip.pid)"
>   
>   # Check the process exists.
     kill -s 0 $pid

Hmm. Should the 'kill -s 0 $pid' be moved into the common code for all 
tests to benefit from (detecting early crashes where nbdkit started and 
created the pidfile, but then died)?  If so, you don't need $pid in this 
file any longer.
> +++ b/tests/test-start.sh
> -if ! test -f start.pid; then
> -    echo "$0: PID file was not created"
> -    exit 1
> -fi
> -
> +start_nbdkit -P start.pid -U start.sock example1
>   pid="$(cat start.pid)"
>   
>   # Check the process exists.
and another one.


The stuff I found is minor enough that I don't see a problem with you 
fixing and committing the series, rather than going through a v3.  And I 
love the overall diffstat :)

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org