Libguestfs - Sep 2015 - [PATCH] Add 'make installcheck' rule to test installed packages.

This is my proposed alternative to the complicated test framework
(https://www.redhat.com/archives/libguestfs/2015-August/msg00022.html).

In this patch, when we want to run tests on installed packages we just
copy in the installed binaries over the source directory, and run the
test suite as normal.  This is basically the same as the current (not
used) contrib/make-check-on-installed.pl script, but with a bunch of
fixes.

Rich.

Allow the tests to be run on the installed libguestfs.so library and
on installed programs (but not installed language bindings, yet).

To run the tests, you have to have a copy of the libguestfs source
tree with exactly the same version as the installed libguestfs.  You
have to build the libguestfs source tree.  Then you do:

  make installcheck

I investigated making automake 'make installcheck' work, but it's
basically broken (no surprise there).

This is derived from the old 'make-check-on-installed.pl'
script (which is also removed in this patch).
---
 .gitignore                         |   1 +
 Makefile.am                        |   6 ++
 configure.ac                       |   2 +
 contrib/make-check-on-installed.pl | 179 -------------------------------------
 installcheck.sh.in                 | 107 ++++++++++++++++++++++
 src/guestfs.pod                    |  14 +++
 6 files changed, 130 insertions(+), 179 deletions(-)
 delete mode 100755 contrib/make-check-on-installed.pl
 create mode 100644 installcheck.sh.in

diff --git a/.gitignore b/.gitignore
index a430f6e..c174ccb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -284,6 +284,7 @@ Makefile.in
 /inspector/test-xmllint.sh
 /inspector/virt-inspector
 /inspector/virt-inspector.1
+/installcheck.sh
 /install-sh
 /java/api
 /java/Bindtests.java
diff --git a/Makefile.am b/Makefile.am
index f4916a1..5597954 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -376,6 +376,10 @@ install:
 	    exit 1; \
 	fi
 
+# Test installed packages.
+installcheck-local: installcheck.sh
+	$(builddir)/installcheck.sh
+
 # Manual pages in top level directory.
 
 man_MANS = \
@@ -605,6 +609,8 @@ help:
 	@echo "make check-all                    Runs all 'check*'
rules."
 	@echo "make check-release                Runs 'check*' rules
required for release."
 	@echo
+	@echo "make installcheck                 Test installed libguestfs
packages."
+	@echo
 	@echo "make syntax-check -j1 -k          Check syntax and style problems
in the code."
 	@echo "make print-subdirs                Print subdirectories."
 	@echo
diff --git a/configure.ac b/configure.ac
index cc92e77..8ff6ce5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1704,6 +1704,8 @@
AC_CONFIG_FILES([appliance/libguestfs-make-fixed-appliance],
                 [chmod +x,-w appliance/libguestfs-make-fixed-appliance])
 AC_CONFIG_FILES([inspector/test-xmllint.sh],
                 [chmod +x,-w inspector/test-xmllint.sh])
+AC_CONFIG_FILES([installcheck.sh],
+                [chmod +x,-w installcheck.sh])
 AC_CONFIG_FILES([p2v/virt-p2v-make-disk],
                 [chmod +x,-w p2v/virt-p2v-make-disk])
 AC_CONFIG_FILES([p2v/virt-p2v-make-kickstart],
diff --git a/contrib/make-check-on-installed.pl
b/contrib/make-check-on-installed.pl
deleted file mode 100755
index 775be4d..0000000
--- a/contrib/make-check-on-installed.pl
+++ /dev/null
@@ -1,179 +0,0 @@
-#!/usr/bin/perl -w
-# libguestfs
-# Copyright (C) 2009-2015 Red Hat Inc.
-#
-# This program is free software; you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation; either version 2 of the License, or
-# (at your option) any later version.
-#
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with this program; if not, write to the Free Software
-# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
-
-# This script allows you to run the test suite ('make check' etc) on
-# an installed copy of libguestfs.  Currently only RPM installs are
-# supported, but adding support for dpkg would be relatively
-# straightforward.  It works by examining the installed packages, and
-# copying binaries (eg. '/usr/bin/guestfish') and libraries into the
-# correct place in the local directory.
-#
-# * You MUST have the full source tree unpacked locally.  Either
-#   use the same source tarball as the version you are testing, or
-#   check it out from git and 'git-reset' to the right version tag.
-#
-# * You MUST do a successful local build from source before using
-#   this script (ie. './autogen.sh && make').
-#
-# Run the script from the top builddir.  Usually:
-#
-#   ./contrib/make-check-on-installed.pl
-#
-# If the script runs successfully, then run the test suite as normal:
-#
-#   make check
-#
-# (other make check-* rules can also be used here).
-#
-# To switch back to running the test suite on the locally built
-# version, do:
-#
-#   make clean && make
-
-use strict;
-
-die "wrong directory -- read the file before running\n" unless -f
"BUGS";
-
-my $cmd;
-
-# Remove all libtool crappage.
-$cmd = "find -name 'lt-*' | grep -v '/tests/' | grep
'/.libs/lt-' | xargs -r rm";
-system ($cmd) == 0 or die "$cmd: failed\n";
-
-$cmd = "find -name 'lib*.so*' | grep -v '/tests/' | grep
'/.libs/lib' | xargs -r rm";
-system ($cmd) == 0 or die "$cmd: failed\n";
-
-# Map of installed file to local file.  Key is a regexp.
-# Remember that ONLY libraries and binaries need to be copied.
-my %mapping = (
-    '/bin/erl-guestfs$' => "erlang",
-    '/bin/libguestfs-test-tool$' => "test-tool",
-    '/bin/guestfish$' => "fish",
-    '/bin/guestmount$' => "fuse",
-    '/bin/virt-alignment-scan$' => "align",
-    '/bin/virt-cat$' => "cat",
-    '/bin/virt-copy-in$' => "fish",
-    '/bin/virt-copy-out$' => "fish",
-    '/bin/virt-df$' => "df",
-    '/bin/virt-diff$' => "diff",
-    '/bin/virt-edit$' => "edit",
-    '/bin/virt-filesystems$' => "cat",
-    '/bin/virt-format$' => "format",
-    '/bin/virt-inspector$' => "inspector",
-    '/bin/virt-list-filesystems$' => "tools",
-    '/bin/virt-list-partitions$' => "tools",
-    '/bin/virt-ls$' => "cat",
-    '/bin/virt-make-fs$' => "tools",
-    '/bin/virt-rescue$' => "rescue",
-    '/bin/virt-resize$' => "resize",
-    '/bin/virt-sparsify$' => "sparsify",
-    '/bin/virt-sysprep$' => "sysprep",
-    '/bin/virt-tar$' => "tools",
-    '/bin/virt-tar-in$' => "fish",
-    '/bin/virt-tar-out$' => "fish",
-    '/bin/virt-win-reg$' => "tools",
-
-    # Ignore this because the daemon is included in the appliance.
-    '/sbin/guestfsd$' => "IGNORE",
-
-    '/erlang/lib/libguestfs-.*/ebin/guestfs\.beam$' =>
"erlang",
-
-    '/girepository-1\.0/Guestfs-1\.0\.typelib$' =>
"gobject",
-    '/gir-1.0/Guestfs-1.0.gir$' => "gobject",
-
-    '/guestfs/supermin.d/.*' => "appliance/supermin.d",
-
-    '/java/libguestfs-.*\.jar$' => "java",
-
-    '/libguestfs\.so.*' => "src/.libs",
-    '/libguestfs_jni\.so.*' => "java/.libs",
-    '/libguestfs-gobject-1\.0\.so.*' => "gobject/.libs",
-
-    '/ocaml/.*\.cmi$' => "IGNORE",
-    '/ocaml/.*\.cmo$' => "ocaml",
-    '/ocaml/.*\.cmx$' => "ocaml",
-    '/ocaml/.*\.cma$' => "ocaml",
-    '/ocaml/.*\.cmxa$' => "ocaml",
-    '/ocaml/.*\.a$' => "ocaml",
-    '/ocaml/.*\.so$' => "ocaml",
-    '/ocaml/.*\.so.owner$' => "IGNORE",
-    '/ocaml/.*META$' => "IGNORE",
-    '/ocaml/.*/guestfs\.mli$' => "IGNORE",
-    '/ocaml/.*/guestfs\.ml$' => "IGNORE",
-
-    '/perl5/.*/Guestfs\.so$' =>
"perl/blib/arch/auto/Sys/Guestfs",
-    '/perl5/.*/Guestfs.pm$' =>
"perl/blib/lib/Sys/Guestfs.pm",
-    '/perl5/.*/Lib.pm$' =>
"perl/blib/lib/Sys/Guestfs/Lib.pm",
-
-    '/php/modules/guestfs_php\.so$' =>
"php/extension/modules",
-    '/php/modules/guestfs_php\.so$' =>
"php/extension/.libs",
-
-    '/python.*/libguestfsmod\.so$' => "python/.libs",
-    '/python.*/guestfs\.py' => "IGNORE",
-    '/python.*/guestfs\.pyc$' => "python/guestfs.pyc",
-    '/python.*/guestfs\.pyo$' => "python/guestfs.pyo",
-
-    '/ruby/.*/_guestfs\.so$' => "ruby/ext/guestfs",
-    '/ruby/.*/guestfs\.rb$' => "IGNORE",
-
-    '/share/doc/' => "IGNORE",
-    '/share/javadoc/' => "IGNORE",
-    '/share/locale/' => "IGNORE",
-    '/share/man/' => "IGNORE",
-
-    '^/etc/' => "IGNORE",
-    '/systemd/' => "IGNORE",
-    '/include/guestfs\.h$' => "IGNORE",
-    '/include/guestfs-gobject\.h$' => "IGNORE",
-    '/libguestfs\.pc$' => "IGNORE",
-);
-
-# Get list of installed files.
-$cmd = 'rpm -ql $(rpm -qa | grep -i guestf | grep -v debug) | sort';
-my @files;
-open CMD, "$cmd |" or die "$cmd: $!";
-while (<CMD>) {
-    chomp;
-    push @files, $_;
-}
-close CMD;
-
-# Now try to map (copy) installed files to the local equivalents.
-foreach my $file (@files) {
-    my $match = 0;
-    foreach my $regexp (keys %mapping) {
-        if ($file =~ m/$regexp/) {
-            my $dest = $mapping{$regexp};
-            if ($dest ne "IGNORE") {
-                # Make destination writable if it's a file.
-                chmod 0644, "$dest" if -f "$dest"
&& ! -w "$dest";
-
-                # Copy file to destination.
-                $cmd = "cp '$file' '$dest'";
-                system ($cmd) == 0 or die "$cmd: failed\n";
-                print "$file => $dest\n";
-            }
-            $match++;
-        }
-    }
-    if ($match == 0) {
-        if (! -d $file) {
-            warn "WARNING: file '$file' is unmatched\n"
-        }
-    }
-}
diff --git a/installcheck.sh.in b/installcheck.sh.in
new file mode 100644
index 0000000..1a7cb12
--- /dev/null
+++ b/installcheck.sh.in
@@ -0,0 +1,107 @@
+#!/bin/bash
+# @configure_input@
+#
+# (C) Copyright 2015 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+
+# Run 'make check' on installed packages.
+#
+# The version of installed libguestfs being tested, and the version of
+# the libguestfs source tree must be the same.
+
+unset CDPATH
+export LANG=C
+set -e
+set -x
+
+# Grrrrrrr autoconf.
+prefix=@prefix@
+exec_prefix=@exec_prefix@
+
+# Remove all libtool crappage.
+find -name 'lt-*' | grep '/.libs/lt-' | xargs -r rm
+
+# Copy the installed library into libtool directory.
+rm src/.libs/libguestfs.so*
+cp @libdir@/libguestfs.so* src/.libs/
+
+# Copy installed binaries into the right places.
+cp @bindir@/libguestfs-test-tool  test-tool/
+cp @bindir@/guestfish             fish/
+cp @bindir@/guestmount            fuse/
+cp @bindir@/virt-alignment-scan   align/
+cp @bindir@/virt-builder          builder/
+cp @bindir@/virt-cat              cat/
+cp @bindir@/virt-copy-in          fish/
+cp @bindir@/virt-copy-out         fish/
+cp @bindir@/virt-customize        customize/
+cp @bindir@/virt-dib              dib/
+cp @bindir@/virt-diff             diff/
+cp @bindir@/virt-df               df/
+cp @bindir@/virt-edit             edit/
+cp @bindir@/virt-filesystems      cat/
+cp @bindir@/virt-format           format/
+cp @bindir@/virt-get-kernel       get-kernel/
+cp @bindir@/virt-inspector        inspector/
+cp @bindir@/virt-ls               cat/
+cp @bindir@/virt-make-fs          make-fs/
+cp @libexecdir@/virt-p2v          p2v/
+cp @bindir@/virt-rescue           rescue/
+cp @bindir@/virt-resize           resize/
+cp @bindir@/virt-sparsify         sparsify/
+cp @bindir@/virt-sysprep          sysprep/
+cp @bindir@/virt-tar-in           fish/
+cp @bindir@/virt-tar-out          fish/
+cp @bindir@/virt-v2v              v2v/
+cp @bindir@/virt-win-reg          tools/
+
+# virt-list-filesystems, virt-list-partitions and virt-tar are not
+# tested, because they are not routinely installed by Linux distros
+# (being legacy programs).
+
+# XXX No language bindings are copied at the moment.
+
+# Copy the installed appliance.
+rm -rf appliance/supermin.d
+cp -r @libdir@/guestfs/supermin.d appliance/
+# Try to force the appliance not to get rebuilt:
+touch appliance/stamp-supermin
+rm -rf "tmp/.guestfs-$(id -u)"
+
+# Run the tests.
+make check
+
+# Check the library and some critical binaries didn't get rebuilt
+# during the 'make check', which would invalidate the results of
+# the test.
+compare () {
+    sum1=`md5sum $1 | @AWK@ '{print $1}'`
+    sum2=`md5sum $2 | @AWK@ '{print $1}'`
+    if [ "$sum1" != "$sum2" ]; then
+        echo "$2 was overwritten during the test.  Test results are
invalid."
+        exit 1
+    fi
+}
+compare @libdir@/libguestfs.so src/.libs/libguestfs.so
+compare @bindir@/guestfish  fish/guestfish
+compare @bindir@/guestmount fuse/guestmount
+compare @bindir@/virt-df    df/virt-df
+compare @bindir@/virt-v2v   v2v/virt-v2v
+compare @libdir@/guestfs/supermin.d/daemon.tar.gz \
+                            appliance/supermin.d/daemon.tar.gz
+
+# Now do a make clean to remove all the above.
+make clean >/dev/null 2>&1 ||:
diff --git a/src/guestfs.pod b/src/guestfs.pod
index b189fd2..9ec7bbc 100644
--- a/src/guestfs.pod
+++ b/src/guestfs.pod
@@ -4334,6 +4334,20 @@ check-slow
 
 =back
 
+=item C<make installcheck>
+
+Run C<make check> on the installed copy of libguestfs.
+
+The version of installed libguestfs being tested, and the version of
+the libguestfs source tree must be the same.
+
+Do:
+
+ ./autogen.sh
+ make clean ||:
+ make
+ make installcheck
+
 =back
 
 =head2 DAEMON CUSTOM PRINTF FORMATTERS
-- 
2.5.0