Libguestfs - Oct 2016 - [PATCH 0/2] builder: Add test that a serial console is set up for guest templates we supply.

The first of two sets of patches.

This checks that the templates we supply have the serial console
enabled, or in two cases, that the serial console can be enabled by
fiddling with some grub config.

The second set of patches (to follow) will test that the --password
and --root-password options work.  This implementation of this is
quite complex.

Rich.

On debian-8 and ubuntu-16.04.
---
 builder/website/index     | 13 +++++++++++++
 builder/website/index.asc | 42 +++++++++++++++++++++++++++---------------
 2 files changed, 40 insertions(+), 15 deletions(-)

diff --git a/builder/website/index b/builder/website/index
index 8fca86b..ee2fdf5 100644
--- a/builder/website/index
+++ b/builder/website/index
@@ -238,6 +238,12 @@ notes=Debian 8 (Jessie).
  This image does not contain SSH host keys.  To regenerate them use:
  
  --firstboot-command "dpkg-reconfigure openssh-server"
+ 
+ The serial console is not working in this image.  To enable it, do:
+ 
+ --edit '/etc/default/grub:
+           
s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=tty0
console=ttyS0,115200n8"/' \
+ --run-command update-grub
 
 [fedora-18]
 name=Fedora® 18
@@ -872,6 +878,13 @@ notes=Ubuntu 16.04 (Xenial).
  
  --firstboot-command "dpkg-reconfigure openssh-server"
  
+ The serial console is not working in this image (apparently a bug
+ in Ubuntu).  To enable it, do:
+ 
+ --edit '/etc/default/grub:
+           
s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=tty0
console=ttyS0,115200n8"/' \
+ --run-command update-grub
+ 
  IMPORTANT NOTE: It seems to be impossible to create an Ubuntu >= 14.04
  image using preseed without creating a user account.  Therefore this
  image contains a user account 'builder'.  I have disabled it, so that
diff --git a/builder/website/index.asc b/builder/website/index.asc
index 1e11416..12a0e9b 100644
--- a/builder/website/index.asc
+++ b/builder/website/index.asc
@@ -241,6 +241,12 @@ notes=Debian 8 (Jessie).
  This image does not contain SSH host keys.  To regenerate them use:
  
  --firstboot-command "dpkg-reconfigure openssh-server"
+ 
+ The serial console is not working in this image.  To enable it, do:
+ 
+ --edit '/etc/default/grub:
+           
s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=tty0
console=ttyS0,115200n8"/' \
+ --run-command update-grub
 
 [fedora-18]
 name=Fedora® 18
@@ -875,25 +881,31 @@ notes=Ubuntu 16.04 (Xenial).
  
  --firstboot-command "dpkg-reconfigure openssh-server"
  
+ The serial console is not working in this image (apparently a bug
+ in Ubuntu).  To enable it, do:
+ 
+ --edit '/etc/default/grub:
+           
s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=tty0
console=ttyS0,115200n8"/' \
+ --run-command update-grub
+ 
  IMPORTANT NOTE: It seems to be impossible to create an Ubuntu >= 14.04
  image using preseed without creating a user account.  Therefore this
  image contains a user account 'builder'.  I have disabled it, so that
  people who don't read release notes don't get caught out, but you
might
  still wish to delete it completely.
 -----BEGIN PGP SIGNATURE-----
-Version: GnuPG v1
-
-iQIcBAEBAgAGBQJXhizFAAoJEJFzj3Pht2igUIQQAKga2A4gL9wAaJbYCb6AS3+7
-ngWN5coZdbu2FUj1I1xd10Ozjzid25UzdmzEZmSs1menwMeUyUJZtI2SXW8MWAor
-KvkFr9iPWVfLXPMzPTrPZZRTWkryae3JFthcGWW1pHErZU6Icv/Vz0oNDKm7ZQoo
-3fgYNM/19mVSskIrcqMTsKCY3DJaokRt7LfWuEL8gf47+KQQ4qaRQ+iuqqU4/sFx
-pFbCHBhYqnnNDUCQI6FwiOCegR8Tsf36BvXCWYMb5FltE0IggcrwraRdCa67DpjJ
-3D2W+Wvt4sk7TMWJCz6ZiO8GdAqR9+WGEeKxQQLRxX8k8fagZHw/+tdMkLGuHhPr
-Ys2yofR+5vCvA9RDMqObolvmFvyUaUmjH3UFMZM3P8N0mAzlAuBOIFKq2RWhcVll
-+EUc0B7UmrusMe76czAqOpltAlrIOJXfxnvTremZixRA2svWMntSuU52fe+TxWuj
-aZVbJdwvDfQ0yrHKneoP/v9/EXUCfgwvjHTIbZEG8b8TI4VbF8//9K+JEjJz8zlF
-QeqvWwBu9AlDF0RnSTNEFNDvqbfgs5E26ZGg/cDQv+x8UePkiETuAfH8E5ePIp0f
-U6NYmlxVvL4sjE6mU4KmEVuYgL12QbaRJCCFYiP6/XuZ4MnoD/w+ko5as2W7OpaR
-LrXbWO32TZL8n6RHFlog
-=HADO
+
+iQIcBAEBAgAGBQJYB3n6AAoJEJFzj3Pht2igIZ8P/iGhQYq1ZwbNvzbrrXR9JOmx
+SX/Q2BH02BEXHzNTCbryaMX3DrwM8ZYaMLITygFBCB/c5Ks3r7K873GbRJHVsBkl
+QY+MHM5uVijLOcsMws6/BB6y8d/Ar4MRB1LoLkIe1OeSUrDbFs+kHAU9WLjxngfj
+TNOtfHwFuwhAQ6CkD5NjI+Bj+qpFM5zS74gF0OFgIcfSWpPLG9r5vLbTqh/ul2MW
+lBX3ZK220qCqbM4r1LgasILcRJr2cNOZapGuZkhwfYC2ZPCYUo+YI4+b7kp2eiLR
+eYcjeuTb9UHFEXdTCQlGXlwS7jrZbkp9b60fcZV/eNKiwcb5X4AG3Rjs7+479EJ5
+4HwXssHEfhZhKvvT15+UW/7cj/rasWz9sVVFPnU6tdmA2YkbSUdlniW0bKQb/v7F
+YCwMfTdcz/DwS0qe67XdDuvqYu19iVzjCzWMGq/dsRXEcFVmtdhEPp7f8ayNRt8n
+vL5oZpIHVhQu6NDl0xkWGtG4AYE7EkuaPTzAjXd1AN7o3vQGeY1HHLnwiG6Sejio
+fFJNPGK0cJnoUioyBZGeLR/BNHdQs3NVFjAesrAt+EapsKTJGpYNbK5pg49PLrxk
+ehSELePpXDjdOGNS91He9RoCs6a82jXdMFR6ut+lWdonodexC1mo7cNM+vhT83KG
+hR9AJsYYRcxhKuTBIYGR
+=Cz5r
 -----END PGP SIGNATURE-----
-- 
2.9.3

---
 .gitignore              |   1 +
 builder/Makefile.am     |  36 ++++++++++-
 builder/test-console.sh | 164 ++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 200 insertions(+), 1 deletion(-)
 create mode 100755 builder/test-console.sh

diff --git a/.gitignore b/.gitignore
index a4be4ae..4b618f0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -95,6 +95,7 @@ Makefile.in
 /builder/stamp-virt-builder.pod
 /builder/stamp-virt-index-validate.pod
 /builder/test-config/virt-builder/repos.d/test-index.conf
+/builder/test-console-*.sh
 /builder/test-simplestreams/virt-builder/repos.d/cirros.conf
 /builder/test-website/virt-builder/repos.d/libguestfs.conf
 /builder/virt-builder
diff --git a/builder/Makefile.am b/builder/Makefile.am
index 55d97bc..b1ccd49 100644
--- a/builder/Makefile.am
+++ b/builder/Makefile.am
@@ -23,6 +23,7 @@ EXTRA_DIST = \
 	$(SOURCES_MLI) $(SOURCES_ML) $(SOURCES_C) \
 	libguestfs.gpg \
 	opensuse.gpg \
+	test-console.sh \
 	test-index \
 	test-simplestreams/streams/v1/index.json \
 	test-simplestreams/streams/v1/net.cirros-cloud_released_download.json \
@@ -253,11 +254,44 @@ endif
 check-valgrind:
 	$(MAKE) VG="@VG@" check
 
-SLOW_TESTS = test-virt-builder-planner.sh
+# Slow tests.
+
+SLOW_TESTS = \
+	$(console_test_scripts) \
+	test-virt-builder-planner.sh
 
 check-slow:
 	$(MAKE) check TESTS="$(SLOW_TESTS)" SLOW=1
 
+# Test that the supplied guests boot with a serial console.
+#
+# Note that in future we might decide to make the serial console a
+# feature, eg. `virt-builder --add-serial-console' or `virt-builder
+# --remove-serial-console', so don't assume that having these tests
+# means that a serial console is a requirement.
+console_test_scripts := \
+	test-console-centos-7.2.sh \
+	test-console-rhel-6.8.sh \
+	test-console-rhel-7.2.sh \
+	test-console-debian-7.sh \
+	test-console-debian-8.sh \
+	test-console-fedora-24.sh \
+	test-console-ubuntu-12.04.sh \
+	test-console-ubuntu-14.04.sh \
+	test-console-ubuntu-16.04.sh
+
+test-console-%.sh:
+	rm -f $@ $@-t
+	f=`echo "$@" | $(SED) 's/test-console-\(.*\).sh/\1/'`; \
+	echo 'script=$@ exec $$srcdir/test-console.sh' "$$f" >
$@-t
+	chmod 0755 $@-t
+	mv $@-t $@
+
+CLEANFILES += \
+	$(console_test_scripts) \
+	console-*.img \
+	console-*.out
+
 # Dependencies.
 depend: .depend
 
diff --git a/builder/test-console.sh b/builder/test-console.sh
new file mode 100755
index 0000000..209fd21
--- /dev/null
+++ b/builder/test-console.sh
@@ -0,0 +1,164 @@
+#!/bin/bash -
+# Test serial console is present in templates.
+# Copyright (C) 2016 Red Hat Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# This slow test checks that the serial console works.
+#
+# NB. 'test-console.sh' runs the tests, but the various tests are
+# run via the 'test-console-GUESTNAME.sh' wrappers.
+#
+# The script currently assumes a Linux guest.  We should test Windows,
+# FreeBSD in future (XXX).
+
+export LANG=C
+set -e
+
+if [ -z "$SLOW" ]; then
+    echo "$script: use 'make check-slow' to run this test"
+    exit 77
+fi
+
+skipenv="$(echo SKIP_$script | tr a-z.- A-Z__)"
+if [ -n "${!skipenv}" ]; then
+    echo "$script: test skipped because environment variable is set."
+    exit 77
+fi
+
+guestname="$1"
+if [ -z "$guestname" ]; then
+    echo "$script: guestname parameter not set, don't run this test
directly."
+    exit 1
+fi
+
+# If the guest doesn't exist in virt-builder, skip.  This is because
+# we test some RHEL guests which most users won't have access to.
+if ! virt-builder -l "$guestname" >/dev/null 2>&1; then
+    echo "$script: test skipped because \"$guestname\" not known
to virt-builder."
+    exit 77
+fi
+
+# We can only run the tests on x86_64.
+if [ "$(uname -m)" != "x86_64" ]; then
+    echo "$script: test skipped because !x86_64."
+    exit 77
+fi
+
+# Check qemu is installed.
+qemu=qemu-system-x86_64
+if ! $qemu -help >/dev/null 2>&1; then
+    echo "$script: test skipped because $qemu not found."
+    exit 77
+fi
+
+# Some guests need special virt-builder parameters.
+# See virt-builder --notes "$guestname"
+declare -a extra
+case "$guestname" in
+    debian-7)
+        extra[${#extra[*]}]='--edit'
+        extra[${#extra[*]}]='/etc/inittab:
+                                s,^#([1-9].*respawn.*/sbin/getty.*),$1,'
+        ;;
+    debian-8|ubuntu-16.04)
+        # These commands are required to fix the serial console.
+        # See
https://askubuntu.com/questions/763908/ubuntu-16-04-has-no-vmware-console-access-once-booted-on-vmware-vsphere-5-5-clus/764476#764476
+        extra[${#extra[*]}]='--edit'
+        extra[${#extra[*]}]='/etc/default/grub:
+           
s/^GRUB_CMDLINE_LINUX_DEFAULT=.*/GRUB_CMDLINE_LINUX_DEFAULT="console=tty0
console=ttyS0,115200n8"/'
+        extra[${#extra[*]}]='--run-command'
+        extra[${#extra[*]}]='update-grub'
+        ;;
+    *)
+        ;;
+esac
+
+disk="console-$guestname.img"
+rm -f "$disk"
+
+# Build a guest (using virt-builder).
+virt-builder "$guestname" \
+             --quiet \
+             -o "$disk" \
+             "${extra[@]}"
+
+out="console-$guestname.out"
+rm -f "$out"
+
+# Boot the guest in qemu with a serial console.  Allow it to run
+# for a set amount of time, and then kill it.
+$qemu \
+    -nodefconfig \
+    -display none \
+    -machine accel=kvm:tcg \
+    -m 1024 \
+    -boot c \
+    -drive file="$disk",format=raw,if=ide \
+    -serial stdio > "$out" &
+pid=$!
+sleep 180
+kill -9 $pid
+
+# Did we see console output?
+err ()
+{
+    set +e
+    echo "$script: didn't see $1 in serial console output"
+    echo "$script: full output from the serial console below"
+    echo
+    cat "$out"
+    exit 1
+}
+
+grub_rex="(highlighted|selected) entry will be (started|executed)
automatically"
+
+case "$guestname" in
+    centos-7.*|rhel-7.*)
+        grep -Esq "$grub_rex" "$out" ||
+            err "GRUB messages"
+        grep -Esq "Linux version [0-9]" "$out" || err
"Linux kernel messages"
+        grep -sq "Reached target Basic System" "$out" ||
err "systemd messages"
+        grep -sq "^Kernel [0-9]" "$out" || err "login
banner"
+        grep -sq "login:" "$out" || err "login
prompt"
+        ;;
+    debian-*)
+        grep -Esq "$grub_rex" "$out" ||
+            err "GRUB messages"
+        # Debian boots with 'quiet' so no kernel messages seen.
+        grep -sq "^Debian GNU/Linux" "$out" || err
"login banner"
+        grep -sq "login:" "$out" || err "login
prompt"
+        ;;
+    fedora-*)
+        grep -Esq "$grub_rex" "$out" ||
+            err "GRUB messages"
+        grep -Esq "Linux version [0-9]" "$out" || err
"Linux kernel messages"
+        grep -sq "Reached target Basic System" "$out" ||
err "systemd messages"
+        grep -sq "^Kernel.*(ttyS0)" "$out" || err
"login banner"
+        grep -sq "login:" "$out" || err "login
prompt"
+        ;;
+    ubuntu-*)
+        # Ubuntu boot is very quiet, but we should see a banner and
+        # a login prompt at the end.
+        grep -sq "^Ubuntu" "$out" || err "login
banner"
+        grep -sq "login:" "$out" || err "login
prompt"
+        ;;
+    *)
+        # Fall back to only checking we see a login prompt.
+        grep -sq "login:" "$out" || err "login
prompt"
+        ;;
+esac
+
+rm "$disk" "$out"
-- 
2.9.3

On Wednesday, 19 October 2016 15:22:06 CEST Richard W.M. Jones
wrote:
> The first of two sets of patches.
> 
> This checks that the templates we supply have the serial console
> enabled, or in two cases, that the serial console can be enabled by
> fiddling with some grub config.
> 
> The second set of patches (to follow) will test that the --password
> and --root-password options work.  This implementation of this is
> quite complex.
LGTM.

Thanks,
-- 
Pino Toscano