Libguestfs - Oct 2015 - Re: [PATCH 3/6] v2v:test:win: actually check the eventual layout

On Mon, Oct 05, 2015 at 03:40:03PM +0300, Roman Kagan
wrote:
> v2v/test-v2v-windows-conversion.sh used to query if the expected
> directories and filed were present in the VM upon conversion; however it
> would ignore the results of that query.
> 
> That lead to the test passing even though the checks failed.
Good point.  Fix is kinda ugly though.  How about:

output=$(
  guestfish --ro -a $d/windows-sda -i <<EOF
  is-dir "/Program Files/Red Hat/Firstboot"
  ...
)
if [ "$output" != "true
true
true
true" ]; then
  echo "$0: firstboot files were not copied into the guest"
  echo "$0: test output was:"
  echo "$output"
  exit 1
fi

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

On Mon, Oct 05, 2015 at 02:24:18PM +0100, Richard W.M. Jones
wrote:
> On Mon, Oct 05, 2015 at 03:40:03PM +0300, Roman Kagan wrote:
> > v2v/test-v2v-windows-conversion.sh used to query if the expected
> > directories and filed were present in the VM upon conversion; however
it
> > would ignore the results of that query.
> > 
> > That lead to the test passing even though the checks failed.
> 
> Good point.  Fix is kinda ugly though.  How about:
> 
> output=$(
>   guestfish --ro -a $d/windows-sda -i <<EOF
>   is-dir "/Program Files/Red Hat/Firstboot"
>   ...
> )
> if [ "$output" != "true
> true
> true
> true" ]; then
>   echo "$0: firstboot files were not copied into the guest"
>   echo "$0: test output was:"
>   echo "$output"
>   exit 1
> fi
This actually falls into two questions: how to match the expected output
and how to report the failure.

For the former, Pino has made a similar argument; however, considering
the next patch in the series, "[PATCH 4/6] tests: use fake virtio-win
drivers", I think that using a constant string for the expected output
would look even uglier.

For the latter, I toggled tracing on and off around the guestfish
commands exactly to make the relevant report appear in case of a
failure: you can see individually which files or directories are
missing.  Here's a sample output generated when qxl.sys is removed from
the source directory:

# make V=1 INSTALLDIRS=vendor -j32 -C v2v check
TESTS=test-v2v-windows-conversion.sh
make: Entering directory '/home/rkagan/work/git/libguestfs/v2v'
make
make[1]: Entering directory '/home/rkagan/work/git/libguestfs/v2v'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/home/rkagan/work/git/libguestfs/v2v'
make  check-TESTS
make[1]: Entering directory '/home/rkagan/work/git/libguestfs/v2v'
/home/rkagan/work/git/libguestfs/run --test ./test-v2v-windows-conversion.sh
[   0.0] Opening the source -i libvirt -ic
test:///home/rkagan/work/git/libguestfs/tests/guests/guests.xml windows
[   0.0] Creating an overlay to protect the source from being modified
[   0.3] Opening the overlay
[   5.1] Initializing the target -o local -os test-v2v-windows-conversion.d
[   5.1] Inspecting the overlay
[   5.6] Checking for sufficient free disk space in the guest
[   5.6] Estimating space required on target for each disk
[   5.6] Converting Microsoft Windows 7 Phony Edition to run on KVM
virt-v2v: This guest has virtio drivers installed.
[   5.8] Mapping filesystem data to avoid copying unused and blank areas
[   5.9] Closing the overlay
[   6.0] Checking if the guest needs BIOS or UEFI to boot
[   6.0] Assigning disks to buses
[   6.0] Copying disk 1/1 to test-v2v-windows-conversion.d/windows-sda (raw)
    (100.00/100%)
[   6.2] Creating output metadata
[   6.2] Finishing off
libguestfs: trace: is_dir "/Program Files/Red Hat/Firstboot"
libguestfs: trace: is_dir = 1
libguestfs: trace: is_file "/Program Files/Red
Hat/Firstboot/firstboot.bat"
libguestfs: trace: is_file = 1
libguestfs: trace: is_dir "/Program Files/Red Hat/Firstboot/scripts"
libguestfs: trace: is_dir = 1
libguestfs: trace: is_dir "/Windows/Drivers/VirtIO"
libguestfs: trace: is_dir = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/netkvm.cat"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/netkvm.inf"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/netkvm.sys"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/qxl.cat"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/qxl.inf"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/qxl.sys"
libguestfs: trace: is_file = 0
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/vioscsi.cat"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/vioscsi.inf"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/vioscsi.sys"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/viostor.cat"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/viostor.inf"
libguestfs: trace: is_file = 1
libguestfs: trace: is_file "/Windows/Drivers/VirtIO/viostor.sys"
libguestfs: trace: is_file = 1
libguestfs: trace: set_trace false
libguestfs: trace: set_trace = 0
/home/rkagan/work/git/libguestfs/run: command failed with exit code 1
FAIL: test-v2v-windows-conversion.sh
=================1 of 1 test failed
=================Makefile:2186: recipe for target 'check-TESTS' failed
make[1]: *** [check-TESTS] Error 1
make[1]: Leaving directory '/home/rkagan/work/git/libguestfs/v2v'
Makefile:2309: recipe for target 'check-am' failed
make: *** [check-am] Error 2
make: Leaving directory '/home/rkagan/work/git/libguestfs/v2v'
#


I tend to think it's more comprehensible than the proposed

...
test-v2v-windows-conversion.sh: test output was:
true
true
true
true
true
true
true
true
true
false
true
true
true
true
true
true
...

Roman.

It's consistent with how we do it everywhere else (see for example:
fish/test-copy.sh), but if having a long list of 'true' words is a
problem you could also do:

output=$(
  guestfish --ro -a $d/windows-sda -i <<EOF
  !echo "dir Firstboot"
  is-dir "/Program Files/Red Hat/Firstboot"
  ...
)
if [ "$output" != "dir Firstboot
true
...
" ]; then
...

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-p2v converts physical machines to virtual machines.  Boot with a
live CD or over the network (PXE) and turn machines into KVM guests.
http://libguestfs.org/virt-v2v

The specific problem with enabling trace around "certain" commands is
that we very often run the tests with LIBGUESTFS_TRACE=1 LIBGUESTFS_DEBUG=1
in the environment.  In this case that doesn't make very much of a
difference, except you're turning off trace at the end of the test
when the person running the tests wanted it to be left on.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW