Libguestfs - Dec 2019 - Long running nbdkit tests

The nbdkit tests take a long time to run especially on armv7.  To try
to get an idea of which tests take the most time, and since I happen
to have a qemu aarch64 *TCG* guest available, I ran the tests on that
guest.

A TCG guest is about the worst case for libguestfs, used by some
tests, since it uses double-nested TCG, and as you can see below there
are tests taking over 6 hours.

In fact I didn't complete the run yet.  test-cow.sh is still running
(after 2+ hours) and there may be further tests to come.

Results:

06:43:00 PASS: test-cache.sh
06:37:48 PASS: test-cache-on-read.sh
[              test-cow.sh possibly here ]
00:35:41 PASS: test-partitioning3.sh
00:34:29 PASS: test-partitioning5.sh
00:34:03 PASS: test-linuxdisk-copy-out.sh
00:33:58 PASS: test-floppy.sh
00:30:45 PASS: test-partitioning2.sh
00:28:41 PASS: test-linuxdisk.sh
00:28:34 PASS: test-iso.sh
00:02:59 PASS: test-cache-max-size.sh
00:02:58 PASS: test-partitioning6.sh
00:02:55 PASS: test-partitioning4.sh
00:02:51 PASS: test-eflags.sh
00:01:11 PASS: test-export-name.sh
00:01:01 PASS: test-parallel-sh.sh
00:00:54 PASS: test-python.sh
00:00:44 PASS: test-sh-errors.sh
00:00:44 PASS: test-cacheextents.sh
00:00:41 PASS: test-sh-extents.sh
00:00:35 PASS: test-info-base64.sh

Ignoring the ones which take mere minutes (those run in seconds on
more reasonable hardware), here is a summary of how the tests are
implemented:

test-cache.sh               Gx2 F D
test-cache-on-read.sh       Gx2 F D
test-cow.sh                 Gx3 F
test-partitioning3.sh       G
test-partitioning5.sh       G   F
test-linuxdisk-copy-out.sh  G     Dx4
test-floppy.sh              G     Dx2
test-partitioning2.sh       G   F
test-linuxdisk.sh           G     D
test-iso.sh                 G

D = using libguestfs download or similar APIs
F = using libguestfs "fill" APIs which write lots of data
G = guestfish
xN = runs multiple times

So there's quite a strong correlation between multiple runs of
guestfish, and using those two APIs, and time taken.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html

I pushed a change which should reduce the time taken in the
test-cache-* and test-cow.sh tests.  Again note this is double TCG
which is the worst case for these tests and explains the excessively
long times which wouldn't be seen on a normal build system.

Updated results:

01:04:18 PASS: test-readahead
01:02:32 PASS: test-cow.sh
00:37:25 PASS: test-floppy.sh
00:34:57 PASS: test-shell
00:34:53 PASS: test-linuxdisk-copy-out.sh
00:33:57 PASS: test-partitioning3.sh
00:33:46 PASS: test-perl
00:33:38 PASS: test-partitioning5.sh
00:33:38 PASS: test-memory
00:33:28 PASS: test-ocaml
00:33:11 PASS: test-lua
00:32:33 PASS: test-tcl
00:32:16 PASS: test-offset
00:31:48 PASS: test-xz
00:31:47 PASS: test-xz-curl
00:30:19 PASS: test-partitioning2.sh
00:28:57 PASS: test-linuxdisk.sh
00:28:46 PASS: test-data
00:28:24 PASS: test-gzip
00:27:57 PASS: test-ext2
00:27:49 PASS: test-curl
00:27:16 PASS: test-iso.sh
00:25:56 PASS: test-nbd
[all others < 7 mins]

Analysis of what these tests do as in the previous email:

test-readahead             G   Fx2
test-cow.sh                Gx2
test-floppy.sh             G       Dx2
test-shell                 G
test-linuxdisk-copy-out.sh G       Dx4
test-partitioning3.sh      G
test-perl                  G
test-partitioning5.sh      G   F
test-memory                G
test-ocaml                 G
test-lua                   G
test-tcl                   G
test-offset                G
test-xz                    G
test-xz-curl               G           W
test-partitioning2.sh      G   F
test-linuxdisk.sh          G       D
test-data                  G
test-gzip                  G
test-ext2                  G
test-curl                  G           W
test-iso.sh                G
test-nbd                   G

D = using libguestfs download or similar APIs
F = using libguestfs "fill" APIs which write lots of data
G = guestfish/libguestfs
W = curl/webserver
xN = runs multiple times

test-readahead needs more work.  test-cow.sh is obdurately difficult
to test.  The others are good tests so there's probably not much we
can or should do to them.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

I pushed these two patches which together reduce the build time in
Koji from 7 hours down to 2 hours - almost all of that spent on the
armv7 builder, the other architectures take only a few minutes.

https://github.com/libguestfs/nbdkit/commit/fdbd0ef69f0c222d0cdc35d3c48297aff549abce
https://github.com/libguestfs/nbdkit/commit/ce8357a883167b75ced16ea9f1488f7933b3e813

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top