similar to: [PATCH] Set LC_ALL=C before running qemu.

The patch I posted last week to disable mandatory locking for readonly drives (https://www.redhat.com/archives/libguestfs/2017-September/msg00013.html) was wrong in a couple of respects. Firstly it didn't work, which I didn't detect because my tests were testing the wrong thing. Oops. Secondly it used a simple version number check to detect qemu binaries implementing mandatory locking.

v2 -> v3: - I addressed everything that Pino mentioned last time. - It's tricky to get a stable run when multiple copies of qemu are involved, because the same cache files get overwritten by parallel libguestfs. So I changed the names of the cache files to include the qemu binary key (size, mtime), which removes this conflict. This is in new patch 4/6. Rich.

Patches 1-4 are almost the same as they are when previously posted here: https://www.redhat.com/archives/libguestfs/2017-September/msg00039.html Patch 5 actually uses the mandatory locking test to turn off locking in the narrow case where a drive is opened readonly, and then only for the drive being inspected. Passes ordinary tests (‘check-direct’ and ‘check-valgrind-direct’). Rich.

Patches 1 & 2 were posted previously here: https://www.redhat.com/archives/libguestfs/2016-May/msg00134.html Patch 3 is a hack so I can test this using my own version of qemu (the `-L ?' stuff is not upstream). Patch 4 is where the real action takes place: Replace the caching of qemu features in blob-like files with a sqlite database. Probably the best way to approach this patch is to

v1 -> v2: - Rebase on top of Pino's version work. Two patches went upstream, these are the two remaining patches. Note the generation number is still inside the qemu.stat file. We could put it in the filename, I have no particular preference. Rich.

These assorted patches end up with adding virtio-scsi support to libguestfs. It passes libguestfs-test-tool, but I haven't yet tried to run the full set of tests. In theory > 26 devices can be added, but it's likely that certain parts of the daemon will break if you actually try this. This of course needs to be fixed. Thanks Paolo Bonzini for invaluable help. Rich.

Doing qemu feature detection in the direct backend takes ~100ms because we need to run `qemu -help' and `qemu -devices ?', and each of those interacts with glibc's very slow link loader. Fixing the link loader is really hard. Instead memoize the output of those two commands. This patch series first separates all the code dealing with qemu into a separate module (src/qemu.c) and

More comprehensive support for virtio-scsi. Passes all the tests. Rich.

The latter is a better way to disable the qemu display output as we need to, without enabling extra devices (which are disabled already, anyway). Also, related to the change above, ban the -display parameter from the ones that can be supplied by the user. --- configure.ac | 8 ++++---- src/launch-direct.c | 12 ++++++++---- src/launch.c | 1 + 3 files changed, 13 insertions(+), 8

This commit uses -bios bios-fast.bin if available, which basically stops SeaBIOS from trying to do PCI probing during boot, which is a waste of time when using the -kernel option. v1 -> v2: - Rebase on top of Pino's work. This still has 3 dependencies: - The qemu memoization work (v2). - Support for '-L ?' in qemu:

NOTE: Not for upstream, yet. This depends on 3 non-upstream patches. - The qemu rework in libguestfs, which in turn depends on what Pino is up to. - Support for '-L ?' in qemu: https://lists.gnu.org/archive/html/qemu-devel/2016-05/threads.html#02596 - Support for bios-fast.bin in qemu: https://lists.gnu.org/archive/html/qemu-devel/2016-05/threads.html#02408 This commit

This attempts to implement the idea proposed in https://www.redhat.com/archives/libguestfs/2010-April/msg00087.html The idea is that an externally managed QEMU (manual, or via libvirt) can boot the appliance kernel/initrd. libguestfs can then be just told of the UNIX domain socket associated with the guest daemon. An example based on guestfish. 1. Step one, find the appliance kernel/initrd

This set of 10 patches removes the need for any vmchannel implementation, although we can still choose to use vmchannel if we want. In this so-called "null vmchannel" configuration, the appliance connects directly to a port on the library. The exact method is described in patch 9/10. This method still requires SLIRP (user mode networking) so it is not a panacea, because recent

Make use of the common version code, and avoid a separate regexp. --- src/qemu.c | 22 ++-------------------- 1 file changed, 2 insertions(+), 20 deletions(-) diff --git a/src/qemu.c b/src/qemu.c index 11bf5cf..844c3f0 100644 --- a/src/qemu.c +++ b/src/qemu.c @@ -37,16 +37,12 @@ #include <libxml/uri.h> -#include <pcre.h> - #include "ignore-value.h" #include

Dr. David Gilbert pointed out to me that the first line of the qemu -help output includes the qemu version, so we don't need to run qemu -version at all. This saves about 0.04s on the launch time when using the direct backend. --- src/launch-direct.c | 54 +++++++++++++++++++---------------------------------- 1 file changed, 19 insertions(+), 35 deletions(-) diff --git a/src/launch-direct.c

The previous code duplicated a lot of common code for reading and writing the cache file per data field. This change simply factors out that common code. This makes it simpler to add new tests in future. This is just refactoring, it should have no effect. --- lib/qemu.c | 375 +++++++++++++++++++++++++++++++++++++++---------------------- 1 file changed, 238 insertions(+), 137 deletions(-)

Inspired by libvirt's virCommand* internal mini-library, this adds some internal APIs for running commands. The first patch contains the new APIs. The subsequent patches change various parts of the library over to use it. Rich.

qemu feature detection takes about 95ms on my laptop. The overhead is almost all due to the time taken by the glibc link loader opening the 170+ libraries that qemu is linked to (×2 because we need to run qemu twice). Fixing that is seriously hard work. Therefore memoize the results of guestfs_int_test_qemu. This is keyed on the size and mtime of the qemu binary, so if the user changes the

Lots of changes since v2, too much to remember or summarize. Please ignore patch 11/11, it's just for my testing. Rich.

This also changes the name of the "vmchannel" buffer to just "buf", reflecting the fact that it's just used as a temporary buffer, and that the word vmchannel is overloaded. Rich. -- Richard Jones, Emerging Technologies, Red Hat http://et.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages.