search for: qemu_data

The patch I posted last week to disable mandatory locking for readonly drives (https://www.redhat.com/archives/libguestfs/2017-September/msg00013.html) was wrong in a couple of respects. Firstly it didn't work, which I didn't detect because my tests were testing the wrong thing. Oops. Secondly it used a simple version number check to detect qemu binaries implementing mandatory locking.

Patches 1-4 are almost the same as they are when previously posted here: https://www.redhat.com/archives/libguestfs/2017-September/msg00039.html Patch 5 actually uses the mandatory locking test to turn off locking in the narrow case where a drive is opened readonly, and then only for the drive being inspected. Passes ordinary tests (‘check-direct’ and ‘check-valgrind-direct’). Rich.

v2 -> v3: - I addressed everything that Pino mentioned last time. - It's tricky to get a stable run when multiple copies of qemu are involved, because the same cache files get overwritten by parallel libguestfs. So I changed the names of the cache files to include the qemu binary key (size, mtime), which removes this conflict. This is in new patch 4/6. Rich.

...ons(-) diff --git a/lib/qemu.c b/lib/qemu.c index bdd9947a8..34775041f 100644 --- a/lib/qemu.c +++ b/lib/qemu.c @@ -37,6 +37,8 @@ #include <libxml/uri.h> +#include <yajl/yajl_tree.h> + #include "ignore-value.h" #include "guestfs.h" @@ -50,9 +52,13 @@ struct qemu_data { char *qemu_help; /* Output of qemu -help. */ char *qemu_devices; /* Output of qemu -device ? */ + char *qmp_commands; /* Output of QMP query-commands. */ + char *qmp_schema; /* Output of QMP query-qmp-schema. */ /* The following fields a...

...++- 3 files changed, 105 insertions(+), 30 deletions(-) diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h index 841fd515b..f5eed82a1 100644 --- a/lib/guestfs-internal.h +++ b/lib/guestfs-internal.h @@ -777,6 +777,7 @@ extern struct version guestfs_int_qemu_version (guestfs_h *g, struct qemu_data * extern int guestfs_int_qemu_supports (guestfs_h *g, const struct qemu_data *, const char *option); extern int guestfs_int_qemu_supports_device (guestfs_h *g, const struct qemu_data *, const char *device_name); extern int guestfs_int_qemu_mandatory_locking (guestfs_h *g, const struct qemu_data...

....c | 375 +++++++++++++++++++++++++++++++++++++++---------------------- 1 file changed, 238 insertions(+), 137 deletions(-) diff --git a/lib/qemu.c b/lib/qemu.c index 48e283d48..bdd9947a8 100644 --- a/lib/qemu.c +++ b/lib/qemu.c @@ -44,6 +44,10 @@ #include "guestfs_protocol.h" struct qemu_data { + int generation; /* MEMO_GENERATION read from qemu.stat */ + uint64_t prev_size; /* Size of qemu binary when cached. */ + uint64_t prev_mtime; /* mtime of qemu binary when cached. */ + char *qemu_help; /* Output of qemu -help. */ char *qemu_...

NOTE: Not for upstream, yet. This depends on 3 non-upstream patches. - The qemu rework in libguestfs, which in turn depends on what Pino is up to. - Support for '-L ?' in qemu: https://lists.gnu.org/archive/html/qemu-devel/2016-05/threads.html#02596 - Support for bios-fast.bin in qemu: https://lists.gnu.org/archive/html/qemu-devel/2016-05/threads.html#02408 This commit

Doing qemu feature detection in the direct backend takes ~100ms because we need to run `qemu -help' and `qemu -devices ?', and each of those interacts with glibc's very slow link loader. Fixing the link loader is really hard. Instead memoize the output of those two commands. This patch series first separates all the code dealing with qemu into a separate module (src/qemu.c) and

...-------- lib/qemu.c | 47 --------------------------- 3 files changed, 24 insertions(+), 111 deletions(-) diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h index 5e9d97c07..a04ccff09 100644 --- a/lib/guestfs-internal.h +++ b/lib/guestfs-internal.h @@ -983,7 +983,6 @@ struct qemu_data; extern struct qemu_data *guestfs_int_test_qemu (guestfs_h *g, struct version *qemu_version); extern int guestfs_int_qemu_supports (guestfs_h *g, const struct qemu_data *, const char *option); extern int guestfs_int_qemu_supports_device (guestfs_h *g, const struct qemu_data *, const char *device...

This commit uses -bios bios-fast.bin if available, which basically stops SeaBIOS from trying to do PCI probing during boot, which is a waste of time when using the -kernel option. v1 -> v2: - Rebase on top of Pino's work. This still has 3 dependencies: - The qemu memoization work (v2). - Support for '-L ?' in qemu:

v1 -> v2: - Rebase on top of Pino's version work. Two patches went upstream, these are the two remaining patches. Note the generation number is still inside the qemu.stat file. We could put it in the filename, I have no particular preference. Rich.

Patches 1 & 2 were posted previously here: https://www.redhat.com/archives/libguestfs/2016-May/msg00134.html Patch 3 is a hack so I can test this using my own version of qemu (the `-L ?' stuff is not upstream). Patch 4 is where the real action takes place: Replace the caching of qemu features in blob-like files with a sqlite database. Probably the best way to approach this patch is to

...b/qemu.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 61 insertions(+) diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h index 2ca258cb3..ab839de4d 100644 --- a/lib/guestfs-internal.h +++ b/lib/guestfs-internal.h @@ -983,6 +983,7 @@ extern struct qemu_data *guestfs_int_test_qemu (guestfs_h *g); extern struct version guestfs_int_qemu_version (guestfs_h *g, struct qemu_data *); extern int guestfs_int_qemu_supports (guestfs_h *g, const struct qemu_data *, const char *option); extern int guestfs_int_qemu_supports_device (guestfs_h *g, const struct qem...

...moizes this data in its own place. --- src/qemu.c | 154 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 146 insertions(+), 8 deletions(-) diff --git a/src/qemu.c b/src/qemu.c index feb8bd6..259a6fd 100644 --- a/src/qemu.c +++ b/src/qemu.c @@ -58,23 +58,163 @@ struct qemu_data { guestfs_int_qemu_supports_virtio_scsi */ }; +static int test_qemu (guestfs_h *g, struct qemu_data *data); static void parse_qemu_version (guestfs_h *g, struct qemu_data *data); static void read_all (guestfs_h *g, void *retv, const char *buf, size_t len); +/* T...

..."grep -sq -- 'info.*-U'"); > > This may match something else in future, in case some other command of > qemu-img gets another option with "info" in it... > > TBH this is something we have already, and precisely in the qemu_data > struct, which is memoized. One downside is that using it would mean > memoizing the qemu help/schema in advance, even if the appliance is not > started; so code like `guestfish disk-format foo` will be slower. > OTOH, the upside is that there is no need to "reprobe" qemu-im...

...e user to use multiple qemu binaries in parallel without conflicts. --- lib/qemu.c | 29 ++++++++++++++++++++++++----- 1 file changed, 24 insertions(+), 5 deletions(-) diff --git a/lib/qemu.c b/lib/qemu.c index 097d56929..1549bb33a 100644 --- a/lib/qemu.c +++ b/lib/qemu.c @@ -60,6 +60,7 @@ struct qemu_data { yajl_val qmp_schema_tree; /* qmp_schema parsed into a JSON tree */ }; +static char *cache_filename (guestfs_h *g, const char *cachedir, const struct stat *, const char *suffix); static int test_qemu_help (guestfs_h *g, struct qemu_data *data); static int read_cache_qemu_help (guestfs_...

...+++ b/lib/launch-direct.c @@ -66,6 +66,7 @@ struct backend_direct_data { pid_t recoverypid; /* Recovery process PID. */ struct version qemu_version; /* qemu version (0 if unable to parse). */ + int qemu_mandatory_locking; /* qemu >= 2.10 does mandatory locking */ struct qemu_data *qemu_data; /* qemu -help output etc. */ char guestfsd_sock[UNIX_PATH_MAX]; /* Path to daemon socket. */ @@ -255,11 +256,13 @@ add_drive_standard_params (guestfs_h *g, struct backend_direct_data *data, } else { /* Writable qcow2 overlay on top of read-only drive. */ - append_lis...

...q -- 'info.*-U'"); > > > > > > This may match something else in future, in case some other command of > > > qemu-img gets another option with "info" in it... > > > > > > TBH this is something we have already, and precisely in the qemu_data > > > struct, which is memoized. One downside is that using it would mean > > > memoizing the qemu help/schema in advance, even if the appliance is not > > > started; so code like `guestfish disk-format foo` will be slower. > > > OTOH, the upside is that there i...

https://bugs.launchpad.net/qemu/+bug/1740364 --- lib/guestfs-internal.h | 3 +++ lib/handle.c | 2 ++ lib/info.c | 39 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 44 insertions(+) diff --git a/lib/guestfs-internal.h b/lib/guestfs-internal.h index adeb9478a..c66c55e70 100644 --- a/lib/guestfs-internal.h +++ b/lib/guestfs-internal.h @@ -510,6 +510,9 @@ struct

...--help | " > + "grep -sq -- 'info.*-U'"); This may match something else in future, in case some other command of qemu-img gets another option with "info" in it... TBH this is something we have already, and precisely in the qemu_data struct, which is memoized. One downside is that using it would mean memoizing the qemu help/schema in advance, even if the appliance is not started; so code like `guestfish disk-format foo` will be slower. OTOH, the upside is that there is no need to "reprobe" qemu-img for something that...