similar to: [PATCH nbdkit] locks: Remove debugging messages about

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

v1 -> v2: - Fixed everything mentioned in the review. Rich.

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

This fixes the race conditions for me, using the test described here: https://www.redhat.com/archives/libguestfs/2017-September/msg00226.html Rich.

--- src/locks.c | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) diff --git a/src/locks.c b/src/locks.c index bd8fd99..1724b5a 100644 --- a/src/locks.c +++ b/src/locks.c @@ -56,53 +56,39 @@ lock_init_thread_model (void) void lock_connection (void) { - if (thread_model <= NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS) { - debug ("acquire connection

Mostly code motion. --- src/Makefile.am | 1 + src/connections.c | 14 +++---- src/internal.h | 14 ++++--- src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 ---

Mostly code motion. --- src/Makefile.am | 1 + src/connections.c | 14 +++---- src/internal.h | 14 ++++--- src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 ---

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

There are several race conditions where nbdkit exits (calling plugin_cleanup() which unloads the plugin), while the plugin is either in a callback or in plugin.close(). Avoid these by: (1) Taking a shared lock around all plugin callbacks. (2) Taking the same as an exclusive lock in plugin_cleanup. This delays plugin_cleanup until all callbacks have finished. (3) Removing a few unnecessary

Low-level pthread locks should not fail except in extreme cases of programmer bugs; we're better off calling attention to such bugs rather than just assuming that they work and continuing on with possibly inconsistent state. Signed-off-by: Eric Blake <eblake@redhat.com> --- server/connections.c | 20 ++++++++++++-------- server/locks.c | 42

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

Commit 876d715e17a59bd25df53be34b942afd96e52da9 ("Refactor plugin_* functions into a backend struct.") causes hidden crashes in the test suite (which unfortunately don't cause tests to fail). These all come from handle_single_connection where we may lock the connection, run a single connection, free the backend, and then try to unlock the connection. Unfortunately the unlock

When I added .can_FOO caching in 1.16, I missed the case that the sh plugin itself was calling .can_flush twice in some situations (in order to default .can_fua). Then right after, I regressed it to call .can_zero twice (in order to default .can_fast_zero). I also missed that .thread_model could use better caching, because at the time, I did not add testsuite coverage. Fix that now. Eric Blake

On 01/16/2018 08:11 AM, Richard W.M. Jones wrote: > Mostly code motion. > --- > src/Makefile.am | 1 + > src/connections.c | 14 +++---- > src/internal.h | 14 ++++--- > src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ > src/plugins.c | 77 +++++------------------------------- > 5 files changed, 142 insertions(+), 79

On Fri, Nov 22, 2019 at 9:54 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > These are accessible from the plugin by: > > import nbdkit > > if flags & nbdkit.FLAG_MAY_TRIM: > &c. Nice way to expose the flags! > Many (all?) of these are not yet useful for plugins, some will never > be useful, but they only consume a tiny bit of memory and

These are a couple of patches in preparation for the Block Status implementation. While the patches (especially the second one) are very large they are really just elementary code motion. Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which

On Fri, Nov 22, 2019 at 10:52 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > On Fri, Nov 22, 2019 at 10:05:15PM +0200, Nir Soffer wrote: > > On Fri, Nov 22, 2019 at 9:54 PM Richard W.M. Jones <rjones@redhat.com> wrote: > > > > > > These are accessible from the plugin by: > > > > > > import nbdkit > > > > > > if

On Haiku tests/test-socket-activation failed with: nbdkit: locks.c:96:lock_request: conn != NULL called from server/sockets.c: accept_connection in the fallback path which does: lock_request (); thread_data->sock = set_cloexec (accept (listen_sock, NULL, NULL)); unlock_request () Because there is no current connection in this thread this code fails. However it should be possible to

Rich pointed out that making thread_model dynamic even for plugins makes some sense, so here is the code for doing it for 'sh'. I'm less confident on how to do it for OCaml and Rust (not to mention that those allow the plugin to already compile in their model, rather than the language binding glue forcing a model). The other languages (lua, perl, python, ruby) still need to be