similar to: [nbdkit PATCH] nbd: Update for libnbd 0.1.5

I'm not observing any noticeable performance differences, but I'm liking the diffstat. I can't push this patch until we release a new libnbd version with the _notify API addition, but am posting it now for playing with things. Eric Blake (2): nbd: Move transaction info from heap to stack nbd: Use nbdkit aio_*_notify variants plugins/nbd/nbd.c | 217

The 0.9.8 release breaks API, requiring a number of changes: - Use symbolic constants instead of magic numbers/open-coded strings (well, the string for "base:allocation" was present before this libnbd bump) - Change callbacks to drop the valid_flag parameter - Add _is to nbd_read_only call - Drop the _callback suffix on nbd_aio_FOO calls - Add a struct for managing callback/user_data

Now that we have a way to pass Mutable(Int "error") to callbacks, that's a much nicer way for language bindings to use than relying on the value of errno after returning -1. Update the semantics for block status to match what pread_structured does, getting errno out of the equation, and changing things to call the callback for a second context even after an earlier callback failure

libnbd 0.1.7 was released today, which breaks compilation of nbdkit-nbd-plugin. We could work around it by using #ifdef LIBNBD_HAVE_XXX to learn about the renamed functions, but supporting older versions is not all that important when we don't yet have API stability. So patch 1 copes by just bumping the minimum version instead, except that we have yet another pending libnbd patch with an API

libnbd 0.1.8 is due out soon, which reorders several function arguments. Signed-off-by: Eric Blake <eblake@redhat.com> --- configure.ac | 4 ++-- plugins/nbd/nbd.c | 17 +++++++++-------- 2 files changed, 11 insertions(+), 10 deletions(-) diff --git a/configure.ac b/configure.ac index 3b355667..77f08f46 100644 --- a/configure.ac +++ b/configure.ac @@ -718,12 +718,12 @@

We no longer have to track a linked list of in-flight transactions that are pending resolution, but rely instead on libnbd 0.1.6+ doing it on our behalf. Normally, we will get to call nbdplug_register() prior to the notify callback being reached, but under a heavily-loaded system, it is conceivable that the libnbd state machine can manage to fire off our request and receive a server reply all

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

Patch 1 played with an early draft of Rich's Fedora 30 libnbd package: https://bugzilla.redhat.com/show_bug.cgi?id=1713767#c17 Note that comment 21 provides a newer package 0.1.1-1 with a different API; and that libnbd has more unreleased API changes in the pipeline (whether that will be called 0.2 or 0.1.2); so we'll have to tweak things based on what is actually available in distros.

libnbd-0.1.2-1 is now available in Fedora 29/30 updates-testing, although it was not compiled against libxml2 so it lacks uri support (I ended up testing patch 4 with a self-built libnbd). Diffs since v1 - rebase to master, bump from libnbd 0.1 to 0.1.2, add URI support, better timing results Still not done - patch 5 needs associated tests Eric Blake (5): nbd: Check for libnbd nbd:

libnbd-0.1.4-1 is now available in Fedora 29/30 updates testing. Diffs since v2 - rebase to master, bump from libnbd 0.1.2 to 0.1.3+, add tests to TLS usage which flushed out the need to turn relative pathnames into absolute, doc tweaks Now that the testsuite covers TLS and libnbd has been fixed to provide the things I found lacking when developing v2, I'm leaning towards pushing this on

Hi all, ok, downloaded dovecot-1.2.rc5 and compiled it with no problem. Then, I went to compile the latest dovecot-1.2-sieve-0.1.5 with it, and got the following error : mail-raw.c: In function 'mail_raw_init': mail-raw.c:119: error: 'NAMESPACE_FLAG_INTERNAL' undeclared (first use in this function) mail-raw.c:119: error: (Each undeclared identifier is reported only once

This is a bitmask so using an unsigned type is slightly safer. This is not an ABI change since the types are compatible. Thanks: Eric Blake. --- examples/strict-structured-reads.c | 2 +- generator/generator | 67 ++++++++++++++++-------------- interop/structured-read.c | 2 +- lib/internal.h | 2 +- tests/oldstyle.c | 2 +- 5

As before, but this one has working Python bindings. OCaml still TBD. Rich.

Take advantage of the previous patch to make it easier for language bindings to affect the exact error they want on failure, rather than requiring them to influence errno. Update the python test to tickle the changed bindings, and to prove that we can at least trigger an exception, although we are still lacking bindings for python code to access the last NBD exception and error code gracefully

This change does not affect the output of the generator. Note this requires that all Closure args have the same parameter name whichever method they appear in. An alternate refactoring could work the same way as Flags and Enum where the parameter name is part of the arg, eg: type arg = ... | Closure of string * closure (* name, type *) ... "set_debug_callback", {

A Closure is a list of (usually one, but can be more) closures. In C there is also a singe ‘void *user_data’ parameter which is passed by the caller into the function and through as the first parameter of each callback invocation. By grouping the previously separate Opaque and Callback* parameters together we can avoid the awkward situation where we have to scan through the argument list to try

In a future commit we want to add (non-optional) Flags arg. As a step to doing this, generalize OFlags so it's not tied to just NBD_CMD_FLAG_*. This does not change the C API. It does introduce a small change to the OCaml API -- putting related flags into a submodule, basically renaming them. Note we don't provide guarantees for non-C APIs. --- generator/generator

On 7/16/19 6:04 AM, Richard W.M. Jones wrote: > A Closure is a list of (usually one, but can be more) closures. In C > there is also a singe ‘void *user_data’ parameter which is passed by > the caller into the function and through as the first parameter of > each callback invocation. > > By grouping the previously separate Opaque and Callback* parameters > together we can

Posting now that I got something to compile (at the expense of breaking OCaml bindings), but I'm open to ideas on how to improve it. Eric Blake (2): generator: Tweak print_c_arg_list to take alternate first arg RFC: generator: Handle shared callbacks in Python generator/generator | 556 ++++++++++++++++++++++---------------------- 1 file changed, 280 insertions(+), 276 deletions(-) --