similar to: [PATCH nbdkit 0/2] Change .errno_is_reliable function to .errno_is_preserved constant.

This is a major rewrite compared to my v1 series, where I've tried a lot harder to ensure that we still accommodate building on Haiku (although I have not actually yet fired up a Haiku VM to try it for myself). I also managed to make the sh plugin fully parallel, on capable platforms. See also my question on patch 10 on whether I've picked the best naming convention. Eric Blake (17):

This cleans up the existing code base with regards to implicit use of errno from language bindings, then rebases the previous work in python on top of that. I'm still playing with the perl bindings, but got further after reading 'perldoc perlembed'. Eric Blake (4): plugins: Don't use bogus errno from non-C plugins plugins: Add new nbdkit_set_error() utility function python:

The callback doesn't make much sense: Could the value change per-connection? Unlikely. This is a property of the plugin as a whole. I changed the name to "errno_is_preserved", because it's not about the reliability of errno, but about whether errno is preserved across calls. --- docs/nbdkit-plugin.pod | 37 +++++++++------------------------

On 02/06/2017 08:57 AM, Richard W.M. Jones wrote: > The callback doesn't make much sense: Could the value change > per-connection? Unlikely. This is a property of the plugin as a > whole. > > I changed the name to "errno_is_preserved", because it's not about the > reliability of errno, but about whether errno is preserved across > calls. Makes it possible

v1 -> v2: - Fixed everything mentioned in the review. Rich.

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

POD errors are not nice, and 'pod2man --stderr' does not do what we want. Eric Blake (2): perl: Fix previous patches build: Kill build on POD error .gitignore | 2 ++ configure.ac | 10 +--------- docs/Makefile.am | 8 ++++++-- docs/nbdkit-plugin.pod | 2 +- plugins/curl/Makefile.am | 4 +++-

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

There is a small test provided. I tested this a lot more locally and it seems pretty robust. Rich.

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

Allow a plugin field to declare whether a parallel plugin can tolerate windows where fds are not CLOEXEC, or must take precautions to avoid leaking fds if the plugin may fork. For safety reasons, the flag defaults to off, but many in-tree plugins can set it to on (most commonly because they don't fork after .config_complete; for libvirt because it is documented to clean up fds on fork so it

The concern is a client is blocked while processing a request. The nbdkit server design requires a thread per request being processed regardless of the number of connections or clients. We want to run 1000's of requests in parallel without needing a thread at nbdkit layer per request in flight. Our plugin layer is built around boost asio and a few threads in a worker pool running an io

Needs some work still, see in particular the commit message for patch 3. Rich.

Here's what I'm currently playing with; I'm not ready to commit anything until I rebase my FUA work on top of this, as I only want to break filter ABI once between releases. Eric Blake (2): backend: Rework internal/filter error return semantics filters: Add log filter TODO | 2 - docs/nbdkit-filter.pod | 84 +++++++-- docs/nbdkit.pod

Tested via: term1$ qemu-nbd -k $PWD/sock -t -f raw -x foo junk --trace=nbd_\* term2$ ./nbdkit -f -v -e bar nbd socket=$PWD/sock export=foo term3$ qemu-io -t none -f raw nbd://localhost:10809/bar --trace=nbd_\* and checking the traces to see that 'w 0 1' vs. 'w -f 0 1' was able to influence whether the FUA flag showed up at the server in term1. Still to go: figure out how to

Similar to python and perl. But MUCH easier (especially considering that this is the first time I've every tried to run Ruby). I even had fun making set_error() polymorphic. Eric Blake (2): ruby: Expose nbdkit_set_error to ruby script ruby: Support zero callback plugins/ruby/example.rb | 11 ++++++++ plugins/ruby/nbdkit-ruby-plugin.pod | 54

Rich pointed out that making thread_model dynamic even for plugins makes some sense, so here is the code for doing it for 'sh'. I'm less confident on how to do it for OCaml and Rust (not to mention that those allow the plugin to already compile in their model, rather than the language binding glue forcing a model). The other languages (lua, perl, python, ruby) still need to be

I'm still working my way through the filters before this series will be complete, but this is enough of a start to at least get some feedback on the idea of implementing another NBD protocol extension. Eric Blake (9): server: Internal hooks for implementing NBD_CMD_CACHE plugins: Add .cache callback file, split: Implement .cache with posix_fadvise nbd: Implement NBD_CMD_CACHE

Requires patch 1 and 2 of the python series: https://www.redhat.com/archives/libguestfs/2017-January/msg00126.html This is the perl implementation along the same lines. We still haven't decided if patch 1 of the python series should change OCaml to report errno as reliable or not, but perhaps we can commit that patch as-is now and then touch things up further when we actually get set_error

On Wed, Jan 25, 2017 at 08:42:34PM -0600, Eric Blake wrote: > +eg. NULL or -1. If the call to C<nbdkit_set_error> is omitted, then > +the value of C<errno> will be used instead. [...] > +/* Grab the appropriate error value. > + */ > +static int > +_get_error (void) > +{ > + int err = errno; > + int ret = tls_get_error (); > + > + if (!ret) > +