similar to: Plans for nbdkit 1.18 release?

Eric: Did you want to take this one any further? It might be one that we save for > 1.18: https://www.redhat.com/archives/libguestfs/2020-February/thread.html#00206 Another thing I've been thinking about for some time is splitting .config_complete into .config_complete + .get_ready (new name TBD). At the moment .config_complete is both the place where we finish processing config, and

On Sat, Feb 22, 2020 at 05:11:01AM -0600, Eric Blake wrote: > On 2/22/20 4:37 AM, Richard W.M. Jones wrote: > >Another thing I've been thinking about for some time is splitting > >.config_complete into .config_complete + .get_ready (new name TBD). > >At the moment .config_complete is both the place where we finish > >processing config, and also the last chance we get

I like this change. I think we were overloading the config_complete method before to do two different things (complete configuration; do any allocation/housekeeping necessary before we can start serving). The only questions in my mind are whether we want this before 1.18, and whether the name ("get_ready") is a good one. Rich.

On Thu, Feb 20, 2020 at 05:03:18PM -0600, Eric Blake wrote: > On 2/20/20 3:37 PM, Richard W.M. Jones wrote: > >Hi Eric, > > > >nbdkit 1.16 was released over 3 months ago, and there has been quite a > >lot of upstream development since then, so it could be about time for > >a new stable release. Anything in particular that you would like to > >get in to 1.18?

On 2/20/20 3:37 PM, Richard W.M. Jones wrote: > Hi Eric, > > nbdkit 1.16 was released over 3 months ago, and there has been quite a > lot of upstream development since then, so it could be about time for > a new stable release. Anything in particular that you would like to > get in to 1.18? Or anything we've added that you have reservations > about supporting long-term?

I'd like to propose we backport this to 1.18 and some earlier stable branches too. Rich.

On 2/22/20 4:37 AM, Richard W.M. Jones wrote: > Eric: > > Did you want to take this one any further? It might be one that we > save for > 1.18: > > https://www.redhat.com/archives/libguestfs/2020-February/thread.html#00206 At this point, I'm thinking save it for post-1.18 (we've already got enough churn on VDDK with LD_LIBRARY_PATH changes). > > Another

On Fri, Aug 07, 2020 at 05:00:52PM -0500, Eric Blake wrote: > The next patch wants to add a filter that will prevent DoS attacks > from a plaintext client; to be successful, the filter must guarantee > that nbdkit did not settle on SERIALIZE_CONNECTIONS. The easiest way > to solve this is to expose the final thread model to .get_ready, which > is after the point where

If you have a plugin which either creates background threads itself or uses a library that creates background threads, it turns out you cannot create these in .get_ready (or earlier). The reason is that nbdkit forks when either daemonizing itself or using the --run option, and fork cancels all the background threads in the child process (the daemonized or captive nbdkit). The only good solution

This happened on s390 in Koji, once. The key lines from the log are: + start_t=0 nbdkit: sh[1]: debug: retry 1: waiting 1 seconds before retrying nbdkit: sh[1]: debug: retry 1: waiting 1 seconds before retrying + end_t=1 Somehow nbdkit did wait 2 seconds (or at least, nbdkit_nanosleep (1, 0) was called twice by the retry filter). But in the bash script, time (as measured by $SECONDS) advanced

On 4/15/20 11:16 AM, Richard W.M. Jones wrote: > This allows you to copy an environ, while optionally adding extra > (key, value) pairs to it. > --- > common/utils/Makefile.am | 2 + > common/utils/utils.h | 1 + > common/utils/environ.c | 116 +++++++++++++++++++++++++++++++++++++++ > 3 files changed, 119 insertions(+) > > +++ b/common/utils/utils.h

In v2: - use int instead of bool in the public header - split the tests from the code - don't overload test-layers; instead, add new tests - add a missing fflush exposed by the new tests - other minor cleanups Eric Blake (3): server: Add nbdkit_stdio_safe server: Sanitize stdin/out before running plugin code server: More tests of stdin/out handling docs/nbdkit-plugin.pod |

Patch 3 still needs tests added, but it is at least working from my simple command line tests. Eric Blake (3): server: Implement nbdkit_is_tls for use during .open server: Expose final thread_model to filter's .get_ready tlsdummy: New filter docs/nbdkit-filter.pod | 21 +- docs/nbdkit-plugin.pod | 34 ++- docs/nbdkit-tls.pod

For context see these threads: https://lists.gnu.org/archive/html/qemu-discuss/2020-06/threads.html#00053 https://lists.gnu.org/archive/html/qemu-block/2020-06/threads.html#01496 Rich.

VDDK plugin + --run was broken because of the following sequence of events: (1) After .config_complete, server redirects stdin/stdout to /dev/null. (2) Server then calls vddk_get_ready which reexecs. (3) We restart the server from the top, but with stdin/stdout redirected to /dev/null. So saved_stdin/saved_stdout save /dev/null. (4) run_command is called which "restores"

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

Since v1: - patch 1: check size limits - patch 2: better handling of default export name canonicalization - patch 3: support filters as well as plugins - patch 4: new - patch 5: rewrite sh parser, fix testsuite to actually work and cover more cases (now that libnbd.git is fixed) Eric Blake (4): server: Add exports list functions server: Prepare to use export list from plugin log: Add

This picks up python and ocaml. Some of our languages are lacking a number of bindings (for example, lua and perl lack .extents, so I didn't have anything to copy from), and I felt less comfortable with golang and rust. But for python and ocaml, I was able to test a working implementation. Eric Blake (2): python: Implement .list_exports and friends ocaml: Implement .list_exports and

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the