similar to: [libnbd PATCH 0/7] state machine refactoring

This isn't quite finished because not all of the tests or examples have been updated, but it demonstrates an idea: Should we forget about the concept of having multiple connections managed under a single handle? In this patch there is a single ‘struct nbd_handle *’ which manages a single state machine and connection (and therefore no nbd_connection). To connect to a multi-conn server you must

Proposal for new APIs to list exports. The general shape of the API can probably best be seen from the examples/list-exports.c example. Rich.

Some of these affect attempts to connect to older qemu-nbd versions, some of them were triggered by manual edits to qemu-nbd source code to provoke various other compliant (if uncommon) server behaviors. Eric Blake (4): starttls: Skip error payload if falling back to unencrypted states: Reject payload to NBD_REP_ACK meta-context: Skip error payload if server lacks meta_context states: Add

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake

I still need to wire in the use of *_notify functions into nbdkit to prove whether it makes the code any faster or easier to maintain, but at least the added example shows one good use case for the new API. Eric Blake (6): api: Add nbd_aio_in_flight generator: Allow DEAD state actions to run generator: Allow Int64 in callbacks states: Prepare for aio notify callback api: Add new

Per recent discussion here: https://www.redhat.com/archives/libguestfs/2019-May/thread.html#00175

An incremental improvement on top of listing exports. I still think it's worth experimenting with revisiting how our API for list mode should actually work [1] (so that we can reuse a single connection for both grabbing the list and finally using NBD_OPT_GO), but this change was easier to whip together while still thinking about that. [1]

This was v3 patch 2/22, reworked to address the confusion about how a structured reply header is read in two pieces before getting to the payload portion. I'm still working on rebasing the rest of my v3 series (patches 1, 3-22) from other comments given, but this seemed independent enough that it's worth posting now rather than holding it up for the rest of the series. Eric Blake (4):

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

Instead of repeating a check for valid reply headers in each sub-state machine, let's have a common helper function do all the work. Additionally, a central location will make it easier to uniformly capture any NBD_REP_ERR message payloads. --- generator/generator | 8 +-- generator/states-newstyle-opt-go.c | 40 +++----------

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-

I've mentioned this topic before (in fact, the idea of adding NBD_CMD_FLAG_DF was first mentioned at [1]), but finally finished enough of an implementation to feel confident in posting it. I'd still like to add something under examples/ that uses the new API to implement strict checking of a server's structured replies read implementation (ensure that a server never sends data after

On 7/20/20 9:43 AM, Richard W.M. Jones wrote: > A major missing feature of this library was the ability to list > exports from an NBD server. This implements the feature by adding a > new handle mode and additional functions for querying the list of > export names. > --- > .gitignore | 1 + > examples/Makefile.am | 14

This is a subset of my v2 posting, but limited to just the NBD_OPT_LIST handling. The biggest change since v2 is the addition of added unit testing in all four language bindings (C, python, ocaml, golang). The tests require nbdkit built from git on PATH, and may not be entirely idiomatic, but I at least validated that they catch issues (for example, adding an exit statement near the end of the

If the server passes us a malformed error reply type with a message length longer than the overall structured reply, we would blindly obey that size and get out of sync with the server (perhaps even hanging on a read for data that will never come). Broken since its introduction in commit 28952eda. Fix it by parsing the tail of an error separate from the message, which also lets us add other

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to

Similar to the recent --mask-handshake command line added to nbdkit to test client fallbacks to crippled servers, it can be worth testing server fallbacks to crippled clients. And just as we have exposed whether the client will request structured replies, we can also expose whether the client will understand various handshake flags from the NBD protocol. Of course, we default to supporting all

A major missing feature of this library was the ability to list exports from an NBD server. This implements the feature by adding a new handle mode and additional functions for querying the list of export names. --- .gitignore | 1 + examples/Makefile.am | 14 +++ examples/list-exports.c | 108 +++++++++++++++++++

This is the bare minimum needed to allow the user to take control over the rest of option negotiating. This patch adds several new API: nbd_set_opt_mode() - called during Created to enable the new mode nbd_get_opt_mode() - query whether opt mode is enabled nbd_opt_go() - used in Negotiating state to attempt to use export nbd_opt_abort() - used in Negotiating state to skip Connected state

Appears to have a measurable benefit, see 3/3 for test results. Rich.