similar to: [libnbd PATCH] api: Add way to avoid structured replies

Similar to the recent --mask-handshake command line added to nbdkit to test client fallbacks to crippled servers, it can be worth testing server fallbacks to crippled clients. And just as we have exposed whether the client will request structured replies, we can also expose whether the client will understand various handshake flags from the NBD protocol. Of course, we default to supporting all

Similar to nbd_get_tls_negotiated, for observing what we actually settled on with the server, rather than what was requested. --- generator/generator | 30 +++++++++++++++++++++++++----- lib/handle.c | 6 ++++++ tests/meta-base-allocation.c | 15 +++++++++++++++ tests/oldstyle.c | 7 ++++++- 4 files changed, 52 insertions(+), 6 deletions(-) diff --git

When LIBNBD_TLS_ALLOW is used we don't have a way to find out if TLS was really negotiated. This adds a flag and a way to read it back. Unfortunately there is no test yet, because LIBNBD_TLS_ALLOW is not tested -- it really should be but requires quite a complicated set of tests because ideally we'd like to find out whether it falls back correctly for all supported servers. --- TODO

On Wed, Sep 04, 2019 at 01:28:08PM -0500, Eric Blake wrote: > We want to default to requesting structured replies, whether or not > that request will be honored (it's essential for efficient sparse file > reads and the DF flag for structured pread, as well as for meta > context support even if we do not request a default meta context). > However, for integration testing, it can

This commit adds a new API which can be used from the connected to state to read back which NBD protocol (eg. oldstyle, newstyle-fixed) we are using. It was helpful to add a new state in newstyle negotiation (%NEWSTYLE.FINISHED) so we can route all successful option negotiations through a single path before moving to the %READY state, allowing us to set h->protocol in one place. ---

This doesn't include Eric's new APIs, but if you push those then I can rebase this one on top. Rich.

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake

In v2: - now based on my proposal to add LIBNBD_SHUTDOWN_IMMEDIATE - four flags instead of two: STRICT_FLAGS is new (patch 4), and STRICT_BOUNDS is separate from STRICT_ZERO_SIZE (patch 5) - various refactorings for more shared code and less duplication Eric Blake (5): api: Add xxx_MASK constant for each Flags type generator: Refactor filtering of accepted OFlags api: Add

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to

Qemu was able to demonstrate that knowing whether a zero operation is fast is useful when copying from one image to another: there is a choice between bulk pre-zeroing and then revisiting the data sections (fewer transactions, but depends on the zeroing to be fast), vs. visiting every portion of the disk only once (more transactions, but no time lost to duplicated I/O due to slow zeroes). As

Well, the golang changes (patch 1 and 2/3 of v1) were already committed, all that was left was the OCaml changes. I'm a lot happier with how things turned out with an UNKNOWN constructor in the OCaml variants. Eric Blake (3): tests: Enhance coverage of enum/flag range checking ocaml: Support unknown values for Enum/Flags ocaml: Typesafe returns for REnum/RFlags generator/OCaml.ml

This replaces 13/13 of my v2 series; and now that it has pretty good testsuite coverage and demonstrable performance improvement to nbdinfo, I'm going ahead and pushing this now. We may still want to add further nbd_opt_* commands for other fine-grained tuning of negotiation, but for now, I think things have stabilized on this end, and I can return to polishing .list_exports on the nbdkit

Previous discussion: https://www.redhat.com/archives/libguestfs/2019-August/msg00102.html Last night I experimentally added support for URIs that contain the query parameter tls-psk-file, as part of rewriting the tests to cover more of the URI code. So you can now have a URI like: nbds://alice@localhost/?tls-psk-file=keys.psk However there's an obvious security problem here because now

Drop the SET designation, so that we can reuse these states for OPT_LIST_META_CONTEXT in the next patch. --- generator/Makefile.am | 4 +-- generator/state_machine.ml | 6 ++--- ...t.c => states-newstyle-opt-meta-context.c} | 26 +++++++++---------- .../states-newstyle-opt-structured-reply.c | 4 +-- generator/states-newstyle.c

This is a subset of my v2 posting, but limited to just the NBD_OPT_LIST handling. The biggest change since v2 is the addition of added unit testing in all four language bindings (C, python, ocaml, golang). The tests require nbdkit built from git on PATH, and may not be entirely idiomatic, but I at least validated that they catch issues (for example, adding an exit statement near the end of the

See discussion in this and following messages: https://www.redhat.com/archives/libguestfs/2019-September/msg00020.html --- generator/generator | 131 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 121 insertions(+), 10 deletions(-) diff --git a/generator/generator b/generator/generator index 1cc5c19..8a3905f 100755 --- a/generator/generator +++ b/generator/generator @@ -845,9

This patch adds support for symbol versions. It is based on what libvirt does. The generated syms file looks like: LIBNBD_1.0 { global: nbd_...; nbd_...; local: *; }; In a future stable 1.2 release, new symbols would go into a new section which would look like this: LIBNBD_1.2 { global: nbd_new_symbol; nbd_another_new_symbol; local: *; } LIBNBD_1.0; In my testing the

Proposal for new APIs to list exports. The general shape of the API can probably best be seen from the examples/list-exports.c example. Rich.

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-