similar to: [nbdkit PATCH 0/2] more protocol.h tweaks

We should have only one NBD protocol file. Let's make nbdkit's version the canonical one, and use it in libnbd. Rich.

Experimental and only very lightly tested so far. Rich.

Updated in https://github.com/libguestfs/nbdkit/commit/XXX Requires tweaks to our handling of nbd magic. --- Will update with a real URL, if we like it. lib/nbd-protocol.h | 35 ++++++++++++++++------------------- generator/states-magic.c | 2 +- 2 files changed, 17 insertions(+), 20 deletions(-) diff --git a/lib/nbd-protocol.h b/lib/nbd-protocol.h index fe4d451..bdd1ef2 100644 ---

Diff against libnbd’s copy of this file, and change this one until it matches. --- common/protocol/nbd-protocol.h | 76 +++++++++++++++++----------- server/protocol-handshake-newstyle.c | 26 +++++----- server/protocol-handshake-oldstyle.c | 4 +- server/protocol.c | 25 ++++----- tests/test-layers.c | 14 ++--- 5 files changed, 81 insertions(+), 64

This is the bare minimum needed to allow the user to take control over the rest of option negotiating. This patch adds several new API: nbd_set_opt_mode() - called during Created to enable the new mode nbd_get_opt_mode() - query whether opt mode is enabled nbd_opt_go() - used in Negotiating state to attempt to use export nbd_opt_abort() - used in Negotiating state to skip Connected state

This isn't quite finished because not all of the tests or examples have been updated, but it demonstrates an idea: Should we forget about the concept of having multiple connections managed under a single handle? In this patch there is a single ‘struct nbd_handle *’ which manages a single state machine and connection (and therefore no nbd_connection). To connect to a multi-conn server you must

On 8/11/20 12:12 PM, Richard W.M. Jones wrote: > I think this needs extra documentation in docs/libnbd.pod because it's > definitely not clear just from the rather thin manual page for > set_opt_mode how it works. docs/libnbd.pod is a good place for a > broader description of how it works. Yes, good idea. State-wise, the existing flow was: Created - Progress only by command

# HG changeset patch # User Carl-Daniel Hailfinger <Carl-Daniel.Hailfinger@inf.h-brs.de> # Date 1474903734 -7200 # Mon Sep 26 17:28:54 2016 +0200 # Node ID dbd1ea0a401cedcfa426097a289d852971b14f1e # Parent d7d5078d08c711032837dcac79a4450226ec2ce5 nbdkit: Fix flags in old-style connection diff -r d7d5078d08c7 -r dbd1ea0a401c src/connections.c --- a/src/connections.c Sun Sep 25 05:04:02

Eventually nbdkit will install this as a public header file and we will be able to consume it from $includedir. In the mean time synchronize with the copy in nbdkit. --- lib/nbd-protocol.h | 57 +++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 29 deletions(-) diff --git a/lib/nbd-protocol.h b/lib/nbd-protocol.h index 699aa22..7df411a 100644 ---

These are a couple of patches in preparation for the Block Status implementation. While the patches (especially the second one) are very large they are really just elementary code motion. Rich.

There's no substantial difference over v1, I simply fixed a few whitespace issues, moved one struct around and tidied up the comments. Rich.

Some further small changes are made to allow this header to be used from arbitrary ISO C compilers. It can now be used from other projects. --- common/protocol/Makefile.am | 2 ++ common/protocol/nbd-protocol.h | 38 ++++++++++++++++++++-------------- 2 files changed, 24 insertions(+), 16 deletions(-) diff --git a/common/protocol/Makefile.am b/common/protocol/Makefile.am index

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

Some of these affect attempts to connect to older qemu-nbd versions, some of them were triggered by manual edits to qemu-nbd source code to provoke various other compliant (if uncommon) server behaviors. Eric Blake (4): starttls: Skip error payload if falling back to unencrypted states: Reject payload to NBD_REP_ACK meta-context: Skip error payload if server lacks meta_context states: Add

Proposal for new APIs to list exports. The general shape of the API can probably best be seen from the examples/list-exports.c example. Rich.

This is only lightly tested (against just qemu NBD client), and the code might be structured a little better as the _negotiate_handshake_newstyle_options function has now grown to be huge. Anyway works for me. Rich.

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake

I'm still working on the interleaving (and Rich reminded me on IRC that we still don't have THREAD_MODEL_PARALLEL working anywhere yet, anyways). Since nbdkit doesn't really have a parallel plugin yet, my testing on that front will have to use qemu-nbd as the original server, as well as qemu-io as the driver (qemu-io's aio_read and aio_write commands can be used to trigger

Patch 1 played with an early draft of Rich's Fedora 30 libnbd package: https://bugzilla.redhat.com/show_bug.cgi?id=1713767#c17 Note that comment 21 provides a newer package 0.1.1-1 with a different API; and that libnbd has more unreleased API changes in the pipeline (whether that will be called 0.2 or 0.1.2); so we'll have to tweak things based on what is actually available in distros.

Similar to the recent --mask-handshake command line added to nbdkit to test client fallbacks to crippled servers, it can be worth testing server fallbacks to crippled clients. And just as we have exposed whether the client will request structured replies, we can also expose whether the client will understand various handshake flags from the NBD protocol. Of course, we default to supporting all