similar to: [libnbd PATCH 0/2] Expose export description

This is a subset of my v2 posting, but limited to just the NBD_OPT_LIST handling. The biggest change since v2 is the addition of added unit testing in all four language bindings (C, python, ocaml, golang). The tests require nbdkit built from git on PATH, and may not be entirely idiomatic, but I at least validated that they catch issues (for example, adding an exit statement near the end of the

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

Proposal for new APIs to list exports. The general shape of the API can probably best be seen from the examples/list-exports.c example. Rich.

This rounds up the remaining bugs that I originally identified in: https://www.redhat.com/archives/libguestfs/2020-July/msg00153.html Eric Blake (4): api: Permit export list APIs when Connected info: Support --list with serializing servers info: Fix --json output when list size != 1 info: Permit --size --json generator/API.ml | 6 +++--- info/info-list-json.sh | 9 +++++++++

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-

This adds coalescing of adjacent extents of the same type, as mentioned by Eric Blake in the commit message here: https://github.com/libguestfs/libnbd/commit/46072f6611f80245846a445766da071e457b00cd The patch series is rather long because it detours through adding the <vector.h> library from nbdkit into libnbd and replacing ad hoc uses of realloc, char ** etc in various places. Rich.

This is a rough-and-ready implementation of nbdinfo --map option, to display the allocation and other metadata of NBD exports. I only tested it lightly against nbdkit. It needs at least some tests. Command like these should work: nbdinfo --map nbd://localhost nbdinfo --map=qemu:dirty-bitmap nbd://localhost Rich.

This continues the changes for adding NBD_OPT_LIST support. Now, instead of libnbd malloc'ing storage itself, the user passes a callback that can handle name/description pairs however it likes, and we get rid of the artificial cap at 10000 exports. However, the user will probably end up malloc'ing a list themselves, as we can't call nbd_set_export_name, or even request NBD_OPT_INFO

Fixes bugs with JSON output in the previous patch, and adds some tests. Rich.

This replaces 13/13 of my v2 series; and now that it has pretty good testsuite coverage and demonstrable performance improvement to nbdinfo, I'm going ahead and pushing this now. We may still want to add further nbd_opt_* commands for other fine-grained tuning of negotiation, but for now, I think things have stabilized on this end, and I can return to polishing .list_exports on the nbdkit

This continues the changes for adding NBD_OPT_LIST support. Now, instead of libnbd malloc'ing storage itself, the user passes a callback that can handle name/description pairs however it likes, and we get rid of the artificial cap at 10000 exports. However, the user will probably end up malloc'ing a list themselves, as we can't call nbd_set_export_name, or even request NBD_OPT_INFO

On 7/20/20 9:43 AM, Richard W.M. Jones wrote: > A major missing feature of this library was the ability to list > exports from an NBD server. This implements the feature by adding a > new handle mode and additional functions for querying the list of > export names. > --- > .gitignore | 1 + > examples/Makefile.am | 14

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to

Exposing a description in single mode is harder: we'd have to request NBD_INFO_DESCRIPTION during NBD_OPT_GO. For now, the API only supports descriptions in list mode. --- info/info-list-json.sh | 3 ++- info/info-list.sh | 3 ++- info/nbdinfo.c | 21 ++++++++++++++++----- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/info/info-list-json.sh

A major missing feature of this library was the ability to list exports from an NBD server. This implements the feature by adding a new handle mode and additional functions for querying the list of export names. --- .gitignore | 1 + examples/Makefile.am | 14 +++ examples/list-exports.c | 108 +++++++++++++++++++

If the server fails, nbdinfo can write partial output before the error message (albeit on different channels). Here is an example: $ nbdkit eval open='echo EIO fail >&2; exit 1' --run 'nbdinfo --json "$uri"' { "protocol": "newstyle-fixed", "TLS": false, nbdkit: eval[1]: error: /tmp/nbdkitii3pZW/open: fail nbdkit: eval[1]:

Instead of having to munge a URI to supply a different export name, we can exploit the fact that nbd_opt_mode lets us change the preferred export name after nbd_connect_uri has established its socket. In fact, by doing this, nbdinfo no longer needs to directly link against libxml2, so it can now be built regardless of whether the underlying libnbd.so supports URIs (although use of the program

Use the just-added nbd_opt_list_meta_context() API to give more details about each export (matching what 'qemu-nbd --list' already does). Note that this requires some shuffling: listing meta exports requires being in opt mode, but is easiest to do in list_one_export(), which requires setting opt_mode in more code paths, and deferring ready mode until the last possible minute during

Place the tracing of return values after any generated 'out:' label, as it is important to see even the errors caused by calling a function in the wrong handle state. Fixes: 9df088bfa9 --- Multiple bugs found today, but only time to submit one patch so far. I found them all while trying to implement .list_exports in nbdkit. Bug 1: 'nbdinfo --list --json $uri' produces invalid

On Sat, Jul 15, 2023 at 08:49:51PM -0500, Eric Blake wrote: > A compliant server should not send NBD_REPLY_TYPE_BLOCK_STATUS unless > we successfully negotiated a meta context. And our default strictness > settings refuse to let us send NBD_CMD_BLOCK_STATUS unless we > negotiated a meta context. But when you mix non-default settings > (using nbd_set_strict to disable