similar to: [libnbd PATCH 0/4] More nbdinfo fixes

An incremental improvement on top of listing exports. I still think it's worth experimenting with revisiting how our API for list mode should actually work [1] (so that we can reuse a single connection for both grabbing the list and finally using NBD_OPT_GO), but this change was easier to whip together while still thinking about that. [1]

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

This is a rough-and-ready implementation of nbdinfo --map option, to display the allocation and other metadata of NBD exports. I only tested it lightly against nbdkit. It needs at least some tests. Command like these should work: nbdinfo --map nbd://localhost nbdinfo --map=qemu:dirty-bitmap nbd://localhost Rich.

Fixes bugs with JSON output in the previous patch, and adds some tests. Rich.

This adds coalescing of adjacent extents of the same type, as mentioned by Eric Blake in the commit message here: https://github.com/libguestfs/libnbd/commit/46072f6611f80245846a445766da071e457b00cd The patch series is rather long because it detours through adding the <vector.h> library from nbdkit into libnbd and replacing ad hoc uses of realloc, char ** etc in various places. Rich.

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-

This is a subset of my v2 posting, but limited to just the NBD_OPT_LIST handling. The biggest change since v2 is the addition of added unit testing in all four language bindings (C, python, ocaml, golang). The tests require nbdkit built from git on PATH, and may not be entirely idiomatic, but I at least validated that they catch issues (for example, adding an exit statement near the end of the

This replaces 13/13 of my v2 series; and now that it has pretty good testsuite coverage and demonstrable performance improvement to nbdinfo, I'm going ahead and pushing this now. We may still want to add further nbd_opt_* commands for other fine-grained tuning of negotiation, but for now, I think things have stabilized on this end, and I can return to polishing .list_exports on the nbdkit

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to

Place the tracing of return values after any generated 'out:' label, as it is important to see even the errors caused by calling a function in the wrong handle state. Fixes: 9df088bfa9 --- Multiple bugs found today, but only time to submit one patch so far. I found them all while trying to implement .list_exports in nbdkit. Bug 1: 'nbdinfo --list --json $uri' produces invalid

If the server fails, nbdinfo can write partial output before the error message (albeit on different channels). Here is an example: $ nbdkit eval open='echo EIO fail >&2; exit 1' --run 'nbdinfo --json "$uri"' { "protocol": "newstyle-fixed", "TLS": false, nbdkit: eval[1]: error: /tmp/nbdkitii3pZW/open: fail nbdkit: eval[1]:

Exposing a description in single mode is harder: we'd have to request NBD_INFO_DESCRIPTION during NBD_OPT_GO. For now, the API only supports descriptions in list mode. --- info/info-list-json.sh | 3 ++- info/info-list.sh | 3 ++- info/nbdinfo.c | 21 ++++++++++++++++----- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/info/info-list-json.sh

Use the just-added nbd_opt_list_meta_context() API to give more details about each export (matching what 'qemu-nbd --list' already does). Note that this requires some shuffling: listing meta exports requires being in opt mode, but is easiest to do in list_one_export(), which requires setting opt_mode in more code paths, and deferring ready mode until the last possible minute during

--- info/Makefile.am | 4 + info/info-map-base-allocation-json.sh | 52 ++++++++++ info/info-map-base-allocation.sh | 49 ++++++++++ info/nbdinfo.c | 134 ++++++++++++++++++++++++-- info/nbdinfo.pod | 39 +++++++- 5 files changed, 271 insertions(+), 7 deletions(-) diff --git a/info/Makefile.am b/info/Makefile.am index

A rather trivial decoding; we may enhance it further if qemu extends things to give an integer depth alongside its tri-state encoding. --- I'll wait to push this to libnbd until the counterpart qemu patches land upstream, although it looks like I've got positive review. info/nbdinfo.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index

Now that we have an API, it's time to put it to use. --- info/nbdinfo.pod | 6 ++++++ info/nbdinfo.c | 18 ++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/info/nbdinfo.pod b/info/nbdinfo.pod index 9761551..d0d20a9 100644 --- a/info/nbdinfo.pod +++ b/info/nbdinfo.pod @@ -45,6 +45,9 @@ L<https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md>):

Instead of having to munge a URI to supply a different export name, we can exploit the fact that nbd_opt_mode lets us change the preferred export name after nbd_connect_uri has established its socket. In fact, by doing this, nbdinfo no longer needs to directly link against libxml2, so it can now be built regardless of whether the underlying libnbd.so supports URIs (although use of the program

We have now resolved some TODO entries, making nbdinfo --list able to use a single connection except when doing content probing. --- info/nbdinfo.pod | 2 ++ TODO | 13 ----------- info/nbdinfo.c | 56 +++++++++++++++++++++++++++--------------------- 3 files changed, 34 insertions(+), 37 deletions(-) diff --git a/info/nbdinfo.pod b/info/nbdinfo.pod index d0d20a9..19305bf 100644

Based on Laszlo's approval of my idea here: https://listman.redhat.com/archives/libguestfs/2023-September/032661.html but as I would like to resync human-size.h back to nbdkit, I'm reluctant to apply patch 1 this until I get Rich's consent to relicensing (this email serves as my consent for my contribution here): https://listman.redhat.com/archives/libguestfs/2023-October/032755.html

This continues the changes for adding NBD_OPT_LIST support. Now, instead of libnbd malloc'ing storage itself, the user passes a callback that can handle name/description pairs however it likes, and we get rid of the artificial cap at 10000 exports. However, the user will probably end up malloc'ing a list themselves, as we can't call nbd_set_export_name, or even request NBD_OPT_INFO