similar to: [PATCH libnbd 0/5] info: --map: Coalesce adjacent extents of the same type.

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-

This is a rough-and-ready implementation of nbdinfo --map option, to display the allocation and other metadata of NBD exports. I only tested it lightly against nbdkit. It needs at least some tests. Command like these should work: nbdinfo --map nbd://localhost nbdinfo --map=qemu:dirty-bitmap nbd://localhost Rich.

Fixes bugs with JSON output in the previous patch, and adds some tests. Rich.

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake

This is a subset of my v2 posting, but limited to just the NBD_OPT_LIST handling. The biggest change since v2 is the addition of added unit testing in all four language bindings (C, python, ocaml, golang). The tests require nbdkit built from git on PATH, and may not be entirely idiomatic, but I at least validated that they catch issues (for example, adding an exit statement near the end of the

A rather trivial decoding; we may enhance it further if qemu extends things to give an integer depth alongside its tri-state encoding. --- I'll wait to push this to libnbd until the counterpart qemu patches land upstream, although it looks like I've got positive review. info/nbdinfo.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index

This rounds up the remaining bugs that I originally identified in: https://www.redhat.com/archives/libguestfs/2020-July/msg00153.html Eric Blake (4): api: Permit export list APIs when Connected info: Support --list with serializing servers info: Fix --json output when list size != 1 info: Permit --size --json generator/API.ml | 6 +++--- info/info-list-json.sh | 9 +++++++++

This replaces 13/13 of my v2 series; and now that it has pretty good testsuite coverage and demonstrable performance improvement to nbdinfo, I'm going ahead and pushing this now. We may still want to add further nbd_opt_* commands for other fine-grained tuning of negotiation, but for now, I think things have stabilized on this end, and I can return to polishing .list_exports on the nbdkit

An incremental improvement on top of listing exports. I still think it's worth experimenting with revisiting how our API for list mode should actually work [1] (so that we can reuse a single connection for both grabbing the list and finally using NBD_OPT_GO), but this change was easier to whip together while still thinking about that. [1]

Necessary when writing a client that wants to avoid unnecessary EINVAL errors from sending unaligned requests. At some point, we may want to add synchronous convenience API wrappers that do request splitting or read-modify-write to obey server constraints while still appearing to the library client as accepting any possible request. But such a wrapper should only be synchronous and not copied to

If the server fails, nbdinfo can write partial output before the error message (albeit on different channels). Here is an example: $ nbdkit eval open='echo EIO fail >&2; exit 1' --run 'nbdinfo --json "$uri"' { "protocol": "newstyle-fixed", "TLS": false, nbdkit: eval[1]: error: /tmp/nbdkitii3pZW/open: fail nbdkit: eval[1]:

Previous discussion: https://www.redhat.com/archives/libguestfs/2019-August/msg00102.html Last night I experimentally added support for URIs that contain the query parameter tls-psk-file, as part of rewriting the tests to cover more of the URI code. So you can now have a URI like: nbds://alice@localhost/?tls-psk-file=keys.psk However there's an obvious security problem here because now

Otherwise, we get a failure due to Numerical result out of range. And for safety's sake, we are best capping our request to an aligned value, if the server insists on minimum alignment. Fixes: f3fd935c --- info/nbdinfo.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index 1afdf98..2b22f51 100644 --- a/info/nbdinfo.c +++

Use the just-added nbd_opt_list_meta_context() API to give more details about each export (matching what 'qemu-nbd --list' already does). Note that this requires some shuffling: listing meta exports requires being in opt mode, but is easiest to do in list_one_export(), which requires setting opt_mode in more code paths, and deferring ready mode until the last possible minute during

This continues the changes for adding NBD_OPT_LIST support. Now, instead of libnbd malloc'ing storage itself, the user passes a callback that can handle name/description pairs however it likes, and we get rid of the artificial cap at 10000 exports. However, the user will probably end up malloc'ing a list themselves, as we can't call nbd_set_export_name, or even request NBD_OPT_INFO

Appears to have a measurable benefit, see 3/3 for test results. Rich.

This isn't quite finished because not all of the tests or examples have been updated, but it demonstrates an idea: Should we forget about the concept of having multiple connections managed under a single handle? In this patch there is a single ‘struct nbd_handle *’ which manages a single state machine and connection (and therefore no nbd_connection). To connect to a multi-conn server you must

Another series on top of my exportname filter, marking off another todo bullet point. With this, you can now use the NBD plugin as a transparent passthrough of all export names served by the remote server in both directions (list advertisement server to client, and export name from client to server). Eric Blake (3): nbd: Implement .default_export, .export_description nbd: Add

Based on Laszlo's approval of my idea here: https://listman.redhat.com/archives/libguestfs/2023-September/032661.html but as I would like to resync human-size.h back to nbdkit, I'm reluctant to apply patch 1 this until I get Rich's consent to relicensing (this email serves as my consent for my contribution here): https://listman.redhat.com/archives/libguestfs/2023-October/032755.html