search for: nbdinfo

This is a rough-and-ready implementation of nbdinfo --map option, to display the allocation and other metadata of NBD exports. I only tested it lightly against nbdkit. It needs at least some tests. Command like these should work: nbdinfo --map nbd://localhost nbdinfo --map=qemu:dirty-bitmap nbd://localhost Rich.

Fixes bugs with JSON output in the previous patch, and adds some tests. Rich.

--- info/Makefile.am | 4 + info/info-map-base-allocation-json.sh | 52 ++++++++++ info/info-map-base-allocation.sh | 49 ++++++++++ info/nbdinfo.c | 134 ++++++++++++++++++++++++-- info/nbdinfo.pod | 39 +++++++- 5 files changed, 271 insertions(+), 7 deletions(-) diff --git a/info/Makefile.am b/info/Makefile.am index 5d0d288..496692a 100644 --- a/info/Makefile.am +++ b/info/Makefile.am @@ -25,6...

Now that we have an API, it's time to put it to use. --- info/nbdinfo.pod | 6 ++++++ info/nbdinfo.c | 18 ++++++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/info/nbdinfo.pod b/info/nbdinfo.pod index 9761551..d0d20a9 100644 --- a/info/nbdinfo.pod +++ b/info/nbdinfo.pod @@ -45,6 +45,9 @@ L<https://github.com/NetworkBlockDevice/nbd/blob/master/doc...

On 9/26/20 11:36 AM, Richard W.M. Jones wrote: > --- > info/Makefile.am | 4 + > info/info-map-base-allocation-json.sh | 52 ++++++++++ > info/info-map-base-allocation.sh | 49 ++++++++++ > info/nbdinfo.c | 134 ++++++++++++++++++++++++-- > info/nbdinfo.pod | 39 +++++++- > 5 files changed, 271 insertions(+), 7 deletions(-) An interesting followup might be adding options for --start and --length to map out only a subset of the file. That was rec...

...lake (4): api: Permit export list APIs when Connected info: Support --list with serializing servers info: Fix --json output when list size != 1 info: Permit --size --json generator/API.ml | 6 +++--- info/info-list-json.sh | 9 +++++++++ info/info-list.sh | 9 +++++++++ info/nbdinfo.c | 41 ++++++++++++++++++++++++++++------------- 4 files changed, 49 insertions(+), 16 deletions(-) -- 2.28.0

A rather trivial decoding; we may enhance it further if qemu extends things to give an integer depth alongside its tri-state encoding. --- I'll wait to push this to libnbd until the counterpart qemu patches land upstream, although it looks like I've got positive review. info/nbdinfo.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index 24ec129..1afdf98 100644 --- a/info/nbdinfo.c +++ b/info/nbdinfo.c @@ -780,6 +780,13 @@ extent_description (const char *metacontext, uint32_t type) case 1: return "dirty"; } } +...

We have now resolved some TODO entries, making nbdinfo --list able to use a single connection except when doing content probing. --- info/nbdinfo.pod | 2 ++ TODO | 13 ----------- info/nbdinfo.c | 56 +++++++++++++++++++++++++++--------------------- 3 files changed, 34 insertions(+), 37 deletions(-) diff --git a/info/nbdinfo.pod b/inf...

If the server fails, nbdinfo can write partial output before the error message (albeit on different channels). Here is an example: $ nbdkit eval open='echo EIO fail >&2; exit 1' --run 'nbdinfo --json "$uri"' { "protocol": "newstyle-fixed", "TLS": false,...

Otherwise, we get a failure due to Numerical result out of range. And for safety's sake, we are best capping our request to an aligned value, if the server insists on minimum alignment. Fixes: f3fd935c --- info/nbdinfo.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/info/nbdinfo.c b/info/nbdinfo.c index 1afdf98..2b22f51 100644 --- a/info/nbdinfo.c +++ b/info/nbdinfo.c @@ -31,6 +31,8 @@ #include <libnbd.h> +#define MIN(a,b) ((a) < (b) ? (a) : (b)) + static const char *pro...

Instead of having to munge a URI to supply a different export name, we can exploit the fact that nbd_opt_mode lets us change the preferred export name after nbd_connect_uri has established its socket. In fact, by doing this, nbdinfo no longer needs to directly link against libxml2, so it can now be built regardless of whether the underlying libnbd.so supports URIs (although use of the program will fail if libnbd.so was not built against libxml2). --- info/Makefile.am | 6 ++--- info/nbdinfo.c | 63 +++++++++++++++++++------...

...Jones wrote: > This is not related to the current patch, but generally for libnbd: > > (1) Should we be testing interop separately for qemu-storage-daemon > (qsd), or is qsd basically qemu-nbd in a new wrapper so it's not worth > doing it? > > (2) I found a bug in the new nbdinfo behaviour: > > $ nbdkit -fv file dir=/scratch > > (where /scratch is a directory with a lot of files in it) > > $ nbdinfo --version > libnbd 1.5.3 > $ nbdinfo --list nbd://localhost > [... lots of files shown ...] > nbd_opt_info: recv: Connection reset by peer >...

...G NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT +# OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +source ./functions.sh +set -e +set -x + +# This test works with newer libnbd, showing that dynamic mode affects +# export listing. +requires_plugin sh +requires nbdinfo --version + +# Does the nbd plugin support dynamic lists? +if ! nbdkit --dump-plugin nbd | grep -sq libnbd_dynamic_list=1; then + echo "$0: nbd plugin built without dynamic export list support" + exit 77 +fi + +base=test-nbd-dynamic-list +sock1=`mktemp -u` +sock2=`mktemp -u` +pid1=&...

When “nbdkit ... --run 'nbdinfo ...' > out” is expected to fail, we cannot be sure of the content of the "out" file. In particular nbdinfo produces its output incrementally, so in JSON mode there's usually a stray "{" in the output, and in non-JSON mode it usually prints the "protocol: "...

On 10/6/20 11:54 AM, Richard W.M. Jones wrote: > If the server fails, nbdinfo can write partial output before the error > message (albeit on different channels). Here is an example: > > $ nbdkit eval open='echo EIO fail >&2; exit 1' --run 'nbdinfo --json "$uri"' > { > "protocol": "newstyle-fixed", &g...

...rated 'out:' label, as it is important to see even the errors caused by calling a function in the wrong handle state. Fixes: 9df088bfa9 --- Multiple bugs found today, but only time to submit one patch so far. I found them all while trying to implement .list_exports in nbdkit. Bug 1: 'nbdinfo --list --json $uri' produces invalid JSON if there are 0 or more than 1 exports. We need to move the '"exports":[' and ']' into list_all_exports(), not list_one_export(), and manage correct trailing comma output. (We only tested nbdinfo with qemu-nbd, which only list...

Based on Laszlo's approval of my idea here: https://listman.redhat.com/archives/libguestfs/2023-September/032661.html but as I would like to resync human-size.h back to nbdkit, I'm reluctant to apply patch 1 this until I get Rich's consent to relicensing (this email serves as my consent for my contribution here): https://listman.redhat.com/archives/libguestfs/2023-October/032755.html

...doing it? > > Any thoughts on this one? It's a new wrapper, but using the same qemu NBD server code under the hood. Testing it might be good for demonstrating example command lines, but won't tease out any new behavior interactions. > >>> (2) I found a bug in the new nbdinfo behaviour: >>> >>> $ nbdkit -fv file dir=/scratch >>> >>> (where /scratch is a directory with a lot of files in it) >>> >>> $ nbdinfo --version >>> libnbd 1.5.3 >>> $ nbdinfo --list nbd://localhost >>> [... lots of fil...

Exposing a description in single mode is harder: we'd have to request NBD_INFO_DESCRIPTION during NBD_OPT_GO. For now, the API only supports descriptions in list mode. --- info/info-list-json.sh | 3 ++- info/info-list.sh | 3 ++- info/nbdinfo.c | 21 ++++++++++++++++----- 3 files changed, 20 insertions(+), 7 deletions(-) diff --git a/info/info-list-json.sh b/info/info-list-json.sh index 3845875..af29064 100755 --- a/info/info-list-json.sh +++ b/info/info-list-json.sh @@ -33,7 +33,7 @@ cleanup_fn rm -f $img $out $pid $sock rm -...

...io_*, because getting locking right is a bear if you have two non-overlapping client requests that want to visit the same larger-granularity window of the server in parallel. Eric Blake (3): states: Request NBD_INFO_BLOCK_SIZE during NBD_OPT_GO api: Add new API to query negotiated block size nbdinfo: Expose block size constraints info/nbdinfo.pod | 6 +++ lib/internal.h | 7 +++ lib/nbd-protocol.h | 10 ++++- generator/API.ml | 70 +++++++++++++++++++++++++++--- generator/state_machine.ml | 7 +++ generator/st...