similar to: [PATCH nbdkit 0/5 NOT WORKING] nbd: Implement command= and socket-fd= parameters.

I fixed the deadlock - turned out to be an actual bug in the nbd plugin (see patch 8). I changed the command syntax so it's now: nbdkit nbd command=qemu arg=-f arg=qcow2 arg=/path/to/disk.qcow2 Nir wrote: 18:08 < nsoffer> rwmjones: regarding the nbd proxy patches, did you have specific flow that help us? 18:08 < nsoffer> rwmjones: or this is just a way to support qcow2 in the

Another series on top of my exportname filter, marking off another todo bullet point. With this, you can now use the NBD plugin as a transparent passthrough of all export names served by the remote server in both directions (list advertisement server to client, and export name from client to server). Eric Blake (3): nbd: Implement .default_export, .export_description nbd: Add

With new enough libnbd, we already support vsock by virtue of uri=; however, it's also nice to allow direct exposure of the nbd_connect_vsock() api. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/nbd/nbdkit-nbd-plugin.pod | 29 +++++++++++--- plugins/nbd/nbd.c | 63 ++++++++++++++++++++++++------- 2 files changed, 73 insertions(+), 19 deletions(-) diff --git

libnbd-0.1.2-1 is now available in Fedora 29/30 updates-testing, although it was not compiled against libxml2 so it lacks uri support (I ended up testing patch 4 with a self-built libnbd). Diffs since v1 - rebase to master, bump from libnbd 0.1 to 0.1.2, add URI support, better timing results Still not done - patch 5 needs associated tests Eric Blake (5): nbd: Check for libnbd nbd:

libnbd-0.1.4-1 is now available in Fedora 29/30 updates testing. Diffs since v2 - rebase to master, bump from libnbd 0.1.2 to 0.1.3+, add tests to TLS usage which flushed out the need to turn relative pathnames into absolute, doc tweaks Now that the testsuite covers TLS and libnbd has been fixed to provide the things I found lacking when developing v2, I'm leaning towards pushing this on

Patch 1 played with an early draft of Rich's Fedora 30 libnbd package: https://bugzilla.redhat.com/show_bug.cgi?id=1713767#c17 Note that comment 21 provides a newer package 0.1.1-1 with a different API; and that libnbd has more unreleased API changes in the pipeline (whether that will be called 0.2 or 0.1.2); so we'll have to tweak things based on what is actually available in distros.

Accepting only Unix sockets can be a bit limiting; let's be more flexible. Eric Blake (2): nbd: Refactor Unix socket connection nbd: Support TCP socket plugins/nbd/nbdkit-nbd-plugin.pod | 36 ++++-- plugins/nbd/nbd.c | 175 ++++++++++++++++++++++-------- TODO | 3 - 3 files changed, 161 insertions(+), 53 deletions(-) -- 2.20.1

With new enough libnbd, we already support vsock by virtue of uri=; however, it's also nice to allow direct exposure of the nbd_connect_vsock() api. Signed-off-by: Eric Blake <eblake@redhat.com> --- As with commit 7ce9feef, there is no easy way to add testsuite coverage for this. plugins/nbd/nbdkit-nbd-plugin.pod | 30 +++++++++----- plugins/nbd/nbd.c | 65

I got annoyed by qemu-nbd's default of only allowing a single connection; combine that with nbdkit's nbd plugin, and even 'qemu-nbd --list' of nbdkit counts as the single connection and immediately hangs up. If we introduce a shared mode, then 'qemu-nbd --list' can connect as many times as it wants without killing the original qemu-nbd wrapped by nbdkit. But this in turn

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

On Thu, Aug 06, 2020 at 09:23:46PM -0500, Eric Blake wrote: > + if (!filename == !directory) { A bit tricksy. In plugins/nbd/nbd.c I used: int c = !!sockname + !!hostname + !!uri + (command.size > 0) + (socket_fd >= 0) + !!raw_cid; /* Check the user passed exactly one connection parameter. */ if (c > 1) { nbdkit_error ("cannot mix Unix ‘socket’, TCP

Since the v1 series (0/4, at [1]), I've applied patches 1 and 2, rewritten patch 3 [Forbid NUL in export and context names] into patch 4 here, patch 4 there turned into patch 6 here, and everything else here is new. [1]https://www.redhat.com/archives/libguestfs/2019-September/msg00180.html I don't know if there is a handy reusable function for checking whether a string contains valid

I'm still working on the interleaving (and Rich reminded me on IRC that we still don't have THREAD_MODEL_PARALLEL working anywhere yet, anyways). Since nbdkit doesn't really have a parallel plugin yet, my testing on that front will have to use qemu-nbd as the original server, as well as qemu-io as the driver (qemu-io's aio_read and aio_write commands can be used to trigger

Experimental and only very lightly tested so far. Rich.

We should have only one NBD protocol file. Let's make nbdkit's version the canonical one, and use it in libnbd. Rich.

This is the sort of thing I had in mind for option (1) here: https://www.redhat.com/archives/libguestfs/2019-September/msg00047.html It does reveal that the way we currently list exports is naive to say the least ... Rich.

The docs are a lot more useful with a graphic showing how to wire together nbdkit as a bridge from old-to-new. The converse, bridging new-to-old, is best deferred until I add support for the nbd plugin connecting to a TCP socket. It is also worth mentioning use of nbdkit filters (after all, qemu-nbd 4.0 was able to deprecate its --partition option by pointing to 'nbdkit --filter=partition nbd

To avoid bitrot, any new feature needs testsuite coverage ;) Still to come: once I get my work on parallel nbd finished, I will add a test-parallel-nbd.sh that closely mirrors what my other series added in test-parallel-file.sh. If desired, it might be a fun exercise to tweak test-nbd into using a for-loop of user-controlled depth for how deep you want to nest the forwarding tree, to see where

More nbd-protocol.h improvements Eric Blake (2): common/protocol: Switch nbdmagic to uint64_t common/protocol: Declare additional constants common/protocol/nbd-protocol.h | 16 ++++++++++------ server/protocol-handshake-newstyle.c | 2 +- server/protocol-handshake-oldstyle.c | 2 +- plugins/nbd/nbd-standalone.c | 2 +- tests/test-layers.c | 2 +- 5 files

This is similar to the recent --no-sr option - it's a change that is unlikely to ever be used except by someone testing whether a client is compliant to the protocol, but in that niche case, it can be quite handy (it's a lot nicer to be able to purposefully cripple a server from the command line than from a one-off compile, when testing if a client's fallback for a spec-compliant but