similar to: Planning for libnbd 1.2

This program allows you to turn a network block device source into a FUSE filesystem containing a virtual file: $ nbdkit memory 128M $ mkdir mp $ nbdfuse mp/ramdisk nbd://localhost & $ ls -l mp total 0 -rw-rw-rw-. 1 rjones rjones 134217728 Oct 12 15:09 ramdisk $ dd if=/dev/urandom bs=1M count=128 of=mp/ramdisk conv=notrunc,nocreat 128+0 records in 128+0 records out

On 10/12/19 9:21 AM, Richard W.M. Jones wrote: > This program allows you to turn a network block device source into a > FUSE filesystem containing a virtual file: > > $ nbdkit memory 128M > $ mkdir mp > $ nbdfuse mp/ramdisk nbd://localhost & > $ ls -l mp > total 0 > -rw-rw-rw-. 1 rjones rjones 134217728 Oct 12 15:09 ramdisk > $ dd

I'm pleased to announce the releases of libnbd 1.2 and nbdkit 1.16. These are a high performance Network Block Device (NBD) client library and server. Key features of libnbd: * Synchronous API for ease of use. * Asynchronous API for writing non-blocking, multithreaded clients. You can mix both APIs freely. * High performance. * Minimal dependencies for the basic library. *

This adds a new API for connecting to AF_VSOCK protocol (https://wiki.qemu.org/Features/VirtioVsock). For example: nbd_connect_vsock (nbd, 2, 10809); There is no test of this feature because it only works between guest and host. You cannot start a server and client on the host and talk between them, which is what we'd need to write a sane test. --- configure.ac | 2 ++

Test both the TLS enabled and fallback paths. nbd-server doesn't appear to support TLS at all, and qemu-nbd is known not to allow fallback to unencrypted, and therefore it only makes sense to test nbdkit at the moment. --- .gitignore | 4 ++++ TODO | 3 --- interop/Makefile.am | 54 +++++++++++++++++++++++++++++++++++++++++++++ interop/interop.c | 30

This is the same, but now we have a symbol for it. --- docs/libnbd.pod | 6 ++++-- interop/interop.c | 2 +- lib/connect.c | 2 +- tests/aio-parallel-load.c | 2 +- tests/aio-parallel.c | 2 +- tests/connect-tls.c | 2 +- tests/synch-parallel.c | 2 +- 7 files changed, 10 insertions(+), 8 deletions(-) diff --git a/docs/libnbd.pod b/docs/libnbd.pod

This neutral refactoring adds -DTLS_MODE. We can in future change the requested TLS mode, but not in this commit. It also checks that nbd_get_tls_negotiated returns true after connecting, when the requested mode was set to LIBNBD_TLS_REQUIRE. --- interop/Makefile.am | 4 ++++ interop/interop.c | 26 ++++++++++++++++++++------ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git

This borrows from a trick in libvirt - by defining $(NULL) to expand to an empty string, we can more consistently write multi-line macros where all useful lines terminate with \, making it easier to add/remove lines without worrying about whether \ needs to be touched up on neighboring lines. --- Looks big, but is fairly mechanical. I'm also doing a similar patch for nbdkit, where it would

This simply compiles, passes tests, but is only able open a handle. This commit does not contain the full bindings. --- Makefile.am | 2 + configure.ac | 32 +++++ generator/GoLang.ml | 116 ++++++++++++++++++ generator/GoLang.mli | 19 +++ generator/Makefile.am

This simply compiles, passes tests, but is only able open a handle. This commit does not contain the full bindings. --- Makefile.am | 2 + configure.ac | 32 ++++ generator/GoLang.ml | 150 ++++++++++++++++++ generator/GoLang.mli | 19 +++ generator/Makefile.am

The NBD protocol is adding an extension for servers to report the initial state of the image (for now, whether it is sparse and/or reads as all zeroes). Time to expose this to clients, via the new API nbd_get_init_flags(). The patch requires refreshing the nbd-protocol.h file from a corresponding contemporary nbdkit commit. Testing is possible with recent enough qemu. --- .gitignore

This adds new APIs for running a local NBD server and connecting to it using systemd socket activation (instead of stdin/stdout). This includes interop tests against nbdkit and qemu-nbd which I believe are the only NBD servers supporting socket activation. (If we find others then we can add more interop tests in future.) The upstream spec for systemd socket activation is here:

This is a series of patches to libnbd and nbdkit adding AF_VSOCK support. On the host side it allows you to start an nbdkit instance which listens on a virtio-vsock socket: $ ./nbdkit -fv --vsock memory 1G ... nbdkit: debug: bound to vsock 2:10809 On the guest side you can then use libnbd to connect to the server: $ ./run nbdsh -c 'h.connect_vsock(2, 10809)' -c

This adds new APIs for running a local NBD server and connecting to it using systemd socket activation (instead of stdin/stdout). This includes interop tests against nbdkit and qemu-nbd which I believe are the only NBD servers supporting socket activation. (If we find others then we can add more interop tests in future.) The upstream spec for systemd socket activation is here:

This is slightly redundant with just trying nbd_set_tls(nbd, 2) then checking for failure; however, this function does not set errors and looks more similar to nbd_supports_uri. --- This is borderline enough that I figured I'd post it to check if we want it. generator/generator | 45 ++++++++++++++++++++++++++++++++++++++------- interop/interop.c | 4 ++++ lib/handle.c | 12

The test interop-qemu-nbd-tls-certs frequently fails on slow (32 bit) machines in Fedora Koji. (Is crypto slow on these already overloaded machines?) As we cannot wait for a signal when qemu-nbd is ready start serving, we have to use a sleep. The current sleep is 5 seconds, which is not long enough. Making the sleep longer would work but is inconsiderate for people using faster machines.

On Fedora 29, nbdkit-devel at 1.10.4 is too old for any of the tests/Makefile.am actions to run, but this also means we don't create any tls pki/ or keys.psk files that we then want to reuse during qemu testing, as evidenced by 'make check' failures such as: FAIL: interop-qemu-nbd-tls-certs ================================ qemu-nbd: Unable to access credentials

On 6/27/19 5:07 AM, Martin Kletzander wrote: > This just defines the namespace, its contexts and related constants and the only > supported one is currently base:allocation. The names of the constants are not > very future-proof, but shorter than LIBNBD_META_NS_CONTEXT_BASE_ALLOCATION or > similar. > > Currently the output looks like this: > > /* "base"

We use a similar trick to libvirt and libguestfs to allow external C programs that use libnbd to be compiled against the built (but not installed) libnbd with: ../libnbd/run ./configure make What actually happens is we have a second pkg-config file (lib/local/libnbd.pc) which points to the locally built libnbd. The ./run script sets up PKG_CONFIG_PATH to point to this directory. Assuming

Now that we have a way to pass Mutable(Int "error") to callbacks, that's a much nicer way for language bindings to use than relying on the value of errno after returning -1. Update the semantics for block status to match what pread_structured does, getting errno out of the equation, and changing things to call the callback for a second context even after an earlier callback failure