similar to: [libnbd PATCH] maint: Use $(NULL) for all Makefile.am macro lists

This neutral refactoring adds -DTLS_MODE. We can in future change the requested TLS mode, but not in this commit. It also checks that nbd_get_tls_negotiated returns true after connecting, when the requested mode was set to LIBNBD_TLS_REQUIRE. --- interop/Makefile.am | 4 ++++ interop/interop.c | 26 ++++++++++++++++++++------ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git

Qemu was able to demonstrate that knowing whether a zero operation is fast is useful when copying from one image to another: there is a choice between bulk pre-zeroing and then revisiting the data sections (fewer transactions, but depends on the zeroing to be fast), vs. visiting every portion of the disk only once (more transactions, but no time lost to duplicated I/O due to slow zeroes). As

Test both the TLS enabled and fallback paths. nbd-server doesn't appear to support TLS at all, and qemu-nbd is known not to allow fallback to unencrypted, and therefore it only makes sense to test nbdkit at the moment. --- .gitignore | 4 ++++ TODO | 3 --- interop/Makefile.am | 54 +++++++++++++++++++++++++++++++++++++++++++++ interop/interop.c | 30

We want to default to requesting structured replies, whether or not that request will be honored (it's essential for efficient sparse file reads and the DF flag for structured pread, as well as for meta context support even if we do not request a default meta context). However, for integration testing, it can be nice to easily request a client that does not request structured replies. We can

This adds new APIs for running a local NBD server and connecting to it using systemd socket activation (instead of stdin/stdout). This includes interop tests against nbdkit and qemu-nbd which I believe are the only NBD servers supporting socket activation. (If we find others then we can add more interop tests in future.) The upstream spec for systemd socket activation is here:

This adds new APIs for running a local NBD server and connecting to it using systemd socket activation (instead of stdin/stdout). This includes interop tests against nbdkit and qemu-nbd which I believe are the only NBD servers supporting socket activation. (If we find others then we can add more interop tests in future.) The upstream spec for systemd socket activation is here:

The NBD protocol is adding an extension for servers to report the initial state of the image (for now, whether it is sparse and/or reads as all zeroes). Time to expose this to clients, via the new API nbd_get_init_flags(). The patch requires refreshing the nbd-protocol.h file from a corresponding contemporary nbdkit commit. Testing is possible with recent enough qemu. --- .gitignore

This simply compiles, passes tests, but is only able open a handle. This commit does not contain the full bindings. --- Makefile.am | 2 + configure.ac | 32 +++++ generator/GoLang.ml | 116 ++++++++++++++++++ generator/GoLang.mli | 19 +++ generator/Makefile.am

This simply compiles, passes tests, but is only able open a handle. This commit does not contain the full bindings. --- Makefile.am | 2 + configure.ac | 32 ++++ generator/GoLang.ml | 150 ++++++++++++++++++ generator/GoLang.mli | 19 +++ generator/Makefile.am

I was a little surprised to find that LIBNBD_TLS_ALLOW worked out of the box, so I had to examine the logs whereupon I saw the magic message ... libnbd: debug: nbd1: nbd_connect_command: server refused TLS (policy), continuing with unencrypted connection I don't believe this path has ever been tested before. It's possible the tests could be improved if they actually checked for this

It's worth testing that our transition to the DEAD state works as expected, by intentionally killing a server. This test also makes it possible to test what happens when pending commands are stranded, so that an upcoming patch to add notifiers can show the difference it makes. The test was surprisingly hard to write. For starters, sending SIGINT to nbdkit was sometimes enough to kill the

We use a similar trick to libvirt and libguestfs to allow external C programs that use libnbd to be compiled against the built (but not installed) libnbd with: ../libnbd/run ./configure make What actually happens is we have a second pkg-config file (lib/local/libnbd.pc) which points to the locally built libnbd. The ./run script sets up PKG_CONFIG_PATH to point to this directory. Assuming

Copies heavily after a similar addition recently made in nbdkit. --- docs/Makefile.am | 7 +++++++ docs/libnbd-security.pod | 32 ++++++++++++++++++++++++++++++++ docs/libnbd.pod | 1 + Makefile.am | 1 + .gitignore | 3 ++- SECURITY | 14 ++++++++++++++ 6 files changed, 57 insertions(+), 1 deletion(-) create mode 100644

Now that we have implemented systemd socket activation, we can use this to run qemu-nbd in tests. The first patch leaves some dead code around (the -DSERVE_OVER_TCP=1 path). It's possible we might want to use this to test against a putative future NBD server that only supports TCP, but on the other hand maybe we should just remove it. Tests & valgrind still pass for me. Rich.

Previously closures had a crude flag which tells if they are persistent or transient. Transient closures (flag = false) last for the lifetime of the currently called libnbd function. Persistent closures had an indefinite lifetime which could last for as long as the handle. In language bindings handling persistent closures was wasteful as we needed to register a "close callback" to

Copies heavily after a similar addition recently made in nbdkit. --- I'm not sure if .1 or .3 fits better for the man page. With nbdkit, .1 made sense because 'nbdkit' is a standalone program; but with libnbd, our only standalone is nbdsh, yet naming it nbdsh-security seems off. docs/Makefile.am | 7 +++++++ docs/libnbd-security.pod | 32 ++++++++++++++++++++++++++++++++

Instead of having to munge a URI to supply a different export name, we can exploit the fact that nbd_opt_mode lets us change the preferred export name after nbd_connect_uri has established its socket. In fact, by doing this, nbdinfo no longer needs to directly link against libxml2, so it can now be built regardless of whether the underlying libnbd.so supports URIs (although use of the program

On Fri, Sep 11, 2020 at 09:31:11AM -0500, Eric Blake wrote: > As mentioned in commits 176fc4ea and 609c25f0, our original plan in > adding a flags argument to nbd_shutdown was to let us specify > different behaviors at the libnbd level, rather than NBD protocol > flags (for that, the user has nbd_aio_disconnect). But when we later > parameterized OFlags to accept various bitmasks

--- This is what I used to provoke the deadlocks; before my patch series, it was succeeding for a fully-parallel server: nbdkit -U - memory 2M --run './deadlock $unixsocket' as well as for a serialized server that didn't trip up Linux kernel buffering limits: nbdkit -U - --filter=noparallel memory 256k --run './deadlock $unixsocket' but was reliably hanging with both client

Added merely as an example to be run by hand, rather than automated into 'make check', since nbdkit --filter=noparallel is quite new. --- .gitignore | 1 + examples/Makefile.am | 10 ++ examples/batched-read-write.c | 214 ++++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 examples/batched-read-write.c diff --git