similar to: [libnbd PATCH] tests: Add test for abrupt server death

Some clients need to know when it is safe to issue NBD_CMD_DISC, or to decide whether calling poll(POLLIN) will block indefinitely because the server isn't expected to respond. Make this easier to learn by tracking the count of commands we have queued up to send, as well as the count of commands where we are waiting on the server's response. Update tests/aio-parallel* and

Added merely as an example to be run by hand, rather than automated into 'make check', since nbdkit --filter=noparallel is quite new. --- .gitignore | 1 + examples/Makefile.am | 10 ++ examples/batched-read-write.c | 214 ++++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 examples/batched-read-write.c diff --git

Automake insists that any file present in a Makefile.am 'include' must be present, but it has no idea about the GNU make extension 'sinclude'. Use this to our advantage: as long as Makefile.inc has nothing other than variable modifications (done in the previous patch), and our rules properly generate it when needed (thanks to generator_built), then running 'make' will

Having the client polling thread perform an O(n) loop over all known in-flight commands after each time the poll woke up is somewhat inefficient, and in a multi-threaded setup requires additional locking beyond libnbd to track the set of known command handles. Better is a way for aio commands to call a notify callback the moment a specific command is ready to complete, and then a separate thread

‘make clean && make’ was not rebuilding the docs/*.3 files. The reason is obscure: - docs/Makefile has rules: MANS = $(man_MANS) all: all-am all-am: Makefile $(MANS) - sinclude docs/Makefile.inc happened long after MANS is defined, so MANS held the earlier version of $(man_MANS) without the api-built man pages listed. This was confirmed by looking at the output of

On Fri, Sep 11, 2020 at 09:31:11AM -0500, Eric Blake wrote: > As mentioned in commits 176fc4ea and 609c25f0, our original plan in > adding a flags argument to nbd_shutdown was to let us specify > different behaviors at the libnbd level, rather than NBD protocol > flags (for that, the user has nbd_aio_disconnect). But when we later > parameterized OFlags to accept various bitmasks

--- This is what I used to provoke the deadlocks; before my patch series, it was succeeding for a fully-parallel server: nbdkit -U - memory 2M --run './deadlock $unixsocket' as well as for a serialized server that didn't trip up Linux kernel buffering limits: nbdkit -U - --filter=noparallel memory 256k --run './deadlock $unixsocket' but was reliably hanging with both client

We no longer have to track a linked list of in-flight transactions that are pending resolution, but rely instead on libnbd 0.1.6+ doing it on our behalf. Normally, we will get to call nbdplug_register() prior to the notify callback being reached, but under a heavily-loaded system, it is conceivable that the libnbd state machine can manage to fire off our request and receive a server reply all

Demonstrate a use of the new nbd_pread_structured_verify API by writing a strict validation that a server's structured replies comply with the specification (well, 99% strict, as I did not check that the server does not return an error at the same offset twice). I was able to test that qemu-nbd is compliant. An example run: $ qemu-img create -f qcow2 file 32m $ for i in `seq 32`; do

As mentioned in commits 176fc4ea and 609c25f0, our original plan in adding a flags argument to nbd_shutdown was to let us specify different behaviors at the libnbd level, rather than NBD protocol flags (for that, the user has nbd_aio_disconnect). But when we later parameterized OFlags to accept various bitmasks (commit f891340b), we failed to mark nbd_shutdown as using a different bitmask than

When using the nbd_aio_FOO_callback commands, there is nothing further to be learned about the command by calling nbd_aio_command_completed() compared to what the callback already had access to. There are still scenarios where manually retiring the command after the fact is useful (whether the return was 0 to keep the status unchanged, or -1 to alter the retirement status to *error), but by

--- .gitignore | 1 + examples/Makefile.am | 12 + examples/concurrent-writer.c | 450 +++++++++++++++++++++++++++++++++++ 3 files changed, 463 insertions(+) diff --git a/.gitignore b/.gitignore index 30438c1..e4dad91 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ Makefile.in /docs/libnbd-api.3 /docs/libnbd-api.pod /examples/batched-read-write

--- .gitignore | 1 + examples/Makefile.am | 12 + examples/concurrent-writer.c | 450 +++++++++++++++++++++++++++ examples/threaded-reads-and-writes.c | 2 +- 4 files changed, 464 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 30438c1..e4dad91 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ Makefile.in

Let's check for a quite a few more errors. Among other things, this adds some coverage for a few things I've patched recently. --- And these enhancements set me up for my next fix: making NBD_CMD_DISC prevent future commands. tests/errors.c | 167 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 154 insertions(+), 13 deletions(-) diff --git a/tests/errors.c

Reverts commit 387cbe67c3db27e8a61117fedb6e7fad76e409ef. --- generator/generator | 18 +++++++++++++++++- lib/handle.c | 28 +++++++++++++++++++++++++++- tests/closure-lifetimes.c | 4 +++- 3 files changed, 47 insertions(+), 3 deletions(-) diff --git a/generator/generator b/generator/generator index 2cd83f1..25e4aa5 100755 --- a/generator/generator +++ b/generator/generator

Rich recently patched things to generate one man page per function rather than libnbd-api.3 (nice), but in doing so got stumped by a problem with a fresh git clone (automake fails for any 'include' directive that does not already exist). I've figured out how to hack around it, but the hack requires GNU make. We already use GNU make constructs elsewhere (such as $(wildcard)), but

On 6/29/19 8:28 AM, Eric Blake wrote: > > We also want the client to be aware of any issued/in-flight commands > that failed because they were stranded when the state machine moved to > CLOSED or DEAD. Previously, nbd_aio_command_completed() would never > locate such stranded commands, but adding a common point to fire the > notifier for such commands makes it also possible to

It is finally time to introduce our first negotiating option command. With this change, we can now enter NEWSTYLE.START more than once; as such, it needs to know whether it is the first entry (proceed with gflags/cflags, TLS, and structured reply) or a later entry (all nbd_opt_* will cause an IssueCommand event to kick the state machine out of NEGOTIATING, at which point we want to jump to the

The original nbd_aio_* (non-callback) functions are removed and replaced with the renamed callback variants. This is a simple mechanical change to the API: (1) Any existing call to nbd_aio_*_callback can simply be renamed to nbd_aio_* (2) Any existing call to nbd_aio_* must have two extra NULL parameters added before the final flags parameter. In non-C languages, only change (1) is

On 7/25/19 8:07 AM, Richard W.M. Jones wrote: > Previously closures had a crude flag which tells if they are > persistent or transient. Transient closures (flag = false) last for > the lifetime of the currently called libnbd function. Persistent > closures had an indefinite lifetime which could last for as long as > the handle. In language bindings handling persistent closures