similar to: [nbdkit PATCH] Experiment: nbd: Use ppoll() instead of pipe-to-self

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

Since Eric's improvements to the command queue in commit 6af72b8 (and following) there's now a queue of commands waiting to be issued stored in the handle, and there's no need to issue commands only from the ready state. We can therefore remove the want_to_send logic, queue as many commands as possible, and don't need to test if the socket is ready for POLLOUT. This commit also

On 5/31/19 11:01 AM, Richard W.M. Jones wrote: > This is a continuation of a discussion we were having on IRC. The > problems with IRC are it's not recorded and it's hard to have deep > technical conversations. I hope this is a decent summary. > > Problem simply stated: Certain NBD servers (qemu-nbd in particular) Also nbdkit for a plugin with

We no longer have to track a linked list of in-flight transactions that are pending resolution, but rely instead on libnbd 0.1.6+ doing it on our behalf. Normally, we will get to call nbdplug_register() prior to the notify callback being reached, but under a heavily-loaded system, it is conceivable that the libnbd state machine can manage to fire off our request and receive a server reply all

--- This is what I used to provoke the deadlocks; before my patch series, it was succeeding for a fully-parallel server: nbdkit -U - memory 2M --run './deadlock $unixsocket' as well as for a serialized server that didn't trip up Linux kernel buffering limits: nbdkit -U - --filter=noparallel memory 256k --run './deadlock $unixsocket' but was reliably hanging with both client

Added merely as an example to be run by hand, rather than automated into 'make check', since nbdkit --filter=noparallel is quite new. --- .gitignore | 1 + examples/Makefile.am | 10 ++ examples/batched-read-write.c | 214 ++++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 examples/batched-read-write.c diff --git

klibc fails to build on ia64 because it has neither fork nor the ppoll syscalls. This patch marks the former in SYSCALLS.def, and puts a guard wrapper around the ppoll definition in ppoll.c Should these calls return ENOSYS rather than simply not existing on some systems? If yes, please apply this and I'll follow up with a patch shortly that does ENOSYS for this and pselect at least.

--- .gitignore | 1 + examples/Makefile.am | 12 + examples/concurrent-writer.c | 450 +++++++++++++++++++++++++++ examples/threaded-reads-and-writes.c | 2 +- 4 files changed, 464 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index 30438c1..e4dad91 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ Makefile.in

--- .gitignore | 1 + README | 2 + configure.ac | 9 + examples/Makefile.am | 22 ++ examples/glib-main-loop.c | 511 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 545 insertions(+) diff --git a/.gitignore b/.gitignore index ce02aef..9a8ba37 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ Makefile.in

--- .gitignore | 1 + README | 2 + configure.ac | 9 + examples/Makefile.am | 22 ++ examples/glib-main-loop.c | 501 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 535 insertions(+) diff --git a/.gitignore b/.gitignore index edbf941..79b95b5 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ Makefile.in

Mention things to remember for avoiding the deadlock of polling for a POLLIN from a server that has no replies to send. Perhaps we could split the READY state into two - one for when there are no commands in flight (and get_direction returns 0 to state that polling is pointless, although a multi-threaded reader can still poll for POLLIN), and the other when there ARE commands in flight. Such a

--- .gitignore | 1 + examples/Makefile.am | 12 + examples/concurrent-writer.c | 450 +++++++++++++++++++++++++++++++++++ 3 files changed, 463 insertions(+) diff --git a/.gitignore b/.gitignore index 30438c1..e4dad91 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ Makefile.in /docs/libnbd-api.3 /docs/libnbd-api.pod /examples/batched-read-write

When we first used libnbd (nbdkit 1.14), it was brand new, so we left in the ability to build a limited nbd plugin without the use of libnbd for comparison. But now that libnbd has had more time under its belt, and we have learned that our nbd-standalone code is not getting much testing, it's time to follow through with our plan to finish off the old code, and now build the nbd plugin only

This is a continuation of a discussion we were having on IRC. The problems with IRC are it's not recorded and it's hard to have deep technical conversations. I hope this is a decent summary. Problem simply stated: Certain NBD servers (qemu-nbd in particular) are able to simultaneously read and write on a socket. ie. They can be simultaneously reading a request and writing the reply to

ThreadSanitizer [1] pointed out that in the nbd plugin, nbd_close() can attempt close() in the main thread while the worker thread is still attempting to start a read(). Normally, if the read() loses the race, it will get a harmless EBADF that exits the worker thread (which is what we want, as we are closing the connection anyway); but if another connection happens to start in that window, we

Some clients need to know when it is safe to issue NBD_CMD_DISC, or to decide whether calling poll(POLLIN) will block indefinitely because the server isn't expected to respond. Make this easier to learn by tracking the count of commands we have queued up to send, as well as the count of commands where we are waiting on the server's response. Update tests/aio-parallel* and

There are a couple of problems with filters trying to sleep. First, when it is time to shut down nbdkit, we wait until all pending transactions have had a chance to wind down. But consider what happens if one or more of those pending transactions are blocked in a sleep. POSIX says nanosleep is interrupted with EINTR if that thread handles a signal, but wiring up signal masks just to ensure a

Your analysis looks pretty complete to me and your solutions seems plausible. That said, I don't know that I would have the level of confidence yet that we haven't missed an important point that I would want before going down this route. Losing stack checking is risky; it might be eventually possible to provide some support for this to be handled via a thread-local variable. Ensuring

It's not necessary to cache this, and the libnbd API doesn't guarantee that it always stays the same. --- plugins/nbd/nbd.c | 7 +------ 1 file changed, 1 insertion(+), 6 deletions(-) diff --git a/plugins/nbd/nbd.c b/plugins/nbd/nbd.c index 20c06e2a..8997174e 100644 --- a/plugins/nbd/nbd.c +++ b/plugins/nbd/nbd.c @@ -71,7 +71,6 @@ struct transaction { struct handle { /* These fields