similar to: [PATCH libnbd v4] lib/errors.c: Fix assert fail in exit path in multi-threaded code

On Thu, Mar 09, 2023 at 09:50:00AM +0000, Richard W.M. Jones wrote: > When a highly multi-threaded program such as nbdcopy encounters an > error, there is a race condition in the library which can cause an > assertion failure and thus a core dump: > > (1) An error occurs on one of the threads. nbdcopy calls exit(3). > > (2) In lib/errors.c, the destructor calls

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

On Thu, Mar 09, 2023 at 08:44:51AM +0000, Richard W.M. Jones wrote: > When a highly multi-threaded program such as nbdcopy encounters an > error, there is a race condition in the library which can cause an > assertion failure and thus a core dump: > > (1) An error occurs on one of the threads. nbdcopy calls exit(3). > > (2) In lib/errors.c, the destructor calls

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

This version simply removes the call to pthread_key_destroy. It fixes the problem and allows us to leave the asserts alone so we can still catch real errors. Of course this leaks pthread_key_t in the case where you dlclose() the library. glibc has a limit of 128 thread-specific keys (and the first 32 are somehow "fast" in a way I could quite follow), so that's a thing. Rich.

This patch is a straightforward refactoring of libnbd?s nbdcopy program, and not very interesting. However I have plans for nbdcopy (see full todo below). I would like to use this utility for virt-v2v as a replacement for ?qemu-img convert?. qemu-img has caused us a series of problems: - change in zeroing behaviour caused a big performance regression - qemu-img reads extents up-front which

Wrap "copy/nbd-ops.c" at 80 characters. I couldn't find a way to test that this change is unobservable. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 Signed-off-by: Laszlo Ersek <lersek at redhat.com> --- copy/nbd-ops.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/copy/nbd-ops.c b/copy/nbd-ops.c index d3e50864125f..843b7c1746e3

libnbd's copy/copy-nbd-error.sh was triggering an assertion failure in nbdkit: $ nbdcopy -- [ nbdkit --exit-with-parent -v --filter=error pattern 5M error-pread-rate=0.5 ] null: ... nbdkit: pattern.2: debug: error-inject: pread count=262144 offset=4718592 nbdkit: pattern.2: debug: pattern: pread count=262144 offset=4718592 nbdkit: pattern.1: debug: error-inject: pread count=262144

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 v1: https://listman.redhat.com/archives/libguestfs/2023-April/031258.html Please refer to the Notes section in each patch for the v2 updates. BR, Laszlo Laszlo Ersek (3): copy: rename DESTINATION_IS_ZERO_OPTION to TARGET_IS_ZERO_OPTION copy: fix layout of "long_options" table copy: rewrap error message about

We have been inconsistent on how much we reject client-side without even consulting the server, vs. how much we depend on the server to detect failure (even if our request can be deemed undefined per NBD protocol). I'd like to change it so that by default, we reject as much as we can client-side for less traffic, but where the user can also change things on the fly for server-side integration

If the user calls nbd_kill_subprocess, we shouldn't kill the process again when we close the handle (since the process has likely gone and we might be killing a different process). --- lib/handle.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/lib/handle.c b/lib/handle.c index 2af25fe..5ad818e 100644 --- a/lib/handle.c +++ b/lib/handle.c @@ -315,6 +315,8 @@

Proposal for new APIs to list exports. The general shape of the API can probably best be seen from the examples/list-exports.c example. Rich.

This is a subset of my v2 posting, but limited to just the NBD_OPT_LIST handling. The biggest change since v2 is the addition of added unit testing in all four language bindings (C, python, ocaml, golang). The tests require nbdkit built from git on PATH, and may not be entirely idiomatic, but I at least validated that they catch issues (for example, adding an exit statement near the end of the

I still need to wire in the use of *_notify functions into nbdkit to prove whether it makes the code any faster or easier to maintain, but at least the added example shows one good use case for the new API. Eric Blake (6): api: Add nbd_aio_in_flight generator: Allow DEAD state actions to run generator: Allow Int64 in callbacks states: Prepare for aio notify callback api: Add new

v1 was posted here: https://www.redhat.com/archives/libguestfs/2019-September/thread.html#00337 v2: - Drop the first patch. - Hopefully fix the multiple issues with fork-safety and general behaviour on error paths. Note this requires execvpe for which there seems to be no equivalent on FreeBSD, except some kind of tedious path parsing (but can we assign to environ?) Rich.

stubdom: fix and clean pthread minimal support Signed-off-by: Samuel Thibault <samuel.thibault@eu.citrix.com> diff -r f4a293cbec1b extras/mini-os/include/posix/pthread.h --- a/extras/mini-os/include/posix/pthread.h Thu May 22 14:28:40 2008 +0100 +++ b/extras/mini-os/include/posix/pthread.h Thu May 22 16:08:29 2008 +0100 @@ -1,18 +1,56 @@ #ifndef _POSIX_PTHREAD_H #define _POSIX_PTHREAD_H

Since v2: - rebase to Rich's new API calls - more refactoring in patch 1 (retitled) - new patches 3 and 4 - fix data corruption in patch 6 (was 4) - more tweaks to the reproducer example (including using new API from 3) Eric Blake (7): lib: Refactor command_common() to do more common work commands: Allow for a command queue commands: Expose FIFO ordering of server completions

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

In v2: - now based on my proposal to add LIBNBD_SHUTDOWN_IMMEDIATE - four flags instead of two: STRICT_FLAGS is new (patch 4), and STRICT_BOUNDS is separate from STRICT_ZERO_SIZE (patch 5) - various refactorings for more shared code and less duplication Eric Blake (5): api: Add xxx_MASK constant for each Flags type generator: Refactor filtering of accepted OFlags api: Add