similar to: no way to force-close NBD handle in nbdsh

On Fri, Feb 24, 2023 at 04:41:20PM -0600, Eric Blake wrote: > I'm out of time this weekend, but while trying to write a test for an > nbdkit bug fix (a nasty assertion failure when the client disconnects > uncleanly without NBD_CMD_DISC, which is what nbdcopy does if it gets > an EIO read response from the server), I realized it is extremely hard > to trigger this using nbdsh,

On Sat, Feb 25, 2023 at 09:57:41AM +0000, Richard W.M. Jones wrote: > I can have a go at adding this since I unexpectedly have some free > time this weekend. https://gitlab.com/nbdkit/libnbd/-/commit/b7f85610ac19085a1631439fbcb7dd241f1e9ebf https://gitlab.com/nbdkit/libnbd/-/commit/71757015ea15815b5ff1c04ef482152b089eb305

This patch is a straightforward refactoring of libnbd?s nbdcopy program, and not very interesting. However I have plans for nbdcopy (see full todo below). I would like to use this utility for virt-v2v as a replacement for ?qemu-img convert?. qemu-img has caused us a series of problems: - change in zeroing behaviour caused a big performance regression - qemu-img reads extents up-front which

Wrap "copy/nbd-ops.c" at 80 characters. I couldn't find a way to test that this change is unobservable. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 Signed-off-by: Laszlo Ersek <lersek at redhat.com> --- copy/nbd-ops.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/copy/nbd-ops.c b/copy/nbd-ops.c index d3e50864125f..843b7c1746e3

libnbd's copy/copy-nbd-error.sh was triggering an assertion failure in nbdkit: $ nbdcopy -- [ nbdkit --exit-with-parent -v --filter=error pattern 5M error-pread-rate=0.5 ] null: ... nbdkit: pattern.2: debug: error-inject: pread count=262144 offset=4718592 nbdkit: pattern.2: debug: pattern: pread count=262144 offset=4718592 nbdkit: pattern.1: debug: error-inject: pread count=262144

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

I'm pleased to announce the release of libnbd 1.4.0, a high performance Network Block Device (NBD) client library. Key features of libnbd: * Synchronous API for ease of use. * Asynchronous API for writing non-blocking, multithreaded clients. You can mix both APIs freely. * High performance. * Minimal dependencies for the basic library. * Well-documented, stable API. * Bindings

When using nbdsh for scripting, it is convenient to let nbdsh fail with status 1 when encountering an API failure. However, doing so by letting the nbd.Error exception leak all the way causes ABRT (on Fedora 32 at least) to assume the program crashed from a programming error, and needlessly complicates clients to have to add try: blocks. Better is if nbdsh itself prints the same stack trace that

On Tue, Sep 22, 2020 at 11:15:39AM -0500, Eric Blake wrote: > + try: > + for c in args.command: > + if c != '-': > + exec(c, d, d) > + else: > + exec(sys.stdin.read(), d, d) > + except nbd.Error as ex: > + if os.environ.get("LIBNBD_DEBUG", "0")

When using nbdsh for scripting, it is convenient to let nbdsh fail with status 1 when encountering an API failure. However, doing so by letting the nbd.Error exception leak all the way causes ABRT (at least on Fedora 32 with abrt-python3-handler installed) to assume the program crashed from a programming error, and needlessly complicates clients to have to add try: blocks. Better is if nbdsh

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 v1: https://listman.redhat.com/archives/libguestfs/2023-April/031258.html Please refer to the Notes section in each patch for the v2 updates. BR, Laszlo Laszlo Ersek (3): copy: rename DESTINATION_IS_ZERO_OPTION to TARGET_IS_ZERO_OPTION copy: fix layout of "long_options" table copy: rewrap error message about

Wrap "copy/nbd-ops.c" at 80 characters. I couldn't find a way to test that this change is unobservable. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 Acked-by: Richard W.M. Jones <rjones at redhat.com> Signed-off-by: Laszlo Ersek <lersek at redhat.com> --- Notes: v2: - pick up Rich's ACK copy/nbd-ops.c | 5 ++++- 1 file changed, 4

On Thu, Mar 09, 2023 at 09:50:00AM +0000, Richard W.M. Jones wrote: > When a highly multi-threaded program such as nbdcopy encounters an > error, there is a race condition in the library which can cause an > assertion failure and thus a core dump: > > (1) An error occurs on one of the threads. nbdcopy calls exit(3). > > (2) In lib/errors.c, the destructor calls

This is a cover letter for a set of multi-project patch series all designed to implement 64-bit operations in NBD, and demonstrate interoperability of the new extension between projects. v1 of the project was attempted nearly a year ago: https://lists.nongnu.org/archive/html/qemu-devel/2021-12/msg00453.html Since then, I've addressed a lot of preliminary cleanups in libnbd to make it easier

ThreadSanitizer [1] pointed out that in the nbd plugin, nbd_close() can attempt close() in the main thread while the worker thread is still attempting to start a read(). Normally, if the read() loses the race, it will get a harmless EBADF that exits the worker thread (which is what we want, as we are closing the connection anyway); but if another connection happens to start in that window, we

On 2/27/23 17:44, Richard W.M. Jones wrote: > On Mon, Feb 27, 2023 at 08:42:23AM -0600, Eric Blake wrote: >> Or intentionally choose a hash that can be computed out-of-order, such >> as a Merkle Tree. But we'd need a standard setup for all parties to >> agree on how the hash is to be computed and checked, if it is going to >> be anything more than just a linear hash

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

Typing nbdsh --con (the minimum to get an unambiguous prefix for --connect, different from --command) is annoying compared to having a short option. Since it takes a URI as an argument, using -u/--uri is a nicer mnemonic. We still accept --connect for back-compat, and document it in the man page, but omit it from --help as the new spelling is nicer all around. --- Here's what things evolved

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

Typing nbdsh --con (the minimum to get an unambiguous prefix for --connect, different from --command) is annoying compared to having a short option. Since it takes a URI as an argument, using -u seems like a reasonable mnemonic. --- sh/nbdsh.pod | 6 ++++-- python/nbdsh.py | 2 +- sh/test-context.sh | 8 ++++---- sh/test-pattern.sh | 2 +- 4 files changed, 10 insertions(+), 8