search for: nbdcopi

This patch is a straightforward refactoring of libnbd?s nbdcopy program, and not very interesting. However I have plans for nbdcopy (see full todo below). I would like to use this utility for virt-v2v as a replacement for ?qemu-img convert?. qemu-img has caused us a series of problems: - change in zeroing behaviour caused a big performance regression - qemu-img reads extents up-front which

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 v1: https://listman.redhat.com/archives/libguestfs/2023-April/031258.html Please refer to the Notes section in each patch for the v2 updates. BR, Laszlo Laszlo Ersek (3): copy: rename DESTINATION_IS_ZERO_OPTION to TARGET_IS_ZERO_OPTION copy: fix layout of "long_options" table copy: rewrap error message about

Wrap "copy/nbd-ops.c" at 80 characters. I couldn't find a way to test that this change is unobservable. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 Signed-off-by: Laszlo Ersek <lersek at redhat.com> --- copy/nbd-ops.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/copy/nbd-ops.c b/copy/nbd-ops.c index d3e50864125f..843b7c1746e3

libnbd's copy/copy-nbd-error.sh was triggering an assertion failure in nbdkit: $ nbdcopy -- [ nbdkit --exit-with-parent -v --filter=error pattern 5M error-pread-rate=0.5 ] null: ... nbdkit: pattern.2: debug: error-inject: pread count=262144 offset=4718592 nbdkit: pattern.2: debug: pattern: pread count=262144 offset=4718592 nbdkit: pattern.1: debug: error-inject: pread count=262144

See companion patch: Subject: [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include This is the second part of the patch. It adds the new human_size_parse function to libnbd and then uses it for parsing --queue-size, --request-size and --sparse. The main complication here is that there was already a common/utils/human-size.h header which ends up (eventually)

On Thu, Mar 09, 2023 at 09:50:00AM +0000, Richard W.M. Jones wrote: > When a highly multi-threaded program such as nbdcopy encounters an > error, there is a race condition in the library which can cause an > assertion failure and thus a core dump: > > (1) An error occurs on one of the threads. nbdcopy calls exit(3). > > (2) In lib/errors.c, the destructor calls

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

On Thu, Mar 09, 2023 at 08:44:51AM +0000, Richard W.M. Jones wrote: > When a highly multi-threaded program such as nbdcopy encounters an > error, there is a race condition in the library which can cause an > assertion failure and thus a core dump: > > (1) An error occurs on one of the threads. nbdcopy calls exit(3). > > (2) In lib/errors.c, the destructor calls

When a highly multi-threaded program such as nbdcopy encounters an error, there is a race condition in the library which can cause an assertion failure and thus a core dump: (1) An error occurs on one of the threads. nbdcopy calls exit(3). (2) In lib/errors.c, the destructor calls pthread_key_delete. (3) Another thread which is still running also encounters an error, and inside libnbd the

This version simply removes the call to pthread_key_destroy. It fixes the problem and allows us to leave the asserts alone so we can still catch real errors. Of course this leaks pthread_key_t in the case where you dlclose() the library. glibc has a limit of 128 thread-specific keys (and the first 32 are somehow "fast" in a way I could quite follow), so that's a thing. Rich.

Wrap "copy/nbd-ops.c" at 80 characters. I couldn't find a way to test that this change is unobservable. Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 Acked-by: Richard W.M. Jones <rjones at redhat.com> Signed-off-by: Laszlo Ersek <lersek at redhat.com> --- Notes: v2: - pick up Rich's ACK copy/nbd-ops.c | 5 ++++- 1 file changed, 4

I'm out of time this weekend, but while trying to write a test for an nbdkit bug fix (a nasty assertion failure when the client disconnects uncleanly without NBD_CMD_DISC, which is what nbdcopy does if it gets an EIO read response from the server), I realized it is extremely hard to trigger this using nbdsh, but easy to do in C or the Go bindings. In the Go bindings, we intentionally coded

On 2/27/23 17:44, Richard W.M. Jones wrote: > On Mon, Feb 27, 2023 at 08:42:23AM -0600, Eric Blake wrote: >> Or intentionally choose a hash that can be computed out-of-order, such >> as a Merkle Tree. But we'd need a standard setup for all parties to >> agree on how the hash is to be computed and checked, if it is going to >> be anything more than just a linear hash

On Tue, Feb 28, 2023 at 12:24:04PM +0100, Laszlo Ersek wrote: > On 2/27/23 17:44, Richard W.M. Jones wrote: > > On Mon, Feb 27, 2023 at 08:42:23AM -0600, Eric Blake wrote: > >> Or intentionally choose a hash that can be computed out-of-order, such > >> as a Merkle Tree. But we'd need a standard setup for all parties to > >> agree on how the hash is to be

Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2172516 I figured I'd work on the libnbd line wrapping in shorter waves than how long the nbdkit patch series was. Laszlo Laszlo Ersek (4): copy: rename (LONG|SHORT)_OPTIONS to (LONG|SHORT)_OPTIONS_OPTION copy: rename <purpose>_OPTION to OPT_<purpose> copy: fix layout of "long_options" table copy: rewrap

This is the first part of a pair of patch series which aim to let us use nbdkit_parse_size (or rather, an equivalent common function) in nbdcopy, so we can write: nbdcopy --request-size=32M ... We can't do that now which was annoying me earlier in the week. This commit creates a new function called human_size_parse which is basically nbdkit_parse_size, and turns nbdkit_parse_size into a

The next commit will add a new "human-size.h" header copied in from nbdkit. This breaks existing code that uses common/utils/human-size.h (which we will combine later). As a temporary hack to maintain bisection, make sure we are using the deprecated "human-size.h" header. This hack will be removed later. --- copy/main.c | 2 +- info/show.c | 2 +- 2 files changed, 2

On Mon, Feb 27, 2023 at 08:42:23AM -0600, Eric Blake wrote: > Or intentionally choose a hash that can be computed out-of-order, such > as a Merkle Tree. But we'd need a standard setup for all parties to > agree on how the hash is to be computed and checked, if it is going to > be anything more than just a linear hash of the entire guest-visible > contents. Unfortunately I