similar to: [PATCH nbdkit] tests: Remove QEMU_IO / HAVE_QEMU_IO.

Small reworking of tests to remove $QEMU_IO, making that consistent with other test tools, and to test IPv6 connections.

qemu-io 2.12 will be changing its default alignment to unknown servers so that it does read-modify-write for anything less than 512 bytes. If we implement NBD_OPT_GO, then we can keep qemu-io using 1-byte alignment; but until then, this breaks our parallel tests when using 1-byte alignment because they end up with more delays than expected (thanks to the read-modify-write). Revamp the tests to

I tried reproducing the testsuite failure on test-parallel-*.sh using the same machine Rich posted a log from, but did not quickly hit it on a loop of make -j20 check TESTS=test-parallel-{file,nbd}.sh But I still think this series can't hurt. Eric Blake (2): nbd: Don't advertise writes if nbdkit is readonly tests: Make parallel tests more robust plugins/nbd/nbd.c | 2 ++

On 03/06/2018 06:21 AM, Richard W.M. Jones wrote: > This is for consistency with qemu-img, socat, ss, etc where we test > for these binaries at run time. > --- > configure.ac | 4 ---- > tests/Makefile.am | 8 +++----- > tests/test-parallel-file.sh | 20 +++++++++++++------- > tests/test-parallel-nbd.sh | 20 +++++++++++++------- > 4 files

This is for consistency with qemu-img, socat, ss, etc where we test for these binaries at run time. --- configure.ac | 4 ---- tests/Makefile.am | 8 +++----- tests/test-parallel-file.sh | 21 +++++++++++++-------- tests/test-parallel-nbd.sh | 21 +++++++++++++-------- 4 files changed, 29 insertions(+), 25 deletions(-) diff --git a/configure.ac b/configure.ac index

This is for consistency with qemu-img, socat, ss, etc where we test for these binaries at run time. --- configure.ac | 4 ---- tests/Makefile.am | 8 +++----- tests/test-parallel-file.sh | 20 +++++++++++++------- tests/test-parallel-nbd.sh | 20 +++++++++++++------- 4 files changed, 29 insertions(+), 23 deletions(-) diff --git a/configure.ac b/configure.ac index

With this, I am finally able to get the nbd plugin to do out-of-order responses to the client. Once this series goes in, we should be ready for Rich to cut a release. Eric Blake (4): nbd: Split reading into separate thread nbd: Protect writes with mutex nbd: Enable parallel handling tests: Test parallel nbd behavior plugins/nbd/nbd.c | 217

This works OK on x86_64, but fails on our fast new Amberwing (aarch64) machine. I've attached the test-suite.log file, but I'm not very sure what's going wrong from that. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to

Either patch 1 or patch 2 in isolation is sufficient to fix the problem that Rich forwarded on from an archlinux tester (name so I can credit them?). But both patches should be applied, as well as backported to appropriate stable branches, to maximize cross-version interoperability of nbdkit vs. plugins. Patch 3 will let us detect future similar bugs much faster. I want patch 4 to ensure that

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

On Wed, Mar 07, 2018 at 08:39:05AM -0600, Eric Blake wrote: > On 03/06/2018 06:21 AM, Richard W.M. Jones wrote: > >This is for consistency with qemu-img, socat, ss, etc where we test > >for these binaries at run time. > >--- > > configure.ac | 4 ---- > > tests/Makefile.am | 8 +++----- > > tests/test-parallel-file.sh | 20

This is a major rewrite compared to my v1 series, where I've tried a lot harder to ensure that we still accommodate building on Haiku (although I have not actually yet fired up a Haiku VM to try it for myself). I also managed to make the sh plugin fully parallel, on capable platforms. See also my question on patch 10 on whether I've picked the best naming convention. Eric Blake (17):

Remove the need to use file= (and in future other) parameters for many plugins. eg. Using the file plugin becomes: nbdkit file disk.img Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2018-September/msg00024.html v2: - As discussed in the patch review, tighten up the characters permitted in keys. - Update documentation to note that relative paths can be made safe by prefixing with ./ and absolute paths do not need any extra steps. - I pushed patch 1/6 from the v1 series since it was just a trivial

v1: https://www.redhat.com/archives/libguestfs/2018-September/msg00024.html v2: https://www.redhat.com/archives/libguestfs/2018-September/msg00034.html v3: - Fixed is_config_key zero length test. - Fixed is_config_key so it uses strspn and is not O(n^2). - Changed >= 1.7 to >= 1.8 in the documentation. Rich.

If nbdkit has a bug (such as the nbd-standalone bug fixed in the previous commit), qemu-io ends up waiting forever rather than realizing that if the server disappears unexpectedly then qemu-io should quit. So add timeouts so the testsuite will flag the problem instead of hang (tested by reordering this commit before the previous). It's trickier than I expected: from the command line,

On 11/21/2017 03:21 PM, Richard W.M. Jones wrote: > This works OK on x86_64, but fails on our fast new Amberwing (aarch64) > machine. I've attached the test-suite.log file, but I'm not very sure > what's going wrong from that. > I'll see what I can spot... > FAIL: test-parallel-file.sh > =========================== > > read 1/1 bytes at offset 0 > 1

Previously the file plugin supported ‘rdelay’ and ‘wdelay’ parameters for injecting delays (for testing) into read and write requests. This moves the functionality to a new delay filter so that it can be used with any plugin. --- TODO | 3 - configure.ac | 1 + filters/Makefile.am | 1 + filters/delay/Makefile.am

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both

When we first created the sh plugin, we copied the thread model used in most other language bindings of SERIALIZE_ALL_REQUESTS, because it was easier to reason about and there was no way for a script to override our choice. However, we've since added support for scripts to request a tighter model when needed, and we've also just fixed remaining bugs about fds inadvertently leaked into the