search for: aio_flush

...eanup_fn rm -f test-parallel-file.data test-parallel-file.out # Populate file, and sanity check that qemu-io can issue parallel requests printf '%1024s' . > test-parallel-file.data -qemu-io -f raw -c "aio_write -P 1 0 512" -c "aio_write -P 2 512 512" \ - -c aio_flush test-parallel-file.data || +timeout 10s </dev/null qemu-io -f raw -c "aio_write -P 1 0 512" \ + -c "aio_write -P 2 512 512" -c aio_flush test-parallel-file.data || { echo "'qemu-io' can't drive parallel requests"; exit 77; } # Set up the fi...

...7; } +# Makefile sets $QEMU_IO, but it's also nice if the script runs again +# standalone afterwards for diagnosing any failures : ${QEMU_IO=qemu-io} -# Sanity check that qemu-io can issue parallel requests -$QEMU_IO -f raw -c "aio_write -P 2 1 1" -c "aio_read -P 1 0 1" -c aio_flush \ - file-data || { echo "'$QEMU_IO' can't drive parallel requests"; exit 77; } +trap 'rm -f test-parallel-file.data test-parallel-file.out' 0 1 2 3 15 + +# Populate file, and sanity check that qemu-io can issue parallel requests +printf '%1024s' . > test-pa...

...hen + echo "$0: missing file-data" + exit 77 +fi +if ! qemu-io --version; then + echo "$0: missing qemu-io" + exit 77 +fi # Sanity check that qemu-io can issue parallel requests -$QEMU_IO -f raw -c "aio_write -P 2 1 1" -c "aio_read -P 1 0 1" -c aio_flush \ - file-data || { echo "'$QEMU_IO' can't drive parallel requests"; exit 77; } +qemu-io -f raw -c "aio_write -P 2 1 1" -c "aio_read -P 1 0 1" -c aio_flush \ + file-data || { echo "'qemu-io' can't drive parallel requests"; exit 77; }...

I tried reproducing the testsuite failure on test-parallel-*.sh using the same machine Rich posted a log from, but did not quickly hit it on a loop of make -j20 check TESTS=test-parallel-{file,nbd}.sh But I still think this series can't hurt. Eric Blake (2): nbd: Don't advertise writes if nbdkit is readonly tests: Make parallel tests more robust plugins/nbd/nbd.c | 2 ++

Currently we test for qemu-img, socat, ss, certtool, etc at run time, but we test for qemu-io at compile time (in ./configure). This commit removes this inconsistency. I would consider the opposite patch (which makes qemu-img etc tested at configure time). The main advantage of testing for these binaries at run time is that tests are not "silently" omitted. Instead tests with

Either patch 1 or patch 2 in isolation is sufficient to fix the problem that Rich forwarded on from an archlinux tester (name so I can credit them?). But both patches should be applied, as well as backported to appropriate stable branches, to maximize cross-version interoperability of nbdkit vs. plugins. Patch 3 will let us detect future similar bugs much faster. I want patch 4 to ensure that

...heck that qemu-io can issue parallel requests printf '%1024s' . > test-parallel-file.data -$QEMU_IO -f raw -c "aio_write -P 1 0 512" -c "aio_write -P 2 512 512" \ +qemu-io -f raw -c "aio_write -P 1 0 512" -c "aio_write -P 2 512 512" \ -c aio_flush test-parallel-file.data || - { echo "'$QEMU_IO' can't drive parallel requests"; exit 77; } + { echo "'qemu-io' can't drive parallel requests"; exit 77; } # Set up the file plugin to delay both reads and writes (for a good chance # that parallel...

Small reworking of tests to remove $QEMU_IO, making that consistent with other test tools, and to test IPv6 connections.

...-parallel-sh.data test-parallel-sh.out test-parallel-sh.script + +# Populate file, and sanity check that qemu-io can issue parallel requests +printf '%1024s' . > test-parallel-sh.data +qemu-io -f raw -c "aio_write -P 1 0 512" -c "aio_write -P 2 512 512" \ + -c aio_flush test-parallel-sh.data || + { echo "'qemu-io' can't drive parallel requests"; exit 77; } + +# Set up the sh plugin to delay both reads and writes (for a good +# chance that parallel requests are in flight), and with writes longer +# than reads (to more easily detect if out-o...

...rallel-sh.out test-parallel-sh.script > + > +# Populate file, and sanity check that qemu-io can issue parallel requests > +printf '%1024s' . > test-parallel-sh.data > +qemu-io -f raw -c "aio_write -P 1 0 512" -c "aio_write -P 2 512 512" \ > + -c aio_flush test-parallel-sh.data || > + { echo "'qemu-io' can't drive parallel requests"; exit 77; } > + > +# Set up the sh plugin to delay both reads and writes (for a good > +# chance that parallel requests are in flight), and with writes longer > +# than reads (to mo...

...0203 sec (49.193231 bytes/sec and 49.1932 ops/sec) > /var/tmp/nbdkit-1.1.18/src/nbdkit -v -t 1 -U - > /var/tmp/nbdkit-1.1.18/plugins/file/.libs/nbdkit-file-plugin.so file=file-data rdelay=2 wdelay=1 --run > $QEMU_IO -f raw -c "aio_read 0 1" -c "aio_write -P 2 1 1" -c aio_flush $nbd > > nbdkit: file[1]: debug: handshake complete, processing requests serially When run serially, > read 1/1 bytes at offset 0 > 1 bytes, 1 ops; 0:00:02.00 (0.499933 bytes/sec and 0.4999 ops/sec) > wrote 1/1 bytes at offset 1 > 1 bytes, 1 ops; 0:00:03.01 (0.331378 bytes/sec...

This works OK on x86_64, but fails on our fast new Amberwing (aarch64) machine. I've attached the test-suite.log file, but I'm not very sure what's going wrong from that. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-df lists disk usage of guests without needing to

...ese parameters. =back diff --git a/tests/test-parallel-file.sh b/tests/test-parallel-file.sh index 79a60ac..b9e5f40 100755 --- a/tests/test-parallel-file.sh +++ b/tests/test-parallel-file.sh @@ -49,7 +49,7 @@ $QEMU_IO -f raw -c "aio_write -P 2 1 1" -c "aio_read -P 1 0 1" -c aio_flush \ trap 'rm -f test-parallel-file.out' 0 1 2 3 15 # With --threads=1, the write should complete first because it was issued first -nbdkit -v -t 1 -U - file file=file-data wdelay=2 rdelay=1 --run ' +nbdkit -v -t 1 -U - --filter delay file file=file-data wdelay=2 rdelay=1 --run '...

With this, I am finally able to get the nbd plugin to do out-of-order responses to the client. Once this series goes in, we should be ready for Rich to cut a release. Eric Blake (4): nbd: Split reading into separate thread nbd: Protect writes with mutex nbd: Enable parallel handling tests: Test parallel nbd behavior plugins/nbd/nbd.c | 217

...ot;, (1, 0); + "aio_connect_unix", (1, 0); + "aio_connect_tcp", (1, 0); + "aio_connect_command", (1, 0); + "aio_pread", (1, 0); + "aio_pread_structured", (1, 0); + "aio_pwrite", (1, 0); + "aio_disconnect", (1, 0); + "aio_flush", (1, 0); + "aio_trim", (1, 0); + "aio_cache", (1, 0); + "aio_zero", (1, 0); + "aio_block_status", (1, 0); + "aio_get_fd", (1, 0); + "aio_get_direction", (1, 0); + "aio_notify_read", (1, 0); + "aio_notify_write&...

This doesn't include Eric's new APIs, but if you push those then I can rebase this one on top. Rich.

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

...is will -return an error if L<nbd_can_flush(3)> is false. +generally return an error if L<nbd_can_flush(3)> is false. The C<flags> parameter must be C<0> for now (it exists for future NBD -protocol extensions)."; - see_also = [Link "can_flush"; Link "aio_flush"]; +protocol extensions)." +^ strict_call_description; + see_also = [Link "can_flush"; Link "aio_flush"; + Link "set_strict_mode"]; }; "trim", { @@ -1676,15 +1746,17 @@ Issue a trim command to the NBD server, which if suppor...

This is a major rewrite compared to my v1 series, where I've tried a lot harder to ensure that we still accommodate building on Haiku (although I have not actually yet fired up a Haiku VM to try it for myself). I also managed to make the sh plugin fully parallel, on capable platforms. See also my question on patch 10 on whether I've picked the best naming convention. Eric Blake (17):

...s the choice of MAX_NBD_REQUESTS used in qemu. One easy way to test: term1$ echo hello > junk term1$ ./nbdkit -f -v -r file file=junk rdelay=2s wdelay=1s term2$ qemu-io -f raw nbd://localhost:10809/ --trace='nbd_*' \ -c 'aio_read 0 1' -c 'aio_write -P 0x6c 2 2' -c 'aio_flush' If the write completes before the read, then nbdkit was properly handling things in parallel with out-of-order replies. Signed-off-by: Eric Blake <eblake@redhat.com> --- TODO | 7 ------- docs/nbdkit.pod | 12 +++++++++++- nbdkit.in | 2 +- plugins/file/...