search for: noparallel

...ch 1. This patch is written to apply after my NBD_CMD_CACHE work (since I touched the nocache filter); but can be rearranged if we think this series should go in first while that one undergoes any adjustments from review. Eric Blake (3): server: Allow filters to reduce thread model dynamically noparallel: Implement new filter filters: Use only .thread_model, not THREAD_MODEL docs/nbdkit-filter.pod | 45 ++++++---- filters/fua/nbdkit-fua-filter.pod | 1 + filters/nocache/nbdkit-nocache-filter.pod | 1 + filters/noextents/nbdkit-noextents-filter.pod | 1 +...

...--- filters/cow/cow.c | 2 -- filters/delay/delay.c | 2 -- filters/error/error.c | 2 -- filters/fua/fua.c | 2 -- filters/log/log.c | 2 -- filters/nocache/nocache.c | 2 -- filters/noextents/noextents.c | 2 -- filters/noparallel/noparallel.c | 2 -- filters/nozero/nozero.c | 2 -- filters/offset/offset.c | 2 -- filters/partition/partition.c | 2 -- filters/rate/rate.c | 2 -- filters/readahead/readahead.c | 2 -- filters/stats/stats.c | 2 -- filters/truncate/truncate.c...

...c | 1 - filters/delay/delay.c | 1 - filters/error/error.c | 1 - filters/fua/fua.c | 1 - filters/log/log.c | 1 - filters/nocache/nocache.c | 1 - filters/noextents/noextents.c | 1 - filters/noparallel/noparallel.c | 1 - filters/nozero/nozero.c | 1 - filters/offset/offset.c | 1 - filters/partition/partition.c | 1 - filters/rate/rate.c | 1 - filters/readahead/readahead.c | 1 - filters/stats/stats.c | 1 - filters/tr...

...concurrent clients to C<N>. This parameter is > +optional. If not specified then the limit defaults to 1. You can > +also set this to 0 to make the number of clients unlimited (ie. > +disable the filter). > + To some extent, you can also disable parallel connections with the noparallel-filter set to serialize=connections (but in that mode, the second client blocks waiting for its turn to connect, rather than the server dropping the second client immediately). Probably worth cross-linking the two man pages to highlight this difference in behavior. > +=head1 SEE ALSO > +...

--- This is what I used to provoke the deadlocks; before my patch series, it was succeeding for a fully-parallel server: nbdkit -U - memory 2M --run './deadlock $unixsocket' as well as for a serialized server that didn't trip up Linux kernel buffering limits: nbdkit -U - --filter=noparallel memory 256k --run './deadlock $unixsocket' but was reliably hanging with both client and server on larger buffers: nbdkit -U - --filter=noparallel memory 512k --run './deadlock $unixsocket' Post-patch, you can see the state machine now shift through ISSUE_COMMAND.PAUSE_WRITE_PAYLO...

Rich pointed out that making thread_model dynamic even for plugins makes some sense, so here is the code for doing it for 'sh'. I'm less confident on how to do it for OCaml and Rust (not to mention that those allow the plugin to already compile in their model, rather than the language binding glue forcing a model). The other languages (lua, perl, python, ruby) still need to be

...$sock" cleanup_fn rm -f $files fail=0 @@ -45,7 +45,7 @@ fail=0 # This tests that the delay filter's use of nbdkit_nanosleep is able to # react to both connection death and server shutdown without finishing # the entire delay duration. -start_nbdkit -P shutdown.pid -U $sock --filter=noparallel --filter=delay \ +start_nbdkit -P delay-shutdown.pid -U $sock --filter=noparallel --filter=delay \ null 1M serialize=connections rdelay=60 # Early client death should not stall connection of second client. -- 2.25.0

This is a second version of the limit filter. v1 was posted here: https://www.redhat.com/archives/libguestfs/2020-March/msg00015.html I didn't bother to repost the other three patches because they are the same. The difference is this version of the filter takes security more seriously. It shouldn't be possible for malicious clients to connect more than limit=N times to the plugin now,

...doesn't get to run until the core has learned that the client wants a command serviced). But it may be possible to tweak things to send back-to-back write requests, where even if the first write request gets sent completely, the plugin can delay responding to that first write and use --filter=noparallel to prevent the second command from reaching nbdkit. I'll play with that, to see if I can reproduce the valgrind race, as well as work around it with back-to-back write commands to increase the likelihood of actually preventing nbdkit from consuming the second command. -- Eric Blake, Principa...

...h | \ + sed 's/^thread_model=parallel/thread_model=serialize_all_requests/' # Here, the script further reduces things nbdkit --dump-plugin sh - <<\EOF case $1 in @@ -85,13 +86,13 @@ EOF # Here, the filter further reduces things nbdkit --dump-plugin --filter=noparallel sh serialize=connections } | grep thread_model) -exp="max_thread_model=serialize_all_requests +exp="max_thread_model=parallel thread_model=serialize_all_requests has_thread_model=1 -max_thread_model=serialize_all_requests +max_thread_model=parallel thread_model=serialize_connections...

This might not be the final solution, but it certainly seems to solve a deadlock for me that I could trigger by using 'nbdkit --filter=noparallel memory 512k' and calling nbd_aio_pread for a request larger than 256k (enough for the Linux kernel to block the server until libnbd read()s), immediately followed by nbd_aio_pwrite for a request larger than 256k (enough to block libnbd until the server read()s, but the serialized server won'...

...-io -f raw \ + -c "aio_write -P 2 512 512" -c "aio_read -P 1 0 512" -c aio_flush $nbd' | tee test-parallel-file.out if test "$(grep '512/512' test-parallel-file.out)" != \ "read 512/512 bytes at offset 0 @@ -77,8 +78,8 @@ fi # With --filter=noparallel, the write should complete first because it was issued first nbdkit -v -U - --filter=noparallel --filter=delay file test-parallel-file.data \ - wdelay=2 rdelay=1 --run 'qemu-io -f raw -c "aio_write -P 2 512 512" \ - -c "aio_read -P 1 0 512" -c aio...

Added merely as an example to be run by hand, rather than automated into 'make check', since nbdkit --filter=noparallel is quite new. --- .gitignore | 1 + examples/Makefile.am | 10 ++ examples/batched-read-write.c | 214 ++++++++++++++++++++++++++++++++++ 3 files changed, 225 insertions(+) create mode 100644 examples/batched-read-write.c diff --git a/.gitignore b/.gitignore index...

...ead_model=parallel/thread_model=serialize_all_requests/' > # Here, the script further reduces things > nbdkit --dump-plugin sh - <<\EOF > case $1 in > @@ -85,13 +86,13 @@ EOF > # Here, the filter further reduces things > nbdkit --dump-plugin --filter=noparallel sh serialize=connections > } | grep thread_model) > -exp="max_thread_model=serialize_all_requests > +exp="max_thread_model=parallel > thread_model=serialize_all_requests > has_thread_model=1 > -max_thread_model=serialize_all_requests > +max_thread_model=parallel &g...

...ilters/gzip/Makefile.am | 1 + filters/ip/Makefile.am | 1 + filters/limit/Makefile.am | 1 + filters/log/Makefile.am | 1 + filters/nocache/Makefile.am | 3 +++ filters/noextents/Makefile.am | 3 +++ filters/nofilter/Makefile.am | 3 +++ filters/noparallel/Makefile.am | 3 +++ filters/nozero/Makefile.am | 3 +++ filters/offset/Makefile.am | 1 + filters/partition/Makefile.am | 1 + filters/pause/Makefile.am | 1 + filters/rate/Makefile.am | 1 + filters/readahead/Makefile.am | 1 + filters/retry/Makefile.am...

...ate a server that delays reads and forces only one connection at a time. +# This tests that the delay filter's use of nbdkit_nanosleep is able to +# react to both connection death and server shutdown without finishing +# the entire delay duration. +start_nbdkit -P shutdown.pid -U $sock --filter=noparallel --filter=delay \ + null 1M serialize=connections rdelay=10 + +# Early client death should not stall connection of second client. +trap '' ERR +timeout 1s qemu-io -f raw "nbd+unix:///?socket=$sock" -c 'r 0 512' </dev/null +test $? = 124 || { + echo "Unexpected...

...he user to enable multi-conn on the > handle and then use nbd_connect_command, but I did not make that > change yet. Ouch - I just realized that we created a different problem. If the server does not add multi-conn because it does NOT permit parallel connections (such as 'nbdkit --filter=noparallel memory 1m serialize=connections'), then a client requesting nbd_set_multi_conn(nbd, 2) will now hang because we will never be able to move all our connections through handshake. While we can still try to fire off multiple connections at once, we need to add some logic in nbd_connect_unix() and...

Either patch 1 or patch 2 in isolation is sufficient to fix the problem that Rich forwarded on from an archlinux tester (name so I can credit them?). But both patches should be applied, as well as backported to appropriate stable branches, to maximize cross-version interoperability of nbdkit vs. plugins. Patch 3 will let us detect future similar bugs much faster. I want patch 4 to ensure that

...can be used to specify an NBD URI in ‘--run’ parameters (Eric Blake). New ‘stats’ filter displays elapsed statistics about NBD operations, such as the number of bytes read and written. New ‘nocache’ filter which disables cache requests, used to determine how effective they are (Eric Blake). New ‘noparallel’ filter which can be used to override the plugin’s own choice of thread model. Used to determine how the thread model affects performance, or to serialize plugins if required (Eric Blake). New ‘cacheextents’ filter to cache extents requests, especially useful with VDDK which has a slow implementa...

...That script could send email to an administrator, or even create a filesystem on the user's behalf; then run that filter in front of the file plugin to let the user create images after all. The exclusion between clients is an interesting trick to figure out, though (the existing --filter=noparallel is too strict - you want parallel clients visiting different files, but not parallel clients visiting the same file). > +=head2 Security considerations > + > +You should B<only> use this in an environment where you trust all your > +clients, since clients can use this plugin to...