similar to: [nbdkit PATCH 0/3] Add noparallel filter

No need for filters to specify their model in two different ways; the callback alone is sufficient. Since all filters are in-tree, we can deal with the API/ABI change made here. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-filter.pod | 48 +++++++++++++++------------------ include/nbdkit-filter.h | 6 ++--- filters/cow/cow.c | 2 --

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack. Eric Blake (24): server: Internal hooks for implementing

We documented our intent of only allowing a filter to run with the same version of nbdkit it was compiled against, but up to now, were not actually enforcing that - we had only been insisting on the looser notion of a matching ._api_version, which doesn't help when we've forgotten to bump that macro when making incompatible API/ABI changes (see commit 6934d4c1). However, we can't use

This is a second version of the limit filter. v1 was posted here: https://www.redhat.com/archives/libguestfs/2020-March/msg00015.html I didn't bother to repost the other three patches because they are the same. The difference is this version of the filter takes security more seriously. It shouldn't be possible for malicious clients to connect more than limit=N times to the plugin now,

This creates libnbdkit.so as discussed in the following thread: https://www.redhat.com/archives/libguestfs/2020-March/thread.html#00203 test-delay-shutdown.sh fails for unclear reasons. This series starts by reverting "tests: Don't strand hung nbdkit processes" which is because several other tests fail randomly unless I revert this patch. I didn't investigate this yet so it

Rich pointed out that making thread_model dynamic even for plugins makes some sense, so here is the code for doing it for 'sh'. I'm less confident on how to do it for OCaml and Rust (not to mention that those allow the plugin to already compile in their model, rather than the language binding glue forcing a model). The other languages (lua, perl, python, ruby) still need to be

This is a cover letter to a series of patches being proposed in tandem to four different projects: - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag - qemu: Implement the flag for both clients and server - libnbd: Implement the flag for clients - nbdkit: Implement the flag for servers, including the nbd passthrough client If you want to test the patches together, I've pushed a

When I added .can_FOO caching in 1.16, I missed the case that the sh plugin itself was calling .can_flush twice in some situations (in order to default .can_fua). Then right after, I regressed it to call .can_zero twice (in order to default .can_fast_zero). I also missed that .thread_model could use better caching, because at the time, I did not add testsuite coverage. Fix that now. Eric Blake

Either patch 1 or patch 2 in isolation is sufficient to fix the problem that Rich forwarded on from an archlinux tester (name so I can credit them?). But both patches should be applied, as well as backported to appropriate stable branches, to maximize cross-version interoperability of nbdkit vs. plugins. Patch 3 will let us detect future similar bugs much faster. I want patch 4 to ensure that

This adds a new nbdkit_shutdown() API whereby plugins and filters can request that the server shuts down (asynchronously) during the serving phase. Two new filters are added, one of which depends on this feature and the other not needing it but being somewhat related. Rich.

I still need to add testsuite coverage. Perhaps it might be easier if I create a new '--filter=log logfile=foo' filter that produces a log of which commands a client sent, then compare the log using a known client that uses write_zeroes (qemu-io works well) both with and without --filter=nozero to prove that the change in advertisement changes the commands sent over the wire (that would

Here's changes to the file plugin (which I'm happy with) and a new exportname filter (which is still at RFC stage; I need to finish implementing strict mode in .open, and add tests). I also discovered that we really want .list_exports and .open to know when they are used on plaintext vs. tls clients for --tls=on, and we may want to split out a new .default_export callback rather than

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]

It's easy to use the sh script to demonstrate that nbdkit is inefficiently calling into .get_size, .can_fua, and friends more than necessary. We've also commented on the list in the past that it would be nice to ensure that when filters call into next_ops, they are not violating constraints (as we've have to fix several bugs in the past where we did not have such checking to protect

Similar to the existing fua, nozero and noextents filters, add a filter to make it easy to override the basic caching functionality, in part to facilitate timing tests of whether a plugin's cache implementation is worthwhile. A worthwhile test to add to the testsuite would connect the log filter both before and after the nocache filter, to prove how caching requests are altered. However,

tmpfs has a known bug of O(n^2) behavior with lseek(SEEK_HOLE); this is one situation where the noextents filter can come in handy to avoid the performance penalty of exposing accurate extents. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/noextents/nbdkit-noextents-filter.pod | 7 ++++++- plugins/file/nbdkit-file-plugin.pod | 9 ++++++++- 2 files changed, 14

With these patches, a majority of tests pass. The notable things which are still broken: - Because FreeBSD links /home -> /usr/home, $(pwd) gives a different result from realpath(2). Therefore some tests which implicitly rely on (eg) a plugin which calls nbdkit_realpath internally and then checking that path against $(pwd) fail. - Shebangs (#!) don't seem to work the same way

This is a subset of the last half of the larger 9-patch series. The uncontroversial first half of that series is pushed, but here, I tried to reduce the size of the patches by splitting out some of the more complex changes, so that the rest of the changes remaining in the series are more mechanical. In turn, it forced me to write timing tests, which let me spot another spot where we are wasting

Remove the need to use file= (and in future other) parameters for many plugins. eg. Using the file plugin becomes: nbdkit file disk.img Rich.

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]