similar to: [nbdkit PATCH] noextents: Add hook to cripple SR advertisement

The NBD protocol is adding an extension to let servers advertise initialization state to the client: whether the image contains holes, and whether it is known to read as all zeroes. Most filters just pass through the plugin's result (as it is only checked once when first connecting), but we can do useful things in log (mention the setting), extentlist (answer based on the extents we are

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 22 +++++++++++ docs/nbdkit-plugin.pod | 86 +++++++++++++++++++++++++++++++++++++++++ include/nbdkit-common.h | 14 ++++++- include/nbdkit-filter.h | 12 +++++- include/nbdkit-plugin.h | 6 ++- server/internal.h | 7 +++- server/filters.c |

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 45 +++++++++++++++++ docs/nbdkit-plugin.pod | 106 ++++++++++++++++++++++++++++++++++++++++ include/nbdkit-common.h | 18 ++++++- include/nbdkit-filter.h | 12 ++++- include/nbdkit-plugin.h | 6 ++- server/internal.h | 7 ++- server/extents.c

Similar to the existing fua, nozero and noextents filters, add a filter to make it easy to override the basic caching functionality, in part to facilitate timing tests of whether a plugin's cache implementation is worthwhile. A worthwhile test to add to the testsuite would connect the log filter both before and after the nocache filter, to prove how caching requests are altered. However,

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 64 +++++++++++++++++++++ docs/nbdkit-plugin.pod | 123 ++++++++++++++++++++++++++++++++++++++++ include/nbdkit-common.h | 20 ++++++- include/nbdkit-filter.h | 15 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 7 ++-

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 89 +++++++++++++++++++ docs/nbdkit-plugin.pod | 96 +++++++++++++++++++++ include/nbdkit-common.h | 10 ++- include/nbdkit-filter.h | 22 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + server/extents.c | 186

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 92 +++++++++++++++++++++ docs/nbdkit-plugin.pod | 99 +++++++++++++++++++++++ include/nbdkit-common.h | 10 ++- include/nbdkit-filter.h | 22 +++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + server/extents.c | 171

This new pair of callbacks allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 83 ++++++++++++++++ docs/nbdkit-plugin.pod | 97 +++++++++++++++++++ include/nbdkit-common.h | 10 +- include/nbdkit-filter.h | 22 ++++- include/nbdkit-plugin.h | 6 +- server/internal.h | 4 + server/extents.c | 210

This pair of calls allows plugins to describe which extents in the virtual disk are allocated, holes or zeroes. --- docs/nbdkit-filter.pod | 107 ++++++++ docs/nbdkit-plugin.pod | 103 +++++++ include/nbdkit-common.h | 10 +- include/nbdkit-filter.h | 25 +- include/nbdkit-plugin.h | 6 +- server/internal.h | 5 + server/extents.c | 574

Future patches want to allow a filter to pass a single opaque parameter into another framework (such as ext2fs_open) or even spawn a helper thread, which requires that nxdata be stable for the entire life of the connection between .open and .close. Our current approach of creating a stack-allocated nxdata for every call does not play nicely with that scheme, so rework things into using a

Anywhere that we compute 'f = container_of(b)' then use 'f->backend.', we can instead just use 'b->'. Similarly, during registration, we can use the 'filename' input rather than our just-copied 'f->filename'. Signed-off-by: Eric Blake <eblake@redhat.com> --- server/filters.c | 170 ++++++++++++++++++++++------------------------- 1 file

When we added support for .extents, we had nbdkit unconditionally support structured replies if the client requests them, and the plugin's .can_extents has no impact on what the server advertises. However, while the plugin API doesn't care whether the client requested SR, there are still integration situations where not advertising SR can be useful (such as comparison on what a client does

Since commit 86fdb48c6a5362d66865493d9d2172166f99722e we have stored the connection object in thread-local storage. In this very large, but mostly mechanical change we stop passing the connection pointer around everywhere, and instead use the value stored in thread-local storage. This assumes a 1-1 mapping between the connection and the current thread which is true in *most* places.

Until the next few patches expose and implement new callbacks for filters and plugins, our initial implementation for NBD_CMD_CACHE is to just blindly advertise the feature, and call into .pread with an ignored buffer. Note that for bisection reasons, this patch treats any use of a filter as a forced .can_cache of false to bypass the filter's caching; once all affected filters are patched to

Allow plugins to affect the handling of the new NBD_CMD_FLAG_FAST_ZERO flag, then update affected plugins. In particular, in-memory plugins are always fast; the full plugin is better served by omitting .zero and relying on .pwrite fallback; nbd takes advantage of libnbd extensions proposed in parallel to pass through support; and v2 language bindings expose the choice to their scripts. The

While our plugin code always advertises WRITE_ZEROES on writable connections (because we can emulate .zero by calling .pwrite), and FUA support when .flush exists (because we can emulate the persistence by calling .flush), it is conceivable that a filter may want to explicitly avoid advertising particular bits. (More to the point, I plan on writing a 'nozero' filter that hides

>From the protocol document: "NBD_FLAG_CAN_MULTI_CONN: Indicates that the server operates entirely without cache, or that the cache it uses is shared among all connections to the given device. In particular, if this flag is present, then the effects of NBD_CMD_FLUSH and NBD_CMD_FLAG_FUA MUST be visible across all connections when the server sends its reply to that command to

Both plugin.c and filter.c add the same three fields on top of struct backend, and use very similar code in initializing them. Let's stop the duplication, by moving those three fields into struct backend, and creating a new backend.c for manipulating them. In turn, we can drop the backend->name() accessor in favor of just directly accessing the name field. This is a net reduction in

We documented our intent of only allowing a filter to run with the same version of nbdkit it was compiled against, but up to now, were not actually enforcing that - we had only been insisting on the looser notion of a matching ._api_version, which doesn't help when we've forgotten to bump that macro when making incompatible API/ABI changes (see commit 6934d4c1). However, we can't use

With our recent cleanups to nxdata, the only remaining difference between functions like backend_open() and next_open() was the signature (one used void*, the other struct backend *); the API is compatible. All of our filters are in-tree, and we don't promise API/ABI stability, but it is still a lot of files to touch, so the simplest solution to avoid the redundant hop through wrapper