similar to: [PATCH nbdkit] plugins, filters: Define and use NBDKIT_HANDLE_NOT_NEEDED.

The blocksize filter already rounds sizes down, to avoid having to special-case a partial action at the end of an unaligned file. But our current behavior for cache and cow is not similarly careful, and can end up causing errors by passing a too-large count to the plugin. The bulk of our plugins are somewhat safe, in that they either fail the pread (for example, the file plugin gets a short read

No need for filters to specify their model in two different ways; the callback alone is sufficient. Since all filters are in-tree, we can deal with the API/ABI change made here. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-filter.pod | 48 +++++++++++++++------------------ include/nbdkit-filter.h | 6 ++--- filters/cow/cow.c | 2 --

The cache filter .open has never done any useful work; since its introduction in c10d126f, it appears to exist purely as copy-and-paste from the cow filter (differing only in whether the readonly parameter is massaged before passing on to next). The error filter .open used to track a per-connection handle, but for testing purposes, it was changed to use only global state in commit b33ccbb8. Drop

To allow filters to modify the export name as it passes through the layers this commit makes several changes: The filter .open callback now takes an extra parameter, the export name. This is always non-NULL (for oldstyle it is ""). This string has a short lifetime and filters that need to hang on to it must take a copy. The filter must pass the exportname parameter down to the next

'nbdkit -v' is quite verbose, and traces everything. Sometimes, we want to trace JUST the client interactions. In particular, when debugging another filter, being able to trace what the filter called can be quite useful; and having the log filter in place will make it easier to add testsuite coverage of other filters. Also, it is nice to have timestamps in the log, in order to see if

Although the time spent in memcpy/memset probably pales in comparison to time spent in socket I/O, it's still worth worth reducing the number of times we have to utilize a bounce buffer when we already have aligned data. Note that blocksize, cache, and cow all do block fragmentation and bounce-buffer alignment; this brings the logic in cache and cow (which were copied from one another) more

Filters can be placed in front of plugins to modify their behaviour. This commit adds the <nbdkit-filter.h> header file, the manual page, the ‘filterdir’ directory (like ‘plugindir’), the ‘filters/’ source directory which will contain the actual filters, the ‘--filters’ parameter, and the filters backend logic. --- Makefile.am | 2 +- TODO | 17 +-

These are used at various places in the code already. This refactoring simply moves them to a common header file and should have no other effect. --- common/include/rounding.h | 59 +++++++++++++++++++++++++++++++++++++ filters/cache/Makefile.am | 3 +- filters/cache/cache.c | 2 +- filters/cow/Makefile.am | 3 +- filters/cow/cow.c | 2 +- filters/truncate/truncate.c

Here's the patch (on top of the preceeding one) which uses a bitmap instead of SEEK_DATA. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org --4VrXvz3cwkc87Wze

Make the implicit lock explicit, and hold it around blk_* operations. This allows us to relax the thread model for the filter to NBDKIT_THREAD_MODEL_PARALLEL. --- filters/cache/blk.h | 7 ++++++ filters/cache/cache.c | 57 +++++++++++++++++++++++++++++++------------ 2 files changed, 49 insertions(+), 15 deletions(-) diff --git a/filters/cache/blk.h b/filters/cache/blk.h index 24bf6a1..ab9134e

Now that we can differentiate content based on export name, we also need to be able to differentiate content for a --tls=on server, since TLS is optional according to whether the client has authenticated. For internal code and filters, this means adding a new parameter; the sh plugin can do likewise. For plugins, we can't add a parameter until the V3 protocol, so in the meantime, we add

This method can be used for plugins to get control after the server has forked and changed user but before it accepts a connection. This is very late and the only real use for this is for a plugin to create background threads for its own use. --- docs/nbdkit-filter.pod | 20 +++++++++++++++----- docs/nbdkit-plugin.pod | 27 ++++++++++++++++++++++++++- include/nbdkit-filter.h | 2 ++

--- configure.ac | 5 + filters/Makefile.am | 4 + filters/cow/Makefile.am | 65 +++++++ filters/cow/cow.c | 392 ++++++++++++++++++++++++++++++++++++++ filters/cow/nbdkit-cow-filter.pod | 162 ++++++++++++++++ tests/Makefile.am | 6 + tests/test-cow.sh | 98 ++++++++++ 7 files changed, 732

Now that cleanup.h is in common code, we can use it in our filters where it makes sense. Many uses of pthread_mutex_unlock() are not function-wide, and over small enough snippets of code as to be easier to read when left as-is; but when the scope is indeed function-wide or across multiple exit paths, it is nicer to use the macro for automatic unlock. Signed-off-by: Eric Blake

The cow and cache filters both use a bitmap mapping virtual disk blocks to status stored in the bitmap. The implementation of the bitmaps is very similar because one was derived from the other when the filters were implemented. The main difference is the cow filter uses a simple bitmap (one bit per block), whereas the cache filter uses two bits per block. This commit abstracts the bitmap

The cow and cache filters both use a bitmap mapping virtual disk blocks to status stored in the bitmap. The implementation of the bitmaps is very similar because one was derived from the other when the filters were implemented. The main difference is the cow filter uses a simple bitmap (one bit per block), whereas the cache filter uses two bits per block. This commit abstracts the bitmap

The cow and cache filters both use a bitmap mapping virtual disk blocks to status stored in the bitmap. The implementation of the bitmaps is very similar because one was derived from the other when the filters were implemented. The main difference is the cow filter uses a simple bitmap (one bit per block), whereas the cache filter uses two bits per block. This commit abstracts the bitmap

This refactoring also fixes a bug spotted by Eric Blake: nbdkit --filter truncate pattern size=5G round-up=512 results in a 1G virtual disk. --- common/include/rounding.h | 58 ++++++++++++++++++++++++++++++++++ filters/cache/Makefile.am | 3 +- filters/cache/cache.c | 2 +- filters/cow/Makefile.am | 3 +- filters/cow/cow.c | 2 +- filters/truncate/truncate.c | 5

Take advantage of the fact that we can now detect the type of client during --tls=on in order to provide safe dummy content for plaintext clients without having to rewrite plugins to do so. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-plugin.pod | 4 +- docs/nbdkit-tls.pod | 5 +- filters/tlsdummy/nbdkit-tlsdummy-filter.pod

I'm about to add an 'exportname' filter, and in the process, I noticed a few shortcomings in our API. Having .default_export makes it easy to answer NBD_INFO_NAME in response to a client request during NBD_OPT_GO, but answering NBD_INFO_DESCRIPTION is awkward - there's no guarantee the export name was given with a description in .list_exports. Note, however, that while we map