similar to: [PATCH nbdkit INCOMPLETE 0/6] Introduce filters to nbdkit.

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both

Rebased filters patch. Requires current git master + the locks / thread model fix (https://www.redhat.com/archives/libguestfs/2018-January/msg00128.html) So a few changes here since last time: The "introduce filters" and "implement filters" patches are squashed together. I introduced a concept of .prepare and .finalize. These run before and after the data serving phase

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

Being able to programmatically force nbdkit to be less parallel can be useful during testing. I was less sure about patch 3, but if you like it, I'm inclined to instead squash it into patch 1. This patch is written to apply after my NBD_CMD_CACHE work (since I touched the nocache filter); but can be rearranged if we think this series should go in first while that one undergoes any adjustments

Remove the need to use file= (and in future other) parameters for many plugins. eg. Using the file plugin becomes: nbdkit file disk.img Rich.

v1 was here: https://www.redhat.com/archives/libguestfs/2018-September/msg00024.html v2: - As discussed in the patch review, tighten up the characters permitted in keys. - Update documentation to note that relative paths can be made safe by prefixing with ./ and absolute paths do not need any extra steps. - I pushed patch 1/6 from the v1 series since it was just a trivial

v1: https://www.redhat.com/archives/libguestfs/2018-September/msg00024.html v2: https://www.redhat.com/archives/libguestfs/2018-September/msg00034.html v3: - Fixed is_config_key zero length test. - Fixed is_config_key so it uses strspn and is not O(n^2). - Changed >= 1.7 to >= 1.8 in the documentation. Rich.

This adds a new nbdkit_shutdown() API whereby plugins and filters can request that the server shuts down (asynchronously) during the serving phase. Two new filters are added, one of which depends on this feature and the other not needing it but being somewhat related. Rich.

Here's changes to the file plugin (which I'm happy with) and a new exportname filter (which is still at RFC stage; I need to finish implementing strict mode in .open, and add tests). I also discovered that we really want .list_exports and .open to know when they are used on plaintext vs. tls clients for --tls=on, and we may want to split out a new .default_export callback rather than

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack. Eric Blake (24): server: Internal hooks for implementing

This is a second version of the limit filter. v1 was posted here: https://www.redhat.com/archives/libguestfs/2020-March/msg00015.html I didn't bother to repost the other three patches because they are the same. The difference is this version of the filter takes security more seriously. It shouldn't be possible for malicious clients to connect more than limit=N times to the plugin now,

Patch 3 still needs tests added, but it is at least working from my simple command line tests. Eric Blake (3): server: Implement nbdkit_is_tls for use during .open server: Expose final thread_model to filter's .get_ready tlsdummy: New filter docs/nbdkit-filter.pod | 21 +- docs/nbdkit-plugin.pod | 34 ++- docs/nbdkit-tls.pod

On 01/19/2018 09:23 AM, Richard W.M. Jones wrote: > Filters can be placed in front of plugins to modify their behaviour. > > This commit adds the <nbdkit-filter.h> header file, the manual page, > the ‘filterdir’ directory (like ‘plugindir’), the ‘filters/’ source > directory which will contain the actual filters, the ‘--filters’ > parameter, and the filters backend logic.

This incomplete patch adds a new filter allowing more control over when nbdkit exits. You can now get nbdkit to exit gracefully on certain events, such as a file being created, a pipe held open by another process going away, or when another PID exits. There is also a script option to allow for completely custom events. It is untested at the moment, I'm posting it to get feedback on the

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

I still need to add testsuite coverage. Perhaps it might be easier if I create a new '--filter=log logfile=foo' filter that produces a log of which commands a client sent, then compare the log using a known client that uses write_zeroes (qemu-io works well) both with and without --filter=nozero to prove that the change in advertisement changes the commands sent over the wire (that would

Here's my second attempt at a filter API. As before, .config and .config_complete can only call the next->.config or next->.config_complete methods in the chain respectively. The change is with the connected functions (get_size, pread, pwrite, etc.). Here we pass a struct of next functions allowing the filter to call any functions on the plugin, so for example a write can turn into

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.

Here's what I'm currently playing with; I'm not ready to commit anything until I rebase my FUA work on top of this, as I only want to break filter ABI once between releases. Eric Blake (2): backend: Rework internal/filter error return semantics filters: Add log filter TODO | 2 - docs/nbdkit-filter.pod | 84 +++++++-- docs/nbdkit.pod