similar to: [nbdkit PATCH] filters: Improve docs on .prepare prerequisites

On 7/9/20 12:08 PM, Richard W.M. Jones wrote: > On Thu, Jul 09, 2020 at 11:37:53AM -0500, Eric Blake wrote: >> Since .prepare is called before client negotiation has completed, >> filters have an additional burden to ensure prerequisite functions are >> called in order to avoid triggering assertions in backend.c. >> >> See also:

On Thu, Jul 09, 2020 at 11:37:53AM -0500, Eric Blake wrote: > Since .prepare is called before client negotiation has completed, > filters have an additional burden to ensure prerequisite functions are > called in order to avoid triggering assertions in backend.c. > > See also: https://bugzilla.redhat.com/show_bug.cgi?id=1855330, >

I'm not sure that this fix is really correct. An alternate way I can think to fix this would be for the core server to maintain a readonly flag for each layer (instead of just per- server). You could also argue that our readonly test in server/connections.c:compute_eflags is wrong and/or that the implementations of server/plugins.c:plugin_can_write and server/filters.c:filter_can_write have

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both

Rebased filters patch. Requires current git master + the locks / thread model fix (https://www.redhat.com/archives/libguestfs/2018-January/msg00128.html) So a few changes here since last time: The "introduce filters" and "implement filters" patches are squashed together. I introduced a concept of .prepare and .finalize. These run before and after the data serving phase

I think this is my last round of patches for issues I identified with the retry filter. With this in place, it should be safe to interject another filter in between retry and the plugin. Eric Blake (5): retry: Don't call into closed plugin tests: Refactor test-retry-reopen-fail.sh tests: Enhance retry test to cover failed reopen server: Move prepare/finalize/close recursion to

I still don't have .prepare/.finalize working cleanly across reopen, but did find a nasty bug where a botched assertion means we failed to notice reads beyond EOF in both the xz and retry filter. Refactoring backend.c will make .finalize work easier. Eric Blake (5): xz: Avoid reading beyond EOF retry: Check size before transactions tests: Test retry when get_size values change

It's easy to use the sh script to demonstrate that nbdkit is inefficiently calling into .get_size, .can_fua, and friends more than necessary. We've also commented on the list in the past that it would be nice to ensure that when filters call into next_ops, they are not violating constraints (as we've have to fix several bugs in the past where we did not have such checking to protect

Applies on top of my pending series for the exportname filter, addressing one of the todo's in that cover letter. Eric Blake (2): filters: Add .export_description wrappers ext2: Supply .list_exports and .default_export filters/ext2/nbdkit-ext2-filter.pod | 3 +- tests/Makefile.am | 16 +++- filters/ext2/ext2.c | 125 +++++++++++++++++++---------

I'm still working on another set of patches to have reopen call .finalize/.prepare (so that another filter can safely appear between retry and the plugin), but for tonight, these are the patches I think are ready to go. Eric Blake (4): retry: Handle can_fua and can_fast_zero changes tests: Test retry with different fua/fast-zero flags server: Close backends if a filter's .open fails

On Wed, Jan 31, 2018 at 09:26:37PM -0600, Eric Blake wrote: > Previously, we let a plugin set an error in either thread-local > storage (nbdkit_set_error()) or errno, then connections.c would > decode which error to use. But with filters in the mix, it is > very difficult for a filter to know what error was set by the > plugin (particularly since nbdkit_set_error() has no public

v1 was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00199.html v2: - Adds a fairly simple yet comprehensive test using sh plugin. - Rebase and retest. Patch 1 is a misc patch not really related to the series. Rich.

Since v1: add the blocksize filter, add testsuite coverage of the log filter, several fixes to the log filter based on what adding tests revealed I'm still working on FUA flag support patches on top of this; the patches should all be committed in the same release, as we want to minimize the number of releases that cause a filter ABI/API bump Eric Blake (3): backend: Rework internal/filter

Here's what I'm currently playing with; I'm not ready to commit anything until I rebase my FUA work on top of this, as I only want to break filter ABI once between releases. Eric Blake (2): backend: Rework internal/filter error return semantics filters: Add log filter TODO | 2 - docs/nbdkit-filter.pod | 84 +++++++-- docs/nbdkit.pod

Turn the existing nbdkit-gzip-plugin into a filter so it can be applied on top of files or other sources: nbdkit file --filter=gzip file.gz nbdkit curl --filter=gzip https://example.com/disk.gz Because of the nature of the gzip format which is not blocked based and thus not seekable, this filter caches the whole uncompressed file in a hidden temporary file. This is required in order to

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack. Eric Blake (24): server: Internal hooks for implementing

While working on adding assertions to pthread_mutex_lock calls, I noticed that the truncate filter's use of mutex didn't really protect us, and isn't really necessary. Cleaning that up also spotted a couple of other potential cleanups. Eric Blake (4): filters: Drop useless .open callbacks truncate: Fix corruption when plugin changes per-connection size truncate: Test for safe

The man page claims both offset and range are optional (matching the code), but the --help text claims offset is mandatory, and the comment to the no-op offset_config_complete claims we require both parameters. We did not check for an offset larger than the underlying size when there was no range, and even when there is a range, we were not careful about integer overflow (offset=5E range=5E

Updated proposal, taking into account the default export. Instead of adding a second call, I made a couple of changes to list_exports: (1) If the plugin has a concept of a default export, it should add it as the first element in the exports list. (2) There is a new default_only flag which tells the plugin that the client is trying to request the name of the default export, so the

It's confusing to use the same terminology for a single backend as for the linked list of backends. In particular it's often not clear if we're calling the next backend or the whole chain of backends. --- server/internal.h | 14 ++++++++++-- server/connections.c | 20 ++++++++--------- server/locks.c | 2 +- server/main.c