similar to: [PATCH nbdkit] xz: Do not pass can_write through to the plugin.

Applies on top of my pending series for the exportname filter, addressing one of the todo's in that cover letter. Eric Blake (2): filters: Add .export_description wrappers ext2: Supply .list_exports and .default_export filters/ext2/nbdkit-ext2-filter.pod | 3 +- tests/Makefile.am | 16 +++- filters/ext2/ext2.c | 125 +++++++++++++++++++---------

Using ‘nbdkit --filter=xz file disk.xz’ and loop mounting eventually results in this error: nbdkit: file[1]: debug: xz: pwrite count=4096 offset=1048576 flags=0x0 nbdkit: file[1]: debug: pwrite count=4096 offset=1048576 fua=0 nbdkit: file[1]: error: pwrite: Bad file descriptor nbdkit: file[1]: debug: sending error reply: Bad file descriptor Since the filter did not intercept can_write, it was

v2: - Fixes a number of bugs in corner cases. - Uses a 1M block size to fetch from the underlying plugin. This improves performance considerably. I also tested this much more thoroughly and can't find any more bugs. Rich.

Matt asked if xz should really be a filter rather than a plugin. The answer is yes, of course it should be! That's been something in the todo file for a while. The commit converts the xz plugin code into a filter (leaving the plugin around, but deprecating it). plugin: nbdkit xz file.xz filter: nbdkit --filter=xz file file.xz plugin: # can't be done filter: nbdkit

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

It's been several weeks since I posted v2 (I got distracted by improving libnbd to better test things, which in turn surfaced some major memory leak problems in nbdsh that are now fixed). Many of the patches are minor rebases from v2, with the biggest changes being fallout from: - patch 2: rename nbdkit_add_default_export to nbdkit_use_default_export - overall: this missed 1.22, so update

On Thu, Aug 27, 2020 at 05:03:45PM -0500, Eric Blake wrote: > When extracting an obvious subset of a larger container (ext2, gzip, > partition, tar, xz), it's fairly easy to add a nice updated > description for what the client is seeing. Not all clients request > the description, but if you are worried about this leaking too much > information, it can be silenced with

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack. Eric Blake (24): server: Internal hooks for implementing

This is a subset of the last half of the larger 9-patch series. The uncontroversial first half of that series is pushed, but here, I tried to reduce the size of the patches by splitting out some of the more complex changes, so that the rest of the changes remaining in the series are more mechanical. In turn, it forced me to write timing tests, which let me spot another spot where we are wasting

I think this is my last round of patches for issues I identified with the retry filter. With this in place, it should be safe to interject another filter in between retry and the plugin. Eric Blake (5): retry: Don't call into closed plugin tests: Refactor test-retry-reopen-fail.sh tests: Enhance retry test to cover failed reopen server: Move prepare/finalize/close recursion to

I still don't have .prepare/.finalize working cleanly across reopen, but did find a nasty bug where a botched assertion means we failed to notice reads beyond EOF in both the xz and retry filter. Refactoring backend.c will make .finalize work easier. Eric Blake (5): xz: Avoid reading beyond EOF retry: Check size before transactions tests: Test retry when get_size values change

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.

Rebased filters patch. Requires current git master + the locks / thread model fix (https://www.redhat.com/archives/libguestfs/2018-January/msg00128.html) So a few changes here since last time: The "introduce filters" and "implement filters" patches are squashed together. I introduced a concept of .prepare and .finalize. These run before and after the data serving phase

On 7/22/20 7:42 AM, Richard W.M. Jones wrote: > See also: > https://www.redhat.com/archives/libguestfs/2020-July/msg00090.html > --- > docs/nbdkit-plugin.pod | 57 +++++++++++++++++++++++++++++++++++++--- > docs/nbdkit-protocol.pod | 7 +++-- > 2 files changed, 58 insertions(+), 6 deletions(-) I'm playing with code for an initial implementation of this, and quickly

This is a retry filter implementation as outlined here: https://www.redhat.com/archives/libguestfs/2019-September/msg00167.html It is only lightly tested. One way to test it is to try an SSH copy (see the commit message for patch 2/2), and in the middle of the copy kill the per-connection sshd on the remote machine. You will see that the copy recovers after a few seconds. Add the nbdkit -v

It's easy to use the sh script to demonstrate that nbdkit is inefficiently calling into .get_size, .can_fua, and friends more than necessary. We've also commented on the list in the past that it would be nice to ensure that when filters call into next_ops, they are not violating constraints (as we've have to fix several bugs in the past where we did not have such checking to protect

I'm still working my way through the filters before this series will be complete, but this is enough of a start to at least get some feedback on the idea of implementing another NBD protocol extension. Eric Blake (9): server: Internal hooks for implementing NBD_CMD_CACHE plugins: Add .cache callback file, split: Implement .cache with posix_fadvise nbd: Implement NBD_CMD_CACHE

On Thu, Oct 03, 2019 at 09:54:37PM -0500, Eric Blake wrote: > Although it is unusual for a plugin to shrink size on reopen, it is > not impossible. Failure to check our bounds could result in violating > assumptions in the plugin that all requests are in-bounds. Note that > if the plugin gains a larger size, we merely never access the new tail > of the file (when the NBD protocol

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both