similar to: [nbdkit PATCH] retry: Tweak error message

On 4/1/20 5:12 AM, Richard W.M. Jones wrote: > On Tue, Mar 31, 2020 at 04:00:22PM -0500, Eric Blake wrote: >> The retry filter defaults to 5 retries, but when run with verbose >> details produces some confusing output: >> >> $ rm -f /tmp/inject; (sleep 5s; touch /tmp/inject)& ./nbdkit -f -v -U - \ >> null 1G --filter=retry --filter=noextents --filter=error

On Tue, Mar 31, 2020 at 04:00:22PM -0500, Eric Blake wrote: > The retry filter defaults to 5 retries, but when run with verbose > details produces some confusing output: > > $ rm -f /tmp/inject; (sleep 5s; touch /tmp/inject)& ./nbdkit -f -v -U - \ > null 1G --filter=retry --filter=noextents --filter=error error-rate=100% \ > error-file=/tmp/inject --filter=delay rdelay=1

On Wed, Apr 01, 2020 at 08:46:53AM -0500, Eric Blake wrote: > On 4/1/20 5:12 AM, Richard W.M. Jones wrote: > >On Tue, Mar 31, 2020 at 04:00:22PM -0500, Eric Blake wrote: > >>The retry filter defaults to 5 retries, but when run with verbose > >>details produces some confusing output: > >> > >>$ rm -f /tmp/inject; (sleep 5s; touch /tmp/inject)&

This is a retry filter implementation as outlined here: https://www.redhat.com/archives/libguestfs/2019-September/msg00167.html It is only lightly tested. One way to test it is to try an SSH copy (see the commit message for patch 2/2), and in the middle of the copy kill the per-connection sshd on the remote machine. You will see that the copy recovers after a few seconds. Add the nbdkit -v

v1 was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00199.html v2: - Adds a fairly simple yet comprehensive test using sh plugin. - Rebase and retest. Patch 1 is a misc patch not really related to the series. Rich.

v2 was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00221.html I think this is more like "the one". It handles reopen failing correctly, and there is a second test for that. I also ran my sshd tests locally and it worked in all scenarios I could think up (except of course sshd not being available at the start, but we want that to fail). Rich.

In https://bugzilla.redhat.com/show_bug.cgi?id=1841820, it was pointed out that the retry filter not retrying .open means that an ssh connection (such as in a vmx+ssh v2v conversion) fails when the ssh connection itself cannot be retried. A year ago, this was an inherent limitation of our retry implementation; but in the meantime, my work to allow filters to open independent backends has made it

Now that a filter can open a backend as many times as it wants, there's no longer a technical reason we can't retry .open. However, adding retry logic here does mean we have to weaken an assert in the server backend code, since prepare can now be reached more than once. Test coverage will be added in a separate patch, so that it becomes easy to swap patch order and see that the test

Includes a rework of the previously posted patch for --run improvements (mostly with improved comments and commit message; I decided that waiting for the captive nbdkit to exit was overkill), and four new patches. The tests are intentionally separate, to allow rearranging the order of the series to see the failures being fixed. Eric Blake (6): server: Propagate unexpected nbdkit failure with

I think this is my last round of patches for issues I identified with the retry filter. With this in place, it should be safe to interject another filter in between retry and the plugin. Eric Blake (5): retry: Don't call into closed plugin tests: Refactor test-retry-reopen-fail.sh tests: Enhance retry test to cover failed reopen server: Move prepare/finalize/close recursion to

Being able to programmatically force nbdkit to be less parallel can be useful during testing. I was less sure about patch 3, but if you like it, I'm inclined to instead squash it into patch 1. This patch is written to apply after my NBD_CMD_CACHE work (since I touched the nocache filter); but can be rearranged if we think this series should go in first while that one undergoes any adjustments

On Thu, Oct 03, 2019 at 09:54:37PM -0500, Eric Blake wrote: > Although it is unusual for a plugin to shrink size on reopen, it is > not impossible. Failure to check our bounds could result in violating > assumptions in the plugin that all requests are in-bounds. Note that > if the plugin gains a larger size, we merely never access the new tail > of the file (when the NBD protocol

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the

tmpfs has a known bug of O(n^2) behavior with lseek(SEEK_HOLE); this is one situation where the noextents filter can come in handy to avoid the performance penalty of exposing accurate extents. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/noextents/nbdkit-noextents-filter.pod | 7 ++++++- plugins/file/nbdkit-file-plugin.pod | 9 ++++++++- 2 files changed, 14

The NBD protocol is adding an extension to let servers advertise initialization state to the client: whether the image contains holes, and whether it is known to read as all zeroes. Most filters just pass through the plugin's result (as it is only checked once when first connecting), but we can do useful things in log (mention the setting), extentlist (answer based on the extents we are

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both

This patch isn't complete (patch 6/6 isn't finished) so it's just for discussion, although it does compile and run. This introduces to nbdkit a concept of "filters" which can be placed in front of plugins to modify their behaviour. Some examples where you might use filters: * Serve a subset of the data, such as (offset, range) or a single partition from a disk image.

On Fri, Jan 27, 2023 at 02:41:22PM -0600, Eric Blake wrote: > Now that a filter can open a backend as many times as it wants, > there's no longer a technical reason we can't retry .open. However, > adding retry logic here does mean we have to weaken an assert in the > server backend code, since prepare can now be reached more than once. > > Test coverage will be added

As I've spent really too long today investigating this, I want to document this in a public email, even though there's nothing really that interesting here. One thing you find from search for VDD 6.7 / VixDiskLib_QueryAllocatedBlocks issues with Google is that we must be one of the very few users out there. And the other thing is that it's quite broken. All testing was done using

Either patch 1 or patch 2 in isolation is sufficient to fix the problem that Rich forwarded on from an archlinux tester (name so I can credit them?). But both patches should be applied, as well as backported to appropriate stable branches, to maximize cross-version interoperability of nbdkit vs. plugins. Patch 3 will let us detect future similar bugs much faster. I want patch 4 to ensure that