similar to: [PATCH nbdkit] common/include: Add generic MIN and MAX macros.

The preferred implementation uses __auto_type, a GCC extension also now supported by Clang. Unfortunately OpenBSD ships with GCC 4.2.1 (from 2007!) which predates this extension by quite a few years, so we have to be able to fall back to a plain macro. --- configure.ac | 20 ++++++++++- common/include/minmax.h | 63 +++++++++++++++++++++++++++++++++++

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I

No need for filters to specify their model in two different ways; the callback alone is sufficient. Since all filters are in-tree, we can deal with the API/ABI change made here. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-filter.pod | 48 +++++++++++++++------------------ include/nbdkit-filter.h | 6 ++--- filters/cow/cow.c | 2 --

On 7/4/19 1:38 PM, Thomas Weißschuh wrote: > Hi all, > > It seems GCC 9 does not put read-only zero-initialized variables into the BSS > section anymore; instead it is put into RODATA. > (See the thread at [0], especially [1]) > Thanks for the pointers. > In filter/nozero a 64M large, static, zeroed, read-only array is declared. > The new behavior of GCC puts this array

Similar to the existing fua, nozero and noextents filters, add a filter to make it easy to override the basic caching functionality, in part to facilitate timing tests of whether a plugin's cache implementation is worthwhile. A worthwhile test to add to the testsuite would connect the log filter both before and after the nocache filter, to prove how caching requests are altered. However,

It may be useful to test whether the client's use of NBD_CMD_FLAG_NO_HOLE makes a difference; do this by adding a mode to --filter=nozero to force a non-trimming zero write. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/nozero/nbdkit-nozero-filter.pod | 19 +++++++---- filters/nozero/nozero.c | 45 ++++++++++++++++++++----- tests/test-nozero.sh

The nozero filter can split a large request with FUA into several smaller requests; optimize whether the FUA flag is passed on to the next layer based on whether FUA is emulated with flush (only the last write needs it) or is natively supported (every write needs it). Missed in commit df0cc21d. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/nozero/nozero.c | 17 +++++++++++++++--

--- configure.ac | 2 ++ filters/blocksize/Makefile.am | 1 + filters/cache/Makefile.am | 1 + filters/cacheextents/Makefile.am | 1 + filters/cow/Makefile.am | 1 + filters/ddrescue/Makefile.am | 1 + filters/delay/Makefile.am | 3 +++ filters/error/Makefile.am | 1 + filters/exitlast/Makefile.am | 3 +++

--- docs/nbdkit-captive.pod | 4 ++-- docs/nbdkit-plugin.pod | 2 +- docs/nbdkit-service.pod | 2 +- docs/nbdkit-tls.pod | 2 +- filters/cow/nbdkit-cow-filter.pod | 4 ++-- filters/delay/nbdkit-delay-filter.pod | 2 +- filters/error/nbdkit-error-filter.pod | 6 +++---

--- docs/nbdkit-captive.pod | 4 ++-- docs/nbdkit-plugin.pod | 2 +- docs/nbdkit-service.pod | 2 +- docs/nbdkit-tls.pod | 2 +- filters/cow/nbdkit-cow-filter.pod | 4 ++-- filters/delay/nbdkit-delay-filter.pod | 2 +- filters/error/nbdkit-error-filter.pod | 6 +++---

We documented our intent of only allowing a filter to run with the same version of nbdkit it was compiled against, but up to now, were not actually enforcing that - we had only been insisting on the looser notion of a matching ._api_version, which doesn't help when we've forgotten to bump that macro when making incompatible API/ABI changes (see commit 6934d4c1). However, we can't use

It is easy to demonstrate that our blocksize filter has a bug: if the minblock= setting is a higher granularity than the underlying plugin actually supports, and the client is trying to collect block status for the entire disk, a mid-block transition in the plugin can result in the filter rounding a request so small that it no longer makes progress, causing the client to see: nbd.Error:

From: "Richard W.M. Jones" <rjones at redhat.com> Original commit message: Thanks: Eric Blake Porting notes: We already have the nesting fix in the "common/include/minmax.h" header file, from nbdkit commit 9f462af12e3b ("common/include: Fix MIN and MAX macros so they can be nested", 2022-02-22), via libnbd commit 2f25695212db5. However, that port didn't

In these test functions, when a test fails, the bash ‘trap’ command causes the cleanup function to be called. However bash does not annul or cancel the traps when cleanup is called, so at the end of the cleanup function when ‘exit’ is called, cleanup is called recursively. Avoid this by cancelling the traps at the top of the cleanup function. Also an extra debugging message is emitted here

This assumes bashisms, but bash is required to run the tests. This is mostly simple refactoring. Except for the test-memory*.sh tests where nbdkit used to run in the foreground, but that seems to be a consequence of some left over debugging. --- tests/functions.sh.in | 35 ++++++++++++++++++++ tests/test-blocksize.sh | 23 +++---------- tests/test-cache.sh

See the cross-post cover letter for details: https://www.redhat.com/archives/libguestfs/2019-August/msg00322.html Notably, this series did NOT add fast zero support to the file plugin yet; there, I probably need to do more testing and/or kernel source code reading to learn whether to mark fallocate() as potentially slow, as well as to definitely mark ioctl(BLKZEROOUT) as definitely slow. That

This assumes bashisms, but bash is required to run the tests. This is mostly refactoring. However the changes (simplifications) are quite substantial: - Since the new start_nbdkit helper function cleans up nbdkit on exit, most scripts no longer need to deal with the pid or kill the pid in the cleanup function. - As a result, cleanup functions are radically simpler, and often

The blocksize filter already rounds sizes down, to avoid having to special-case a partial action at the end of an unaligned file. But our current behavior for cache and cow is not similarly careful, and can end up causing errors by passing a too-large count to the plugin. The bulk of our plugins are somewhat safe, in that they either fail the pread (for example, the file plugin gets a short read

Being able to programmatically force nbdkit to be less parallel can be useful during testing. I was less sure about patch 3, but if you like it, I'm inclined to instead squash it into patch 1. This patch is written to apply after my NBD_CMD_CACHE work (since I touched the nocache filter); but can be rearranged if we think this series should go in first while that one undergoes any adjustments

The original plan was to have a background thread doing the reclaim. However that cannot work given the design of filters, because a background thread cannot access the next_ops struct which is only available during requests. Therefore we spread the work over the request threads. Each blk_* function checks whether there is work to do, and if there is will reclaim up to two blocks from the cache