similar to: [PATCH] [v3] ddrescue mapfile filter

This allows to overlay bad sectors according to the mapfile generated by ddrescue, to then see where sectors are used using fsck and trying to copy files around. Signed-off-by: Fran?ois Revol <revol at free.fr> --- configure.ac | 2 + filters/ddrescue/Makefile.am | 75 +++++++ filters/ddrescue/ddrescue.c | 218

This allows to overlay bad sectors according to the mapfile generated by ddrescue, to then see where sectors are used using fsck and trying to copy files around. Signed-off-by: Fran?ois Revol <revol at free.fr> --- configure.ac | 2 + filters/ddrescue/Makefile.am | 75 +++++++ filters/ddrescue/ddrescue.c | 211

On 5/1/20 2:16 PM, Fran?ois Revol wrote: > This allows to overlay bad sectors according to the mapfile generated by > ddrescue, to then see where sectors are used using fsck and trying to > copy files around. > > Signed-off-by: Fran?ois Revol <revol at free.fr> > --- Focusing on just the docs: > +++ b/filters/ddrescue/nbdkit-ddrescue-filter.pod > @@ -0,0 +1,74 @@

Use vector type to store map ranges. Test filenames unique. Remove some unused variables. Break up long lines. --- filters/ddrescue/nbdkit-ddrescue-filter.pod | 6 ++- filters/ddrescue/ddrescue.c | 42 ++++++++++----------- tests/test-ddrescue-filter.sh | 10 ++--- 3 files changed, 29 insertions(+), 29 deletions(-) diff --git

On Fri, May 22, 2020 at 02:13:22AM +0200, François Revol wrote: > Le 01/05/2020 à 23:16, Richard W.M. Jones a écrit : > > On Fri, May 01, 2020 at 09:16:14PM +0200, François Revol wrote: > >> +nbdkit_ddrescue_filter_la_LDFLAGS = \ > >> + -module -avoid-version -shared \ > >> + -Wl,--version-script=$(top_srcdir)/filters/filters.syms \ > >> + $(NULL) >

A few fixes and a possible enhancement to the ddrescue filter. If you think these are all OK, I will squash it into your patch and push it. Rich.

I'm pleased to announce the release of nbdkit 1.22, a high performance plugin-based Network Block Device (NBD) server. https://en.wikipedia.org/wiki/Network_block_device Key features of nbdkit: * Multithreaded NBD server written in C with good performance. * Minimal dependencies for the basic server. * Liberal license (BSD) allows nbdkit to be linked to proprietary libraries or

This can be used to filter a single partition from a disk image. --- TODO | 2 - configure.ac | 1 + filters/Makefile.am | 3 +- filters/delay/delay.c | 12 +-- filters/offset/nbdkit-offset-filter.pod | 17 ++- filters/offset/offset.c |

Take advantage of the fact that we can now detect the type of client during --tls=on in order to provide safe dummy content for plaintext clients without having to rewrite plugins to do so. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-plugin.pod | 4 +- docs/nbdkit-tls.pod | 5 +- filters/tlsdummy/nbdkit-tlsdummy-filter.pod

This very basic filter allows you to select an offset and range within a plugin, for example: nbdkit --filter=offset file file=foo offset=1M range=100M which serves the byte range [ 1M .. 101M-1 ] from file ‘foo’. --- TODO | 2 - configure.ac | 1 + filters/Makefile.am | 3 +- filters/offset/Makefile.am

'nbdkit -v' is quite verbose, and traces everything. Sometimes, we want to trace JUST the client interactions. In particular, when debugging another filter, being able to trace what the filter called can be quite useful; and having the log filter in place will make it easier to add testsuite coverage of other filters. Also, it is nice to have timestamps in the log, in order to see if

This can truncate, extend, or round up/down to a multiple. --- common-rules.mk | 3 +- configure.ac | 1 + filters/offset/nbdkit-offset-filter.pod | 7 +- filters/partition/nbdkit-partition-filter.pod | 1 + filters/truncate/Makefile.am | 60 ++++ filters/truncate/nbdkit-truncate-filter.pod | 87

This can truncate, extend, or round up/down to a multiple. --- common-rules.mk | 3 +- configure.ac | 1 + filters/offset/nbdkit-offset-filter.pod | 7 +- filters/partition/nbdkit-partition-filter.pod | 1 + filters/truncate/Makefile.am | 61 ++++ filters/truncate/nbdkit-truncate-filter.pod | 88

This filter can be used to open tar files. It uses the technique first suggested by Eric Blake here: https://www.redhat.com/archives/libguestfs/2020-July/msg00017.html We suggest that nbdkit-tar-plugin is deprecated in nbdkit 1.26, but it might happen later. --- docs/nbdkit-captive.pod | 4 +- filters/offset/nbdkit-offset-filter.pod | 2 +- filters/tar/nbdkit-tar-filter.pod

Filters can be placed in front of plugins to modify their behaviour. This commit adds the <nbdkit-filter.h> header file, the manual page, the ‘filterdir’ directory (like ‘plugindir’), the ‘filters/’ source directory which will contain the actual filters, the ‘--filters’ parameter, and the filters backend logic. --- Makefile.am | 2 +- TODO | 17 +-

Filters can be placed in front of plugins to modify their behaviour. This commit adds the <nbdkit-filter.h> header file, the manual page, the ‘filterdir’ directory (like ‘plugindir’), the ‘filters/’ source directory which will contain the actual filters, the ‘--filters’ parameter, and the filters backend logic. --- Makefile.am | 2 +- TODO | 17 +-

This filter can be used to transparently reopen/retry when a plugin fails. The connection is closed and reopened which for most plugins causes them to attempt to reconnect to their source. For example if doing a long or slow SSH copy: nbdkit -U - ssh host=remote /var/tmp/test.iso \ --run 'qemu-img convert -p -f raw $nbd -O qcow2 test.qcow2' if the SSH connection or network goes

This filter can be used to transparently reopen/retry when a plugin fails. The connection is closed and reopened which for most plugins causes them to attempt to reconnect to their source. For example if doing a long or slow SSH copy: nbdkit -U - ssh host=remote /var/tmp/test.iso \ --run 'qemu-img convert -p -f raw $nbd -O qcow2 test.qcow2' if the SSH connection or network goes

Here's my second attempt at a filter API. As before, .config and .config_complete can only call the next->.config or next->.config_complete methods in the chain respectively. The change is with the connected functions (get_size, pread, pwrite, etc.). Here we pass a struct of next functions allowing the filter to call any functions on the plugin, so for example a write can turn into

--- configure.ac | 5 + filters/Makefile.am | 4 + filters/cow/Makefile.am | 65 +++++++ filters/cow/cow.c | 392 ++++++++++++++++++++++++++++++++++++++ filters/cow/nbdkit-cow-filter.pod | 162 ++++++++++++++++ tests/Makefile.am | 6 + tests/test-cow.sh | 98 ++++++++++ 7 files changed, 732