similar to: [PATCH nbdkit 0/2] tar: Rewrite the tar plugin (again), this time in C.

On 6/28/20 8:02 AM, Richard W.M. Jones wrote: > --- > plugins/tar/{tar.pl => nbdkit-tar-plugin.pod} | 145 ++------- > configure.ac | 2 - > plugins/tar/Makefile.am | 41 +-- > tests/Makefile.am | 14 +- > plugins/tar/tar.c | 286 ++++++++++++++++++ >

On Sun, Jun 28, 2020 at 4:03 PM Richard W.M. Jones <rjones@redhat.com> wrote: ... > + > +static int > +tar_get_ready (void) > +{ > + FILE *fp; > + CLEANUP_FREE char *cmd = NULL; > + size_t len = 0; > + bool scanned_ok; > + char s[256]; > + > + /* Construct the tar command to examine the tar file. */ > + fp = open_memstream (&cmd, &len); >

For review only, this needs some clean up and more tests. My eyes are going cross-eyed looking at the calculate_offset_of_entry function, so time to take a break ... Rich.

--- plugins/tar/{tar.pl => nbdkit-tar-plugin.pod} | 145 ++------- configure.ac | 2 - plugins/tar/Makefile.am | 41 +-- tests/Makefile.am | 14 +- plugins/tar/tar.c | 286 ++++++++++++++++++ tests/test-dump-plugin.sh | 2 +- tests/test-help-plugin.sh

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]

Patch 3 still needs tests added, but it is at least working from my simple command line tests. Eric Blake (3): server: Implement nbdkit_is_tls for use during .open server: Expose final thread_model to filter's .get_ready tlsdummy: New filter docs/nbdkit-filter.pod | 21 +- docs/nbdkit-plugin.pod | 34 ++- docs/nbdkit-tls.pod

v2: - Fixes a number of bugs in corner cases. - Uses a 1M block size to fetch from the underlying plugin. This improves performance considerably. I also tested this much more thoroughly and can't find any more bugs. Rich.

Applies on top of my pending series for the exportname filter, addressing one of the todo's in that cover letter. Eric Blake (2): filters: Add .export_description wrappers ext2: Supply .list_exports and .default_export filters/ext2/nbdkit-ext2-filter.pod | 3 +- tests/Makefile.am | 16 +++- filters/ext2/ext2.c | 125 +++++++++++++++++++---------

Since v1: add the blocksize filter, add testsuite coverage of the log filter, several fixes to the log filter based on what adding tests revealed I'm still working on FUA flag support patches on top of this; the patches should all be committed in the same release, as we want to minimize the number of releases that cause a filter ABI/API bump Eric Blake (3): backend: Rework internal/filter

There is a small test provided. I tested this a lot more locally and it seems pretty robust. Rich.

Rebased filters patch. Requires current git master + the locks / thread model fix (https://www.redhat.com/archives/libguestfs/2018-January/msg00128.html) So a few changes here since last time: The "introduce filters" and "implement filters" patches are squashed together. I introduced a concept of .prepare and .finalize. These run before and after the data serving phase

Based on Rich's review of my v1 that touched only cache.c, I have now tried to bring all three filters with alignment rounding in line with one another. There is definitely room for future improvements once we teach nbdkit to let filters and plugins advertise block sizes, but I'm hoping to get NBD_CMD_CACHE implemented first. Eric Blake (2): blocksize: Process requests in linear order

Matt asked if xz should really be a filter rather than a plugin. The answer is yes, of course it should be! That's been something in the todo file for a while. The commit converts the xz plugin code into a filter (leaving the plugin around, but deprecating it). plugin: nbdkit xz file.xz filter: nbdkit --filter=xz file file.xz plugin: # can't be done filter: nbdkit

This patch isn't complete (patch 6/6 isn't finished) so it's just for discussion, although it does compile and run. This introduces to nbdkit a concept of "filters" which can be placed in front of plugins to modify their behaviour. Some examples where you might use filters: * Serve a subset of the data, such as (offset, range) or a single partition from a disk image.

v1 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00147.html v2 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00152.html v3: - Use optarg != NULL as a sentinel for has_arg. - Moved some variable decls into the inner loop. - Make nbdkit wrapper depend on config.status, so if srcdir or builddir changes then we rebuild the wrapper. It

This patch series solves the FreeBSD shebang problem in a completely different way, and a few other things besides. I propose that we replace ./nbdkit with a C program. The C program is a straightforward translation of the shell script. Some advantages of this approach are: - We can parse options in exactly the same way as the real program. - Use the more accurate ‘is_short_name’ test for

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

v1 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00147.html v2: - Use stdbool for booleans. - Use __attribute__((format(printf))). - Don't abort on invalid options, exit with failure instead. - Preserve long/short option choices in the output. - Add '=' in long option output, ie. always use --longopt=arg. - Add "--" parameter

This is still tentative and needs a lot of work, but: - partition filter works, supporting MBR & GPT - prepare and finalize methods fixed - open method can now be changed (allowing readonly flag to be modified) - thread_model can be limited I believe I made most of the changes which were previously suggested in email. I think the only one I didn't was preventing inclusion of both

Since the v1 series (0/4, at [1]), I've applied patches 1 and 2, rewritten patch 3 [Forbid NUL in export and context names] into patch 4 here, patch 4 there turned into patch 6 here, and everything else here is new. [1]https://www.redhat.com/archives/libguestfs/2019-September/msg00180.html I don't know if there is a handy reusable function for checking whether a string contains valid