similar to: [nbdkit PATCH] sh: Prefer dd bs=1 over iflag=count_bytes

On Mon, Aug 17, 2020 at 11:35:39AM -0500, Eric Blake wrote: > While iflag=count_bytes combined with bs > 1 allows for more efficient > operation, it is a feature of GNU dd, and not present on other > implementations such as BSD. Sticking to just POSIX features makes > things more portable. I'm not very convinced by this. Maybe we should persuade the BSD folk to implement this

On 9/12/19 4:01 PM, Richard W.M. Jones wrote: > --- > tests/Makefile.am | 3 ++ > tests/test-export-name.sh | 86 +++++++++++++++++++++++++++++++++++++++ > 2 files changed, 89 insertions(+) > +# Create a sh plugin which echos the export name back in the device. echoes? One of those weird words with dual acceptable spellings, where it depends on who you ask for which

On 6/11/19 3:49 AM, Martin Kletzander wrote: > This filter caches the last result of the extents() call and offers a nice > speed-up for clients that only support req_one=1 in combination with plugins > like vddk, which has no overhead for returning information for multiple extents > in one call, but that call is very time-consuming. > > Quick test showed that on a fast

When we first created the sh plugin, we copied the thread model used in most other language bindings of SERIALIZE_ALL_REQUESTS, because it was easier to reason about and there was no way for a script to override our choice. However, we've since added support for scripts to request a tighter model when needed, and we've also just fixed remaining bugs about fds inadvertently leaked into the

On 12/14/18 4:16 PM, Richard W.M. Jones wrote: > This implements something like a readonly 1MB disk reading as zeroes: > > nbdkit sh script=- <<'EOF' > case "$1" in > get_size) echo 1M ;; > pread) dd if=/dev/zero count=$3 iflag=count_bytes ;; > *) exit 2 ;; > esac > EOF > > Use of "-" is analogous to reading

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_one=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map`

A devious plugin can cause an 8-fold increase in the reply size in relation to the request size, by alternating status every byte of the request. In the worst case, this can cause the client to reject our response and kill the connection because the response is too large. What's more, it consumes a lot of memory to track that many extents. Let's put an upper bound on the maximum number of

On Fri, Aug 02, 2019 at 02:26:16PM -0500, Eric Blake wrote: > When we first created the sh plugin, we copied the thread model used > in most other language bindings of SERIALIZE_ALL_REQUESTS, because it > was easier to reason about and there was no way for a script to > override our choice. However, we've since added support for scripts > to request a tighter model when needed,

We use a custom sh plugin to test retries are working. --- tests/Makefile.am | 8 +++ tests/test-retry-reopen-fail.sh | 108 ++++++++++++++++++++++++++++++++ tests/test-retry.sh | 97 ++++++++++++++++++++++++++++ 3 files changed, 213 insertions(+) diff --git a/tests/Makefile.am b/tests/Makefile.am index 1b1e05b..af9b9d9 100644 --- a/tests/Makefile.am +++

--- tests/Makefile.am | 3 ++ tests/test-export-name.sh | 86 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+) diff --git a/tests/Makefile.am b/tests/Makefile.am index b5806bb..f54597b 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -395,6 +395,9 @@ endif HAVE_LIBGUESTFS # Test export flags. TESTS += test-eflags.sh +# Test export name. +TESTS +=

On 8/7/20 5:00 PM, Eric Blake wrote: > Take advantage of the fact that we can now detect the type of client > during --tls=on in order to provide safe dummy content for plaintext > clients without having to rewrite plugins to do so. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- I got a test working (although it still shows that we are bit awkward until nbdkit

On Fri, Dec 14, 2018 at 04:31:32PM -0600, Eric Blake wrote: > On 12/14/18 4:16 PM, Richard W.M. Jones wrote: > >This implements something like a readonly 1MB disk reading as zeroes: > > > >nbdkit sh script=- <<'EOF' > > case "$1" in > > get_size) echo 1M ;; > > pread) dd if=/dev/zero count=$3 iflag=count_bytes ;; > >

Various patches before this one plugged fd leaks of files opened within nbdkit that the shell script need not see. It's time to add testsuite coverage to ensure we don't regress. However, a user may intentionally want to expose an open fd to nbdkit, which in turn remains available all the way through to the shell script, so nbdkit does not close unrecognized inherited fds at startup.

Instead of printing something like ‘type=0’ or ‘type=3’, this changes the output to show the hole and zero flags separately. For example: $ ./nbdkit -U - --filter=log sh - logfile=/dev/stdout \ --run 'qemu-img map $nbd' <<'EOF' case "$1" in get_size) echo 1M ;; pread) dd if=/dev/zero count=$3 iflag=count_bytes ;; can_extents) exit 0 ;;

This implements something like a readonly 1MB disk reading as zeroes: nbdkit sh script=- <<'EOF' case "$1" in get_size) echo 1M ;; pread) dd if=/dev/zero count=$3 iflag=count_bytes ;; *) exit 2 ;; esac EOF Use of "-" is analogous to reading passwords from stdin. --- plugins/sh/nbdkit-sh-plugin.pod | 17 ++++++++++ plugins/sh/sh.c

We documented our intent of only allowing a filter to run with the same version of nbdkit it was compiled against, but up to now, were not actually enforcing that - we had only been insisting on the looser notion of a matching ._api_version, which doesn't help when we've forgotten to bump that macro when making incompatible API/ABI changes (see commit 6934d4c1). However, we can't use

Use the recently added nbdkit_string_intern to make this possible. Testsuite coverage is added in both tls-fallback as well as a cleanup to test-eval-exports.sh. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/eval/nbdkit-eval-plugin.pod | 2 ++ plugins/sh/nbdkit-sh-plugin.pod | 20 +++++++++-- plugins/sh/methods.h | 1 + plugins/eval/eval.c |

A comment in the code mentioned something that didn't actually work, but which can be useful for user-directed logging of what other callbacks they might want to implement. Signed-off-by: Eric Blake <eblake@redhat.com> --- I haven't pushed this one, becuase I'm not sure if we want it; but it was easy enough to whip together after an IRC question earlier today.

On 4/1/19 6:53 AM, Richard W.M. Jones wrote: > Instead of printing something like ‘type=0’ or ‘type=3’, this changes > the output to show the hole and zero flags separately. For example: > > $ ./nbdkit -U - --filter=log sh - logfile=/dev/stdout \ > --run 'qemu-img map $nbd' <<'EOF' > case "$1" in > get_size) echo 1M ;; >

The previous incomplete patch was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00049.html based on earlier discussion here: https://www.redhat.com/archives/libguestfs/2019-September/msg00047.html In v2: - The previous patch was incomplete. This version completes it by adding tests and extending nbdkit-sh-plugin. - nbdkit_export_name now returns NULL for error,