similar to: [nbdkit PATCH] sh: Allow pwrite to not consume all data

On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: > I hit another transient failure in libnbd CI when a poorly-written > eval script did not consume all of stdin during .pwrite. As behaving > as a data sink can be a somewhat reasonable feature of a > quickly-written sh or eval plugin, we should not be so insistent as > treating an EPIPE failure as an immediate return of

On 8/31/23 00:21, Eric Blake wrote: > I hit another transient failure in libnbd CI when a poorly-written > eval script did not consume all of stdin during .pwrite. As behaving > as a data sink can be a somewhat reasonable feature of a > quickly-written sh or eval plugin, we should not be so insistent as > treating an EPIPE failure as an immediate return of EIO to the client. >

On Thu, Aug 31, 2023 at 11:12:59AM +0200, Laszlo Ersek wrote: > On 8/31/23 10:02, Richard W.M. Jones wrote: > > > > On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: > >> I hit another transient failure in libnbd CI when a poorly-written > >> eval script did not consume all of stdin during .pwrite. As behaving > >> as a data sink can be a

On Thu, Aug 31, 2023 at 10:40:53AM +0200, Laszlo Ersek wrote: > On 8/31/23 00:21, Eric Blake wrote: > > I hit another transient failure in libnbd CI when a poorly-written > > eval script did not consume all of stdin during .pwrite. As behaving > > as a data sink can be a somewhat reasonable feature of a > > quickly-written sh or eval plugin, we should not be so

On 8/31/23 10:02, Richard W.M. Jones wrote: > > On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: >> I hit another transient failure in libnbd CI when a poorly-written >> eval script did not consume all of stdin during .pwrite. As behaving >> as a data sink can be a somewhat reasonable feature of a >> quickly-written sh or eval plugin, we should not be so

On 8/31/23 10:55, Richard W.M. Jones wrote: > On Thu, Aug 31, 2023 at 10:40:53AM +0200, Laszlo Ersek wrote: >> On 8/31/23 00:21, Eric Blake wrote: >>> I hit another transient failure in libnbd CI when a poorly-written >>> eval script did not consume all of stdin during .pwrite. As behaving >>> as a data sink can be a somewhat reasonable feature of a

On Thu, Aug 31, 2023 at 12:05:17PM +0200, Laszlo Ersek wrote: > On 8/31/23 11:47, Richard W.M. Jones wrote: > > On Thu, Aug 31, 2023 at 11:12:59AM +0200, Laszlo Ersek wrote: > >> On 8/31/23 10:02, Richard W.M. Jones wrote: > >>> > >>> On Wed, Aug 30, 2023 at 05:21:19PM -0500, Eric Blake wrote: > >>>> I hit another transient failure in libnbd

On Thu, Aug 31, 2023 at 11:18:27AM +0200, Laszlo Ersek wrote: > On 8/31/23 10:55, Richard W.M. Jones wrote: > > On Thu, Aug 31, 2023 at 10:40:53AM +0200, Laszlo Ersek wrote: > >> On 8/31/23 00:21, Eric Blake wrote: > >>> I hit another transient failure in libnbd CI when a poorly-written > >>> eval script did not consume all of stdin during .pwrite. As

In the just-added nbdkit-sh-plugin.pod, you documented that unlike other plugins (where introspection or struct member population) can be used to determine which functionalities are supported, the shell plugin has to implement can_write and friends to return a special status 2 to document not supported, and 3 to indicate false, in part because you can't detect if a pwrite function is

This was a rather longer trip around the houses than I anticipated! The basic purpose of the patch series is to set $nbdkit_stdio_safe to "0" or "1" in sh and eval plugin scripts. To do that, I ended up adding a nicer way to manipulate environ lists, and to do that, I ended up adding a whole generic vector implementation which is applicable in a lot of different places.

See comment for explanation and https://listman.redhat.com/archives/libguestfs/2023-August/032468.html --- tests/Makefile.am | 2 + plugins/sh/call.c | 33 +++++++----- tests/test-sh-pwrite-ignore-stdin.sh | 77 ++++++++++++++++++++++++++++ 3 files changed, 100 insertions(+), 12 deletions(-) diff --git a/tests/Makefile.am b/tests/Makefile.am index

I'm pushing the first one as blatantly obvious. The second one is also simple enough, but not enough of a bug for me to push tonight. The third is something I noticed while working on sh, but is really more about docs vs. plugins in general. There, we could either change the code to match the docs (breaking backwards behavior for a plugin that set .errno_is_preserved=2) [what my patch did],

While nbdkit itself must run with SIGPIPE ignored, many applications expect to inherit SIGPIPE in the default state. What's worse, POSIX states that a non-interactive shell script cannot use 'trap' to undo an inherited SIG_IGN on SIGPIPE. I have seen several bug reports over the years of something that works for a developer but fails under a CI environment, where the root cause was

I thought this should produce a fairly dramatic performance gain. In fact I couldn't measure any performance difference at all. I think what's happening is we're actually paying an extra syscall (to discover the socket would block) and then doing the poll anyway. So I don't know if it's worth having this patch. It could be argued that it makes the code shorter (therefore

This isn't quite finished because not all of the tests or examples have been updated, but it demonstrates an idea: Should we forget about the concept of having multiple connections managed under a single handle? In this patch there is a single ‘struct nbd_handle *’ which manages a single state machine and connection (and therefore no nbd_connection). To connect to a multi-conn server you must

In order to trigger a non-EIO failure, a script had to output _exactly_ 'EINVAL ' or similar, with one trailing space and no message, because we forgot to limit the length of the string comparison. Fix things to tolerate an error name without a message, as well as to be case-insensitive. Also, parse EOVERFLOW (missed in commit 6f8c8084). Signed-off-by: Eric Blake

Since v1: - rebase to applied patches - split out support for Int in callbacks - sort of test that callbacks work in OCaml (see comment in patch 5) - rename API to nbd_pread_structured - expose error as explicit parameter to callback Eric Blake (5): generator: Allow Int in callbacks states: Wire in a read callback states: Add nbd_pread_structured API states: Add tests for

Our replacements for these functions are not undefined for negative values. Thanks: Eric Blake Fixes: commit 9f34db74786fdc92b290a7d47e4b003bd84fec69. --- plugins/sh/call.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/plugins/sh/call.c b/plugins/sh/call.c index b2d4a794..741022b6 100644 --- a/plugins/sh/call.c +++ b/plugins/sh/call.c @@ -443,7 +443,7 @@

Similar to the recent --mask-handshake command line added to nbdkit to test client fallbacks to crippled servers, it can be worth testing server fallbacks to crippled clients. And just as we have exposed whether the client will request structured replies, we can also expose whether the client will understand various handshake flags from the NBD protocol. Of course, we default to supporting all

Previously we performed a single call to recv(2) or send(2) (or the GnuTLS equivalents), and even if more data/space was immediately available to receive/send we would return to poll. Instead of this, loop until the socket returns EAGAIN. --- generator/states.c | 91 ++++++++++++++++++++++------------------------ 1 file changed, 43 insertions(+), 48 deletions(-) diff --git a/generator/states.c