similar to: [nbdkit PATCH 0/3] various nbdkit patches

v1 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00147.html v2 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00152.html v3: - Use optarg != NULL as a sentinel for has_arg. - Moved some variable decls into the inner loop. - Make nbdkit wrapper depend on config.status, so if srcdir or builddir changes then we rebuild the wrapper. It

This patch series solves the FreeBSD shebang problem in a completely different way, and a few other things besides. I propose that we replace ./nbdkit with a C program. The C program is a straightforward translation of the shell script. Some advantages of this approach are: - We can parse options in exactly the same way as the real program. - Use the more accurate ‘is_short_name’ test for

v1 was here: https://www.redhat.com/archives/libguestfs/2018-November/msg00147.html v2: - Use stdbool for booleans. - Use __attribute__((format(printf))). - Don't abort on invalid options, exit with failure instead. - Preserve long/short option choices in the output. - Add '=' in long option output, ie. always use --longopt=arg. - Add "--" parameter

This is a minimal implementation of an NBD forwarder; it lets us convert between old and newstyle connections (great if a client expects one style but the real server only provides the other), or add TLS safety on top of a server without having to rewrite that server. Right now, the real server is expected to live on a named Unix socket, and the transactions are serialized rather than

I worked out why valgrind wasn't being applied to nbdkit when run by many of the tests (patches 1-2). Unfortunately I'm not able to make it actually fail tests when valgrind fails. Although the situation is marginally improved in that you can now manually examine the *.log files and find valgrind failures that way. Also adds valgrinding of the Python plugin (patch 3). Along the way I

On 11/13/18 4:51 PM, Richard W.M. Jones wrote: > There are advantages to having the same code parse the options in the > ./nbdkit wrapper as in the real nbdkit: > > - We can parse options in exactly the same way as the real program. > > - Use the more accurate ‘is_short_name’ test for unadorned > plugin/filter names on the command line. > > - Fixes the FreeBSD

When we are valgrinding we don't really care about the child processes, which might be qemu, libvirtd, etc. So disable tracing into children (at least, as far as is possible with valgrind, which is not entirely disabling it, but suppressing it). --- m4/guestfs_progs.m4 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/m4/guestfs_progs.m4 b/m4/guestfs_progs.m4 index

On Sat, Nov 11, 2017 at 08:33:26PM -0600, Eric Blake wrote: > Note: ./nbdkit is a bit unsafe in that it blindly inherits $valgrind > from the environment; but that was nice because it let me do: > valgrind='gdb --args' ./nbdkit nbd ... > for a gdb debug session to work out the kinks in my code. Maybe we > want to support NBDKIT_GDB similarly to NBDKIT_VALGRIND, and to

On 12/2/18 10:39 AM, Richard W.M. Jones wrote: > By default valgrind suppresses many leaks. I'm not even sure exactly > how it decides which ones to suppress, but certainly global variables > pointing to malloc’d data are suppressed, which is not useful > behaviour. Here's my understanding of why that is the default - if you assign malloc()d storage into a global, then that

There are advantages to having the same code parse the options in the ./nbdkit wrapper as in the real nbdkit: - We can parse options in exactly the same way as the real program. - Use the more accurate ‘is_short_name’ test for unadorned plugin/filter names on the command line. - Fixes the FreeBSD problem with shebangs caused because FreeBSD refuses the use a shell script as a shebang path.

There are advantages to having the same code parse the options in the ./nbdkit wrapper as in the real nbdkit: - We can parse options in exactly the same way as the real program. - Use the more accurate ‘is_short_name’ test for unadorned plugin/filter names on the command line. - Fixes the FreeBSD problem with shebangs caused because FreeBSD refuses to use a shell script as a shebang path.

I had to add several suppressions because Python leaks memory by design, and also increase the max depth of valgrind stack traces because Python stack traces are very deep. Note that to do proper valgrinding of Python itself we would need to specially recompile Python: https://svn.python.org/projects/python/trunk/Misc/README.valgrind --- tests/test-lang-plugins.c | 3 +-

On 11/14/18 8:27 AM, Richard W.M. Jones wrote: > Fix the tests so they only enable valgrind workarounds or skips when > NBDKIT_VALGRIND is exactly set to "1" and not any other values > (including empty string, or not set). > > Thanks: Eric Blake. > --- > +++ b/tests/test-dump-plugin.sh > @@ -55,7 +55,8 @@ fi > # However some of these tests are expected to

On 11/14/18 3:45 AM, Richard W.M. Jones wrote: > On Tue, Nov 13, 2018 at 08:10:34PM -0600, Eric Blake wrote: >> On 11/13/18 4:51 PM, Richard W.M. Jones wrote: > [...] >>> + size_t i; >>> + char *s; >>> + >>> + /* If NBDKIT_VALGRIND=1 is set in the environment, then we run the >>> + * program under valgrind. This is used by the tests.

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

Currently we are unable to properly run guestfsd (the daemon) under valgrind. Attempts to run valgrind inside the appliance have not been successful (see patch 1/7). However we desperately need better valgrind coverage of the daemon, particularly because it is doing a lot of complex parsing of program output. This has been a problem for a long time. A better way to attack this problem is to

In v2: - Kernel command line parsing now moved to the appliance. - In the captive daemon test, the daemon cleanly shuts down on exit. - Add another btrfs test. Rich.

On 9/13/18 11:09 AM, Richard W.M. Jones wrote: > This resurrects the unused tests/functions.sh file (although now we > need to generate it from tests/functions.sh.in). Put the common code > for running a test against every plugin here. > > Because of the previous commits we can now use the plugins list > directly from configure.ac instead of needing to use weird shell >

On Tue, Nov 13, 2018 at 08:10:34PM -0600, Eric Blake wrote: > On 11/13/18 4:51 PM, Richard W.M. Jones wrote: [...] > >+ size_t i; > >+ char *s; > >+ > >+ /* If NBDKIT_VALGRIND=1 is set in the environment, then we run the > >+ * program under valgrind. This is used by the tests. Similarly if > >+ * NBDKIT_GDB=1 is set, we run the program under GDB,

I'm a lot happier with this version: no mucking with dlmopen(). It does add a bit more coupling between nbdkit proper and the vddk plugin (namely, nbdkit now exports a new function that the vddk plugin relies on), but patch 2 adds testsuite coverage of the new function so we shouldn't regress. Patch 1 and 2 are new, patch 3 is unchanged from when Rich posted it in v2, and patch 4 is