similar to: [PATCH 0/3] Alternate way to avoid race conditions when nbdkit exits.

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

v1 -> v2: - Fixed everything mentioned in the review. Rich.

These patches should be ready to go in now; I will also post my work-in-progress for enabling full parallel handling that depends on these, but with that series, I was still getting crashes or hangs with test-socket-activation (I think I've nailed all the crashes I've seen, but the hang is rather insidious; see my other email

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

The messages are not really useful to us, but they do bloat the debugging output of virt-v2v massively: nbdkit: python[1]: debug: acquire global request lock nbdkit: python[1]: debug: acquire per-connection request lock nbdkit: python[1]: debug: acquire unload prevention lock nbdkit: python[1]: debug: pwrite count=2097152 offset=4628414464 fua=0 nbdkit: python[1]: debug: release unload prevention

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

There are several race conditions where nbdkit exits (calling plugin_cleanup() which unloads the plugin), while the plugin is either in a callback or in plugin.close(). Avoid these by: (1) Taking a shared lock around all plugin callbacks. (2) Taking the same as an exclusive lock in plugin_cleanup. This delays plugin_cleanup until all callbacks have finished. (3) Removing a few unnecessary

On 11/16/2017 09:13 PM, Eric Blake wrote: > We can't guarantee what storage duration the caller's request > for a thread name has; and in fact, if the caller uses > plugin_name() for their thread name, then the moment .unload > is called, our threadlocal storage is pointing to la-la-land > and we get a nice SEGV while trying to print any debug message. > So copy the

Version 1 was here: https://www.redhat.com/archives/libguestfs/2019-April/msg00144.html Version 2 makes a couple of much larger changes: The OCaml patch changes the API of the pread method so it matches what other language bindings are already doing, ie. get the language plugin to return a newly allocated buffer, check it is long enough, copy out the data. The server patch implements a

On 11/14/2017 04:25 AM, Richard W.M. Jones wrote: > On Mon, Nov 13, 2017 at 12:42:48PM -0600, Eric Blake wrote: >> I'm observing a difference in timing on nbdkit shutdown that is >> dependent on client behavior, and I wonder if that's a bug, but I can't >> figure out where to patch it. >> >> When there is no connection present, sending SIGINT to nbdkit

We noticed while writing various libnbd tests that when the delay filter is in use, there are scenarios where we had to resort to SIGKILL to get rid of nbdkit, because it was non-responsive to SIGINT. I'm still trying to figure out the best way to add testsuite coverage of this, but already proved to myself that it works from the command line, under two scenarios that both used to cause long

When accepting a connection on a TCP or Unix domain socket we recorded the peer address in both the thread_data struct and thread-local storage. But for no reason because it was never used anywhere. Since we were only allocating a ‘struct sockaddr’ (rather than a ‘struct sockaddr_storage’) it's likely that some peer addresses would have been truncated. Remove all this code, it had no

Mostly code motion. --- src/Makefile.am | 1 + src/connections.c | 14 +++---- src/internal.h | 14 ++++--- src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 ---

Mostly code motion. --- src/Makefile.am | 1 + src/connections.c | 14 +++---- src/internal.h | 14 ++++--- src/locks.c | 115 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/plugins.c | 77 +++++------------------------------- 5 files changed, 142 insertions(+), 79 deletions(-) diff --git a/src/Makefile.am b/src/Makefile.am index 12b9043..1f05eab 100644 ---

On 4/23/19 10:09 AM, Richard W.M. Jones wrote: > If the plugin .pread method did not fill the buffer with data then the > contents of the heap could be leaked back to the client. To avoid > this create a thread-local data buffer which is initialized to zero > and expanded (with zeroes) as required. > > This buffer is shared between pread and pwrite which makes the code a >

This patch isn't complete (patch 6/6 isn't finished) so it's just for discussion, although it does compile and run. This introduces to nbdkit a concept of "filters" which can be placed in front of plugins to modify their behaviour. Some examples where you might use filters: * Serve a subset of the data, such as (offset, range) or a single partition from a disk image.

This is a quick hack to experiment with parallel threading model in the python plugin. Changes: - Use aligned buffers to make it possible to use O_DIRECT. Using parallel I/O does not buy us much when using buffered I/O. pwrite() copies data to the page cache, and pread() reads data from the page cache. - Disable extents in the file plugin. This way we can compare it with the python

I'm probably missing something obvious here, but naively, this makes it kind of awkward to have mutable thread local state... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20141212/53b577ae/attachment.html>

Reza Sheykhi <hajishey at msu.edu> writes: > I got the following answers: > > which perl > /usr/bin/perl > > which llvm-config > /usr/local/bin/llvm-config > > which llvm-as > /usr/local/bin/llvm-as > > /usr/bin/llvm-confing --version > bash: /usr/bin/llvm-confing: No such file or directory Uh, there is a typo on the command above, it should be