similar to: [PATCH nbdkit] Experiment with parallel python plugin

This is a quick hack to experiment with parallel threading model in the python plugin. Changes: - Use aligned buffers to make it possible to use O_DIRECT. Using parallel I/O does not buy us much when using buffered I/O. pwrite() copies data to the page cache, and pread() reads data from the page cache. - Disable extents in the file plugin. This way we can compare it with the python

On Thu, Aug 06, 2020 at 11:22:00PM +0300, Nir Soffer wrote: > This is a quick hack to experiment with parallel threading model in the > python plugin. > > Changes: > > - Use aligned buffers to make it possible to use O_DIRECT. Using > parallel I/O does not buy us much when using buffered I/O. pwrite() > copies data to the page cache, and pread() reads data from the

On 4/23/19 10:09 AM, Richard W.M. Jones wrote: > If the plugin .pread method did not fill the buffer with data then the > contents of the heap could be leaked back to the client. To avoid > this create a thread-local data buffer which is initialized to zero > and expanded (with zeroes) as required. > > This buffer is shared between pread and pwrite which makes the code a >

Version 1 was here: https://www.redhat.com/archives/libguestfs/2019-April/msg00144.html Version 2 makes a couple of much larger changes: The OCaml patch changes the API of the pread method so it matches what other language bindings are already doing, ie. get the language plugin to return a newly allocated buffer, check it is long enough, copy out the data. The server patch implements a

We noticed while writing various libnbd tests that when the delay filter is in use, there are scenarios where we had to resort to SIGKILL to get rid of nbdkit, because it was non-responsive to SIGINT. I'm still trying to figure out the best way to add testsuite coverage of this, but already proved to myself that it works from the command line, under two scenarios that both used to cause long

If the plugin .pread method did not fill the buffer with data then the contents of the heap could be leaked back to the client. To avoid this create a thread-local data buffer which is initialized to zero and expanded (with zeroes) as required. This buffer is shared between pread and pwrite which makes the code a little bit simpler. Also this may improve locality by reusing the same memory for

When accepting a connection on a TCP or Unix domain socket we recorded the peer address in both the thread_data struct and thread-local storage. But for no reason because it was never used anywhere. Since we were only allocating a ‘struct sockaddr’ (rather than a ‘struct sockaddr_storage’) it's likely that some peer addresses would have been truncated. Remove all this code, it had no

On 8/13/19 5:36 PM, Richard W.M. Jones wrote: > The definition of functions that take a callback is changed so that > the callback and user_data are combined into a single structure, eg: > > int64_t nbd_aio_pread (struct nbd_handle *h, > void *buf, size_t count, uint64_t offset, > - int (*completion_callback) (/*..*/), void *user_data, > +

These patches should be ready to go in now; I will also post my work-in-progress for enabling full parallel handling that depends on these, but with that series, I was still getting crashes or hangs with test-socket-activation (I think I've nailed all the crashes I've seen, but the hang is rather insidious; see my other email

On 11/16/2017 09:13 PM, Eric Blake wrote: > We can't guarantee what storage duration the caller's request > for a thread name has; and in fact, if the caller uses > plugin_name() for their thread name, then the moment .unload > is called, our threadlocal storage is pointing to la-la-land > and we get a nice SEGV while trying to print any debug message. > So copy the

** NOT WORKING ** This patch shows how to integrate libnbd and the glib main loop. Posted mainly as a point of discussion as it doesn't quite work yet. Rich.

I just want to state that I hope we never implement the 16-bit Microsoft C++ ABI for -m16. On Tue, Jan 14, 2014 at 12:00 PM, Jim Grosbach <grosbach at apple.com> wrote: > Absolutely fantastic work, David. Thank you! > > On Jan 14, 2014, at 4:35 AM, David Woodhouse <dwmw2 at infradead.org> wrote: > > > Here's a brief update on the current state of 16-bit x86

I'm not sure if this is worth pursuing. On paper, it makes sense (if we know we have multiple commands batched to send over the wire, AND those commands are short in length, we might as well use MSG_MORE), but the measurement numbers with it applied might just be in the noise. Eric Blake (2): examples: Enhance access patterns of threaded-reads-and-writes states: Another use for MSG_MORE

On 2/11/20 11:15 AM, Richard W.M. Jones wrote: > Since we're going to be calling this function a lot, add a short alias > for it. > --- > server/internal.h | 1 + > server/public.c | 6 +++--- > 2 files changed, 4 insertions(+), 3 deletions(-) > > diff --git a/server/internal.h b/server/internal.h > index a1fa7309..1e7b4cf0 100644 > --- a/server/internal.h

This is working now, and incorporates all of the changes in Eric's review, *except* that it still doesn't retire commands (although this seems to make no obvious difference, except possibly a performance and memory impact). Rich.

Previously discussed here: https://www.redhat.com/archives/libguestfs/2019-July/msg00213.html It turns out that deferring callbacks is a PITA. (It would be a bit easier if C has closures.) However by rewriting the example we can avoid the need to use the cookie at all and make it run a bit more efficiently, so let's do that instead. Rich.

The docs are a lot more useful with a graphic showing how to wire together nbdkit as a bridge from old-to-new. The converse, bridging new-to-old, is best deferred until I add support for the nbd plugin connecting to a TCP socket. It is also worth mentioning use of nbdkit filters (after all, qemu-nbd 4.0 was able to deprecate its --partition option by pointing to 'nbdkit --filter=partition nbd

gcc 8 gripes (when using './configure --enable-gcc-warnings'): nbd.c: In function 'nbd_open': nbd.c:470:3: error: 'strncpy' specified bound 108 equals destination size [-Werror=stringop-truncation] strncpy (sock.sun_path, sockname, sizeof (sock.sun_path)); The warning is a false positive, given that we currently reject names >= sizeof(sock.sun_path), and thus we are

While svirt_t can be used for sockets it does not always guarantee that it will be accessible from a virtual machine. The VM might be running under svirt_tcg_t context which will need a svirt_tcg_t label on the socket in order to access it. There is, however, another label, svirt_socket_t, which is accessible from virt_domain: # sesearch -A -s svirt_t -c unix_stream_socket -p connectto ...

Here's a brief update on the current state of 16-bit x86 support... The assembler has support for the .code16 directive and can happily output 16-bit code. In pending patches¹ I have also added an i386-*-*-code16 triple and fixed the disassembler to support 16-bit mode (which was previously present, but could not be invoked and was fairly broken). And added a '-m16' option to clang.