similar to: Fwd: nbdkit async

The concern is a client is blocked while processing a request. The nbdkit server design requires a thread per request being processed regardless of the number of connections or clients. We want to run 1000's of requests in parallel without needing a thread at nbdkit layer per request in flight. Our plugin layer is built around boost asio and a few threads in a worker pool running an io

> ----- Forwarded message ----- > > Date: Sat, 18 Feb 2017 22:21:19 -0500 > Subject: nbdkit async > > Hello, > > Hope this is the right person to contact regarding nbdkit design. > > I have a high latency massively parallel device that I am currently > implementing as an nbdkit plugin in c++ and have run into some design > limitations due to the synchronous

I want to make my nbd forwarding plugin fully parallel - but to do that, I first need to make nbdkit itself fully parallel ;) With this series, I was finally able to demonstrate out-of-order responses when using qemu-io (which is great at sending back-to-back requests prior to waiting for responses) coupled with the nbd file plugin (which has a great feature of rdelay and wdelay, to make it

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

Tested via: term1$ qemu-nbd -k $PWD/sock -t -f raw -x foo junk --trace=nbd_\* term2$ ./nbdkit -f -v -e bar nbd socket=$PWD/sock export=foo term3$ qemu-io -t none -f raw nbd://localhost:10809/bar --trace=nbd_\* and checking the traces to see that 'w 0 1' vs. 'w -f 0 1' was able to influence whether the FUA flag showed up at the server in term1. Still to go: figure out how to

v1 -> v2: - Fixed everything mentioned in the review. Rich.

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

These are a couple of patches in preparation for the Block Status implementation. While the patches (especially the second one) are very large they are really just elementary code motion. Rich.

Version 1 was here: https://www.redhat.com/archives/libguestfs/2019-April/msg00144.html Version 2 makes a couple of much larger changes: The OCaml patch changes the API of the pread method so it matches what other language bindings are already doing, ie. get the language plugin to return a newly allocated buffer, check it is long enough, copy out the data. The server patch implements a

The previous incomplete patch was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00049.html based on earlier discussion here: https://www.redhat.com/archives/libguestfs/2019-September/msg00047.html In v2: - The previous patch was incomplete. This version completes it by adding tests and extending nbdkit-sh-plugin. - nbdkit_export_name now returns NULL for error,

The first three patches are identical to: https://www.redhat.com/archives/libguestfs/2018-January/msg00079.html "[PATCH nbdkit v2 0/3] Refactor plugin_* functions into a backend" The rest of the patches add filters using the new filter API previously described here: https://www.redhat.com/archives/libguestfs/2018-January/msg00073.html This needs a lot more testing -- and tests --

I am making a plugin which extends ActiveRecord::Base I have the plugin working in my models in other aspects. I would like to pass in some data to the plugin, but I can''t get the data I would like. so if I have: class Stuff < ActiveRecord::Base belongs_to :thing my_plugin :id => self.thing.other_id end it chokes on figuring out what to do with self.thing.other_id.

This cleans up the existing code base with regards to implicit use of errno from language bindings, then rebases the previous work in python on top of that. I'm still playing with the perl bindings, but got further after reading 'perldoc perlembed'. Eric Blake (4): plugins: Don't use bogus errno from non-C plugins plugins: Add new nbdkit_set_error() utility function python:

v1 was here: https://www.redhat.com/archives/libguestfs/2020-February/msg00081.html v2 replaces struct connection *conn = GET_CONN; with GET_CONN; which sets conn implicitly and asserts that it is non-NULL. If we actually want to test if conn is non-NULL or behave differently, then you must use threadlocal_get_conn() instead, and some existing uses do that. Rich.

I have class in a plugin that I want to crack open and add some functionality to for a particular application. So, I created a file by the same name as the class in my app/models folder and added some methods to the class, but, I can''t seem to get Rails (2.2.2) to pick-up the extended definition. I did find while trying to debug the problem that if I paste the extended definition into,

The third patch is a large but mechanical change which gets rid of passing around struct connection * entirely within the server, preferring instead to reference the connection through thread-local storage. I hope this is a gateway to simplifying other parts of the code. Rich.

Maaaaaaaaaaaaajor sanity check needed. I wrote a simple little plugin, and it wouldn''t load. I put some comments in it''s init.rb, and it didn''t even spit those out. So, I dug into the initializer.rb (/vendor/rails/railties/lib/initializer.rb) to see wtf is going on, and found the problem... but... can''t explain it. Here is the code in question:

On 4/23/19 10:09 AM, Richard W.M. Jones wrote: > If the plugin .pread method did not fill the buffer with data then the > contents of the heap could be leaked back to the client. To avoid > this create a thread-local data buffer which is initialized to zero > and expanded (with zeroes) as required. > > This buffer is shared between pread and pwrite which makes the code a >

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that