similar to: [PATCH libnbd proposal] api: Add semi-private function for freeing persistent data.

On 8/11/19 8:03 AM, Richard W.M. Jones wrote: > This adds a C-only semi-private function for freeing various types of > persistent data passed to libnbd. > > There are some similarities with nbd_add_close_callback which we > removed in commit 7f191b150b52ed50098976309a6af883d245fc56. > --- > generator/generator | 46 +++++++++++++++++++++++++++++++++++++++++++++ > 1 file

This adds a C-only semi-private function for freeing various types of persistent data passed to libnbd. There are some similarities with nbd_add_close_callback which we removed in commit 7f191b150b52ed50098976309a6af883d245fc56. --- generator/generator | 57 ++++++++++++++++++++ lib/Makefile.am | 1 + lib/free.c | 129 ++++++++++++++++++++++++++++++++++++++++++++ lib/handle.c

As proposed here: https://www.redhat.com/archives/libguestfs/2019-August/msg00130.html I didn't actually read Eric's replies to that yet because I've been concentrating on writing these patches all day. Anyway here they are and I'll look at what Eric said about the proposal next. Rich.

On 8/12/19 11:08 AM, Richard W.M. Jones wrote: > This adds a C-only semi-private function for freeing various types of > persistent data passed to libnbd. > > There are some similarities with nbd_add_close_callback which we > removed in commit 7f191b150b52ed50098976309a6af883d245fc56. > --- > +=head1 FREE CALLBACKS > + > +B<Note:> The API described in this

On 8/12/19 1:13 PM, Richard W.M. Jones wrote: > On Mon, Aug 12, 2019 at 11:29:10AM -0500, Eric Blake wrote: >> On 8/12/19 11:08 AM, Richard W.M. Jones wrote: >>> + typedef void (*nbd_free_callback) (void *ptr, void *user_data); >>> + int nbd_add_free_callback (struct nbd_handle *h, >>> + void *ptr, >>> +

On Mon, Aug 12, 2019 at 11:29:10AM -0500, Eric Blake wrote: > On 8/12/19 11:08 AM, Richard W.M. Jones wrote: > > + typedef void (*nbd_free_callback) (void *ptr, void *user_data); > > + int nbd_add_free_callback (struct nbd_handle *h, > > + void *ptr, > > + nbd_free_callback cb, > > +

By using the free callback mechanism we don't need to manually free the buffer. This reverts part of commit fef1c281a65d061127bf178e5f8cfca0a2475c56. --- generator/generator | 43 +++++++++++++++++++++----------- ocaml/buffer.c | 22 ---------------- ocaml/examples/asynch_copy.ml | 4 +-- ocaml/libnbd-ocaml.pod | 22 ---------------- ocaml/nbd-c.h

v1 was here: https://www.redhat.com/archives/libguestfs/2019-August/msg00103.html In v2 I've made several changes: - Fix Python callbacks so if they don't return something which is int-like, we assume they mean to return 0. - Add nbd.Buffer free() method. Read commit message in patch 2 to see what this is about. - Fixed the asynch copy test to deal with the unbelievably

On Mon, Aug 12, 2019 at 11:00:01PM +0100, Richard W.M. Jones wrote: > On Mon, Aug 12, 2019 at 01:53:56PM -0500, Eric Blake wrote: > > We then have the design question of whether to make an OClosure type, > > where C has two functions nbd_aio_pread and nbd_aio_pread_callback for > > convenience, but where other languages have only a single nbd.aio_pread > > where the

The freeing feature can now be done by associating a free callback with a closure's user_data, so having the valid_flag is no longer necessary and it can be completely removed. This mostly reverts commit 2d9b98e96772e282d51dafac07f16387dadc8afa. --- TODO | 2 - docs/libnbd.pod | 64 ++-------------- examples/glib-main-loop.c |

On Thu, Aug 22, 2019 at 03:39:35PM +0100, Richard W.M. Jones wrote: >Previously the inner loop would issue nbd.pread() requests >synchronously, meaning that we would issue a request for each data >block from the nbdkit server (which would in turn synchronously >request the data from VMware) and wait until nbdkit replies before >continuing. > >This converts the inner loop so it

This doesn't yet work. However it does make me more convinced than ever that we really need to sort out persistent buffer lifetimes in the library (similar to what we did for closures). Rich.

This applies on top of patches 1 & 2 here (instead of patch 3): https://www.redhat.com/archives/libguestfs/2019-May/msg00206.html https://www.redhat.com/archives/libguestfs/2019-May/msg00207.html Rich.

This is based on top of: https://github.com/nertpinx/v2v-conversion-host/commit/0bb2efdcacd975a2cae7380080991ac7fc238d2b The first 4 patches are fairly uncontroversial miscellaneous cleanups. Patch 5 is the interesting one. (Note it doesn't quite work yet, so it's for discussion only.) Patch 5 converts the inner loop to use asynchronous libnbd calls. performance improves quite a bit for

On 8/13/19 10:37 AM, Richard W.M. Jones wrote: > The original nbd_aio_* (non-callback) functions are removed and > replaced with the renamed callback variants. > > This is a simple mechanical change to the API: > > (1) Any existing call to nbd_aio_*_callback can simply be renamed to > nbd_aio_* > > (2) Any existing call to nbd_aio_* must have two extra NULL

RFC because this is an API break, but we haven't declared stable API yet. If we like it, I'm working on using libnbd to implement the nbdkit-nbd plugin; knowing whether it is API version 0.1 or 0.2 will be useful. I also dabbled with allowing optional parameters in python, although my OCaml is weak enough that there may be cleaner ways to approach that. Eric Blake (4): api: Add flags

This is a simple patch which stops nbd_close from waiting too long for a server subprocess to shut down. I wanted to send SIGHUP because the server will be able to catch it and do a clean shutdown if that is required. Is another signal better? Is it right to send a signal here? Rich.

In this patch series we use the newly introduced free callback on the completion function to dererence the OCaml NBD.Buffer. I will make the same kind of change for Python later in a separate series. The completion function is always called at the C level, even if the OCaml program didn't use the optional argument. That's because the free callback doesn't run otherwise. There is a

Instead of using the valid_flag == FREE mechanism, use a free callback to free each closure root. --- generator/generator | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/generator/generator b/generator/generator index 92ce170..109fad6 100755 --- a/generator/generator +++ b/generator/generator @@ -5239,9 +5239,6 @@ let print_ocaml_binding (name, { args; optargs; ret })

Same as the previous commit, but for Python. --- generator/generator | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/generator/generator b/generator/generator index 109fad6..a0322ee 100755 --- a/generator/generator +++ b/generator/generator @@ -4315,9 +4315,6 @@ let print_python_binding name { args; optargs; ret; may_set_error } = ) cbargs; pr