similar to: [PATCH libnbd 0/3] Use free callback to hold ref to AIO buffer.

This is a combination of these two earlier series: https://www.redhat.com/archives/libguestfs/2019-August/msg00235.html https://www.redhat.com/archives/libguestfs/2019-August/msg00240.html plus changes to allow .callback = NULL / .free != NULL, and to reduce the complexity of freeing callbacks. Although it's rather long there's nothing complex here. We might consider squashing some

v1 was here: https://www.redhat.com/archives/libguestfs/2019-August/msg00103.html In v2 I've made several changes: - Fix Python callbacks so if they don't return something which is int-like, we assume they mean to return 0. - Add nbd.Buffer free() method. Read commit message in patch 2 to see what this is about. - Fixed the asynch copy test to deal with the unbelievably

This doesn't yet work. However it does make me more convinced than ever that we really need to sort out persistent buffer lifetimes in the library (similar to what we did for closures). Rich.

Patches 1-4 are basically uncontroversial, straightforward refactoring and IMHO we should just push them. Possibly 1-3 should be squashed together, but I posted them separately so they are easier to review. Patches 5 and 6 together implement OClosure. Patch 5 adds the feature and is simple to understand. Patch 6 changes the Closure completion callbacks into OClosure, but because it doesn't

In this patch series we use the newly introduced free callback on the completion function to dererence the OCaml NBD.Buffer. I will make the same kind of change for Python later in a separate series. The completion function is always called at the C level, even if the OCaml program didn't use the optional argument. That's because the free callback doesn't run otherwise. There is a

v1 was here: https://www.redhat.com/archives/libguestfs/2019-August/msg00168.html I pushed uncontroversial patches 1-4 v2: - The implementation of OClosure (new patch 1) in Python is fixed. - Patch 2 (old patch 5) is unchanged. - I added a new API for removing debug callbacks. I think this approach has some advantages over using OClosure. - I didn't yet do any work on changing the

Patches 1 & 2 are rather complex, but the end result is that we pass closures + user_data + free function in single struct parameters as I described previously in this email: https://www.redhat.com/archives/libguestfs/2019-August/msg00210.html Patch 3 adds a convenient FREE_CALLBACK macro which seems a worthwhile simplification if you buy into 1 & 2. Patch 4 adds another macro which is

v1 was here: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00231 The changes address everything that Eric picked up in his review of the first two patches. I have also added two more patches (4 and 5) which respectively fix docs and change int status -> unsigned status, as discussed. Passes make, check, check-valgrind. Rich.

This implements most of what I wrote here: https://www.redhat.com/archives/libguestfs/2019-July/msg00213.html

As proposed here: https://www.redhat.com/archives/libguestfs/2019-August/msg00130.html I didn't actually read Eric's replies to that yet because I've been concentrating on writing these patches all day. Anyway here they are and I'll look at what Eric said about the proposal next. Rich.

Inspection of the generated code showed several places where we did not release references on all error paths. In particular, switching things to always jump to an out: label, even when the underlying C function cannot fail, makes it easier to clean up when the user passes wrong types to a function call. One of the easiest triggers without this patch was this one-liner: $ nbdsh -c

As before, but this one has working Python bindings. OCaml still TBD. Rich.

I think I've addressed everything that was raised in review. Some of the highlights: - Callbacks should be freed reliably along all exit paths. - There's a simple test of closure lifetimes. - I've tried to use VALID|FREE in all the places where I'm confident that it's safe and correct to do. There may be more places. Note this is an optimization and shouldn't

These are some simplifications to the generator. They don't probably make much sense on their own, but they are preparatory to better handling of enums, and or'd lists of flags. Rich.

On 8/13/19 10:12 AM, Richard W.M. Jones wrote: > An optional Closure parameter, but otherwise works the same way as > Closure. > --- > generator/generator | 57 ++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 46 insertions(+), 11 deletions(-) > > @@ -4394,6 +4399,16 @@ let print_python_binding name { args; optargs; ret; may_set_error } = > ) args; >

On 7/24/19 7:17 AM, Richard W.M. Jones wrote: > Previously closures had a crude flag which tells if they are > persistent or transient. Transient closures (flag = false) last for > the lifetime of the currently called libnbd function. Persistent > closures had an indefinite lifetime which could last for as long as > the handle. In language bindings handling persistent closures

On 7/16/19 6:04 AM, Richard W.M. Jones wrote: > A Closure is a list of (usually one, but can be more) closures. In C > there is also a singe ‘void *user_data’ parameter which is passed by > the caller into the function and through as the first parameter of > each callback invocation. > > By grouping the previously separate Opaque and Callback* parameters > together we can

On 8/13/19 5:06 AM, Richard W.M. Jones wrote: > An optional Closure parameter, but otherwise works the same way as > Closure. > @@ -3778,6 +3777,7 @@ let generate_lib_api_c () = > ) args; > List.iter ( > function > + | OClosure { cbname } -> pr ", %s_callback ? \"<fun>\" : \"NULL\"" cbname Well, it also permits a

On 8/13/19 10:37 AM, Richard W.M. Jones wrote: > The original nbd_aio_* (non-callback) functions are removed and > replaced with the renamed callback variants. > > This is a simple mechanical change to the API: > > (1) Any existing call to nbd_aio_*_callback can simply be renamed to > nbd_aio_* > > (2) Any existing call to nbd_aio_* must have two extra NULL

For commands with long names and/or short descriptors, you can end up going over 72 characters in the first line of the man page (causing podwrapper to complain). Wrap these lines. --- generator/generator | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/generator/generator b/generator/generator index 7d3f656..ad1cb6b 100755 --- a/generator/generator +++ b/generator/generator