similar to: [PATCH libnbd v2 0/3] Implement OClosures.

Patches 1-4 are basically uncontroversial, straightforward refactoring and IMHO we should just push them. Possibly 1-3 should be squashed together, but I posted them separately so they are easier to review. Patches 5 and 6 together implement OClosure. Patch 5 adds the feature and is simple to understand. Patch 6 changes the Closure completion callbacks into OClosure, but because it doesn't

This applies on top of the OClosure v2 series posted a few minutes ago. Rich.

This is a combination of these two earlier series: https://www.redhat.com/archives/libguestfs/2019-August/msg00235.html https://www.redhat.com/archives/libguestfs/2019-August/msg00240.html plus changes to allow .callback = NULL / .free != NULL, and to reduce the complexity of freeing callbacks. Although it's rather long there's nothing complex here. We might consider squashing some

Basically the same as this patch series, but for Python: https://www.redhat.com/archives/libguestfs/2019-August/msg00235.html plus adding the 590 asynch test at the end. Rich.

In v2: - now based on my proposal to add LIBNBD_SHUTDOWN_IMMEDIATE - four flags instead of two: STRICT_FLAGS is new (patch 4), and STRICT_BOUNDS is separate from STRICT_ZERO_SIZE (patch 5) - various refactorings for more shared code and less duplication Eric Blake (5): api: Add xxx_MASK constant for each Flags type generator: Refactor filtering of accepted OFlags api: Add

In this patch series we use the newly introduced free callback on the completion function to dererence the OCaml NBD.Buffer. I will make the same kind of change for Python later in a separate series. The completion function is always called at the C level, even if the OCaml program didn't use the optional argument. That's because the free callback doesn't run otherwise. There is a

Patches 1 & 2 are rather complex, but the end result is that we pass closures + user_data + free function in single struct parameters as I described previously in this email: https://www.redhat.com/archives/libguestfs/2019-August/msg00210.html Patch 3 adds a convenient FREE_CALLBACK macro which seems a worthwhile simplification if you buy into 1 & 2. Patch 4 adds another macro which is

This doesn't yet work. However it does make me more convinced than ever that we really need to sort out persistent buffer lifetimes in the library (similar to what we did for closures). Rich.

On 8/13/19 10:12 AM, Richard W.M. Jones wrote: > An optional Closure parameter, but otherwise works the same way as > Closure. > --- > generator/generator | 57 ++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 46 insertions(+), 11 deletions(-) > > @@ -4394,6 +4399,16 @@ let print_python_binding name { args; optargs; ret; may_set_error } = > ) args; >

As proposed here: https://www.redhat.com/archives/libguestfs/2019-August/msg00130.html I didn't actually read Eric's replies to that yet because I've been concentrating on writing these patches all day. Anyway here they are and I'll look at what Eric said about the proposal next. Rich.

On 8/13/19 5:06 AM, Richard W.M. Jones wrote: > An optional Closure parameter, but otherwise works the same way as > Closure. > @@ -3778,6 +3777,7 @@ let generate_lib_api_c () = > ) args; > List.iter ( > function > + | OClosure { cbname } -> pr ", %s_callback ? \"<fun>\" : \"NULL\"" cbname Well, it also permits a

Now runs a complete set of tests, notably including the AIO test. File descriptors are passed in and out as plain ints (instead of *os.File) for a couple of reasons: (1) We have to pass the plain int to syscall.Select. (2) Turning an fd into an os.File causes golang to set the blocking flag which is deeply unhelpful. Rich.

This doesn't change the C API, except that it is now permitted to pass NULL, NULL for the completion callback. For the Python and OCaml APIs the parameter changes to be an optional parameter. --- generator/generator | 36 +++++++++---------- ocaml/examples/asynch_copy.ml | 5 +-- .../test_505_aio_pread_structured_callback.ml | 12 ++++---

This doesn't change the C API, except that it is now permitted to pass NULL, NULL for the completion callback. For the Python and OCaml APIs the parameter changes to be an optional parameter. --- generator/generator | 36 +++++++++---------- ocaml/examples/asynch_copy.ml | 5 +-- .../test_505_aio_pread_structured_callback.ml | 12 ++++---

This largish series adds several new features to the generator. Enum maps to enumerated types (like enum in C). The only current use for this is replacing the nbd_set_tls (nbd, 0/1/2) parameter with LIBNBD_TLS_DISABLE, LIBNBD_TLS_ALLOW, LIBNBD_TLS_REQUIRE (and natural equivalents in other programming languages). Flags maps to any uint32_t bitmask. It is basically a non-optional, generalized

Inspection of the generated code showed several places where we did not release references on all error paths. In particular, switching things to always jump to an out: label, even when the underlying C function cannot fail, makes it easier to clean up when the user passes wrong types to a function call. One of the easiest triggers without this patch was this one-liner: $ nbdsh -c

These are some simplifications to the generator. They don't probably make much sense on their own, but they are preparatory to better handling of enums, and or'd lists of flags. Rich.

As promised in my earlier thread on libnbd completion callback question. Eric Blake (2): generator: Refactor handling of closures in unlocked functions generator: Free closures on failure docs/libnbd.pod | 2 +- generator/C.ml | 48 +++++++++++------ generator/C.mli | 1 + lib/debug.c | 7 +-- lib/opt.c | 31 ++++++-----

v1 was here: https://www.redhat.com/archives/libguestfs/2019-August/msg00103.html In v2 I've made several changes: - Fix Python callbacks so if they don't return something which is int-like, we assume they mean to return 0. - Add nbd.Buffer free() method. Read commit message in patch 2 to see what this is about. - Fixed the asynch copy test to deal with the unbelievably

On 8/13/19 10:37 AM, Richard W.M. Jones wrote: > The original nbd_aio_* (non-callback) functions are removed and > replaced with the renamed callback variants. > > This is a simple mechanical change to the API: > > (1) Any existing call to nbd_aio_*_callback can simply be renamed to > nbd_aio_* > > (2) Any existing call to nbd_aio_* must have two extra NULL