similar to: [libnbd PATCH] python: Plug some memory leaks on error paths

h.nbd_connect_command was leaking a python object per parameter, and also leaks memory on failure. In fact, it was possible to segfault on something as trivial as: nbdsh -c 'h.connect_command(["true",1])' h.nbd_pread was leaking a read buffer on every single synchronous read. My previous patch to address closure callbacks had a bug in h.nbd_pread_structured and similar: if the

On 8/13/19 10:12 AM, Richard W.M. Jones wrote: > An optional Closure parameter, but otherwise works the same way as > Closure. > --- > generator/generator | 57 ++++++++++++++++++++++++++++++++++++--------- > 1 file changed, 46 insertions(+), 11 deletions(-) > > @@ -4394,6 +4399,16 @@ let print_python_binding name { args; optargs; ret; may_set_error } = > ) args; >

We were previously generating one instance of the Python closure wrapper per (function * Closure arg). However these wrappers didn't actually differ across functions. We can therefore save a lot of code by only generating one wrapper per closure globally. This reduces the amount of generated code by nearly 25%. Before and after: $ wc -l python/methods.c 3275 python/methods.c $ wc -l

On 8/13/19 5:06 AM, Richard W.M. Jones wrote: > An optional Closure parameter, but otherwise works the same way as > Closure. > @@ -3778,6 +3777,7 @@ let generate_lib_api_c () = > ) args; > List.iter ( > function > + | OClosure { cbname } -> pr ", %s_callback ? \"<fun>\" : \"NULL\"" cbname Well, it also permits a

In preparation for closure lifetimes, split up the Closure so it no longer describes a list of closures, but a single callback. This changes the API because functions which take 2 or more closures now pass a separate user_data for each one. --- docs/libnbd.pod | 3 +- examples/strict-structured-reads.c | 2 +- generator/generator | 760

An optional Closure parameter, but otherwise works the same way as Closure. --- generator/generator | 57 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 46 insertions(+), 11 deletions(-) diff --git a/generator/generator b/generator/generator index 8cf95b6..8f15786 100755 --- a/generator/generator +++ b/generator/generator @@ -867,6 +867,7 @@ and arg = | UInt32 of string

An optional Closure parameter, but otherwise works the same way as Closure. --- generator/generator | 54 ++++++++++++++++++++++++++++++++++++--------- 1 file changed, 44 insertions(+), 10 deletions(-) diff --git a/generator/generator b/generator/generator index 01da1c3..3add9a4 100755 --- a/generator/generator +++ b/generator/generator @@ -867,6 +867,7 @@ and arg = | UInt32 of string

Previously closures had a crude flag which tells if they are persistent or transient. Transient closures (flag = false) last for the lifetime of the currently called libnbd function. Persistent closures had an indefinite lifetime which could last for as long as the handle. In language bindings handling persistent closures was wasteful as we needed to register a "close callback" to

If we accept that callbacks will never handle the full range of parameters then we can simplify the generator quite a bit by using a special type for closure args vs normal method args. This removes many asserts and quite a bit of unreachable code (eg. Python code for handling ArrayAndLen in normal methods that was never used). The output of the generator after this commit should be identical.

A Closure is a list of (usually one, but can be more) closures. In C there is also a singe ‘void *user_data’ parameter which is passed by the caller into the function and through as the first parameter of each callback invocation. By grouping the previously separate Opaque and Callback* parameters together we can avoid the awkward situation where we have to scan through the argument list to try

Change the way that we deal with freeing closures in language bindings: * The valid_flag is removed (mostly reverting commit 2d9b98e96772e282d51dafac07f16387dadc8afa). * An extra ‘free’ parameter is added to all callback structures. This is called by the library whenever the closure won't be called again (so the user_data can be freed). This is analogous to valid_flag ==

The definition of functions that take a callback is changed so that the callback and user_data are combined into a single structure, eg: int64_t nbd_aio_pread (struct nbd_handle *h, void *buf, size_t count, uint64_t offset, - int (*completion_callback) (/*..*/), void *user_data, + nbd_completion_callback completion_callback, uint32_t flags); Several

Patches 1-4 are basically uncontroversial, straightforward refactoring and IMHO we should just push them. Possibly 1-3 should be squashed together, but I posted them separately so they are easier to review. Patches 5 and 6 together implement OClosure. Patch 5 adds the feature and is simple to understand. Patch 6 changes the Closure completion callbacks into OClosure, but because it doesn't

On Tue, Aug 13, 2019 at 06:34:11AM -0500, Eric Blake wrote: > On 8/13/19 5:06 AM, Richard W.M. Jones wrote: > > An optional Closure parameter, but otherwise works the same way as > > Closure. > > > @@ -3778,6 +3777,7 @@ let generate_lib_api_c () = > > ) args; > > List.iter ( > > function > > + | OClosure { cbname } -> pr

The freeing feature can now be done by associating a free callback with a closure's user_data, so having the valid_flag is no longer necessary and it can be completely removed. This mostly reverts commit 2d9b98e96772e282d51dafac07f16387dadc8afa. --- TODO | 2 - docs/libnbd.pod | 64 ++-------------- examples/glib-main-loop.c |

This works just like OFlags but is a non-optional argument. --- generator/generator | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/generator/generator b/generator/generator index 2b37cea..96d1148 100755 --- a/generator/generator +++ b/generator/generator @@ -856,6 +856,7 @@ and arg = | BytesPersistOut of string * string | Closure of closure (* function pointer

Rather than having to open-code the list of accepted command flags in the unlocked version of each command, we can store that information in the generator to produce the check directly in the public API. --- generator/API.ml | 53 +++++++++++++++++++++++++++++---------------- generator/API.mli | 3 ++- generator/C.ml | 26 +++++++++++++++------- generator/GoLang.ml | 10 ++++-----

Previously nbd_set_tls had an integer argument which was 0 for disable, 1 for allow and 2 for require. This commit adds a proper enumerated type to describe this, defining LIBNBD_TLS_DISABLE = 0, LIBNBD_TLS_ALLOW = 1 and LIBNBD_TLS_REQUIRE = 2. (Note the C API doesn't change). In C the enumerated type is still defined and passed as an int (not as an enum). While we could define an enum

This allows us to connect directly to a connected socket. How exactly the socket is created and connected to the NBD server is left up to the main program. The only real complexity in this patch is allowing file descriptors to be passed to libnbd APIs. Luckily in C and Python we can treat them exactly as integers, and in OCaml they are also integers but with a different type. ---

For now, there is no change to the generated code (although the man pages are slightly improved), but this will make it easier for later patches to improve OCaml and Golang bindings so that set/get pairs are typed symmetrically. --- generator/API.ml | 16 ++++++++++------ generator/API.mli | 2 ++ generator/C.ml | 20 ++++++++++++-------- generator/GoLang.ml | 13 ++++++++++++-