similar to: [libnbd PATCH] lib: Consolidate free callbacks to just happen at retire time

On Wed, Aug 14, 2019 at 05:38:31PM -0500, Eric Blake wrote: > When we introduced valid_flags, there was an incentive to do as few > callbacks as possible, favoring cb(VALID|FREE) calls over the sequence > cb(VALID);cb(FREE). To make it work, we set .callback=NULL after an > early free, so that the later check during retirement didn't free > again. > > But now that our

Patches 1 & 2 are rather complex, but the end result is that we pass closures + user_data + free function in single struct parameters as I described previously in this email: https://www.redhat.com/archives/libguestfs/2019-August/msg00210.html Patch 3 adds a convenient FREE_CALLBACK macro which seems a worthwhile simplification if you buy into 1 & 2. Patch 4 adds another macro which is

This is a combination of these two earlier series: https://www.redhat.com/archives/libguestfs/2019-August/msg00235.html https://www.redhat.com/archives/libguestfs/2019-August/msg00240.html plus changes to allow .callback = NULL / .free != NULL, and to reduce the complexity of freeing callbacks. Although it's rather long there's nothing complex here. We might consider squashing some

Another simple internal macro, this time encapsulating calling a callback. --- generator/states-reply-simple.c | 8 ++++---- generator/states-reply-structured.c | 31 ++++++++++++++--------------- generator/states-reply.c | 2 +- generator/states.c | 2 +- lib/debug.c | 2 +- lib/internal.h | 4 ++++ 6 files

Simple macro encapsulating the process for freeing a callback. --- generator/states-reply-simple.c | 4 +-- generator/states-reply-structured.c | 42 +++++++++-------------------- generator/states-reply.c | 4 +-- generator/states.c | 4 +-- lib/aio.c | 17 ++++-------- lib/debug.c | 6 +---- lib/internal.h

We have defined the concept of a "null callback" meaning one where the .callback field = NULL. The first two new macros just test this property, and the third one sets a callback to null. This change is neutral refactoring. --- generator/generator | 4 ++-- generator/states-reply-simple.c | 2 +- generator/states-reply-structured.c | 11 ++++++-----

As proposed here: https://www.redhat.com/archives/libguestfs/2019-August/msg00130.html I didn't actually read Eric's replies to that yet because I've been concentrating on writing these patches all day. Anyway here they are and I'll look at what Eric said about the proposal next. Rich.

Change the way that we deal with freeing closures in language bindings: * The valid_flag is removed (mostly reverting commit 2d9b98e96772e282d51dafac07f16387dadc8afa). * An extra ‘free’ parameter is added to all callback structures. This is called by the library whenever the closure won't be called again (so the user_data can be freed). This is analogous to valid_flag ==

Right now, we require the user to supply potential metacontext names in advance, without knowing what the server actually supports until after the connection or nbd_opt_info call is complete. But the NBD protocol also supports a client being able to query what contexts a server supports, including where a client asks with a prefix and the server replies back with all answers that match the

The freeing feature can now be done by associating a free callback with a closure's user_data, so having the valid_flag is no longer necessary and it can be completely removed. This mostly reverts commit 2d9b98e96772e282d51dafac07f16387dadc8afa. --- TODO | 2 - docs/libnbd.pod | 64 ++-------------- examples/glib-main-loop.c |

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake

Mechanical change: Wherever we call any closure with the LIBNBD_CALLBACK_FREE function, we also call nbd_internal_free_callback with the closure's user_data. This allows calls to associate a free callback with any closure via its user_data pointer. --- generator/states-reply-simple.c | 1 + generator/states-reply-structured.c | 24 ++++++++++++++++++------ generator/states-reply.c

Rather than repeating the logic for checking flags at multiple sites, stick it in a new state. --- generator/generator | 7 +++++ generator/states-reply-structured.c | 43 ++++++++++------------------- 2 files changed, 22 insertions(+), 28 deletions(-) diff --git a/generator/generator b/generator/generator index a289741..4c81859 100755 --- a/generator/generator +++

If the server passes us a malformed error reply type with a message length longer than the overall structured reply, we would blindly obey that size and get out of sync with the server (perhaps even hanging on a read for data that will never come). Broken since its introduction in commit 28952eda. Fix it by parsing the tail of an error separate from the message, which also lets us add other

I've mentioned this topic before (in fact, the idea of adding NBD_CMD_FLAG_DF was first mentioned at [1]), but finally finished enough of an implementation to feel confident in posting it. I'd still like to add something under examples/ that uses the new API to implement strict checking of a server's structured replies read implementation (ensure that a server never sends data after

This continues the changes for adding NBD_OPT_LIST support. Now, instead of libnbd malloc'ing storage itself, the user passes a callback that can handle name/description pairs however it likes, and we get rid of the artificial cap at 10000 exports. However, the user will probably end up malloc'ing a list themselves, as we can't call nbd_set_export_name, or even request NBD_OPT_INFO

This was v3 patch 2/22, reworked to address the confusion about how a structured reply header is read in two pieces before getting to the payload portion. I'm still working on rebasing the rest of my v3 series (patches 1, 3-22) from other comments given, but this seemed independent enough that it's worth posting now rather than holding it up for the rest of the series. Eric Blake (4):

As promised in my earlier thread on libnbd completion callback question. Eric Blake (2): generator: Refactor handling of closures in unlocked functions generator: Free closures on failure docs/libnbd.pod | 2 +- generator/C.ml | 48 +++++++++++------ generator/C.mli | 1 + lib/debug.c | 7 +-- lib/opt.c | 31 ++++++-----

The definition of functions that take a callback is changed so that the callback and user_data are combined into a single structure, eg: int64_t nbd_aio_pread (struct nbd_handle *h, void *buf, size_t count, uint64_t offset, - int (*completion_callback) (/*..*/), void *user_data, + nbd_completion_callback completion_callback, uint32_t flags); Several

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-