search for: nbd_internal_string_list_length

This adds coalescing of adjacent extents of the same type, as mentioned by Eric Blake in the commit message here: https://github.com/libguestfs/libnbd/commit/46072f6611f80245846a445766da071e457b00cd The patch series is rather long because it detours through adding the <vector.h> library from nbdkit into libnbd and replacing ad hoc uses of realloc, char ** etc in various places. Rich.

...far. diff --git a/lib/handle.c b/lib/handle.c index a6b2172..e0047b7 100644 --- a/lib/handle.c +++ b/lib/handle.c @@ -313,6 +313,38 @@ nbd_unlocked_add_meta_context (struct nbd_handle *h, const char *name) return 0; } +int +nbd_unlocked_get_nr_meta_contexts (struct nbd_handle *h) +{ + return nbd_internal_string_list_length (h->request_meta_contexts); +} + +char * +nbd_unlocked_get_meta_context (struct nbd_handle *h, int i) +{ + size_t len = nbd_internal_string_list_length (h->request_meta_contexts); + char *ret; + + if (i < 0 || i >= len) { + set_error (EINVAL, "meta context request out of rang...

I'm posting this now, as I'm at the end of a workday and I got things working for manual experimentation. Still to do: - write interop tests for qemu-nbd and nbdkit (including my proposed patch addition of qemu-nbd -A to show qemu:allocation-depth) - figure out if we can make 'nbdinfo --map' use the new API to automatically select all contexts advertised by the server Eric Blake

...x e0047b7..7adc6d1 100644 --- a/lib/handle.c +++ b/lib/handle.c @@ -320,12 +320,12 @@ nbd_unlocked_get_nr_meta_contexts (struct nbd_handle *h) } char * -nbd_unlocked_get_meta_context (struct nbd_handle *h, int i) +nbd_unlocked_get_meta_context (struct nbd_handle *h, size_t i) { size_t len = nbd_internal_string_list_length (h->request_meta_contexts); char *ret; - if (i < 0 || i >= len) { + if (i >= len) { set_error (EINVAL, "meta context request out of range"); return NULL; } -- 2.28.0.rc2

v1 was posted here: https://www.redhat.com/archives/libguestfs/2019-September/thread.html#00337 v2: - Drop the first patch. - Hopefully fix the multiple issues with fork-safety and general behaviour on error paths. Note this requires execvpe for which there seems to be no equivalent on FreeBSD, except some kind of tedious path parsing (but can we assign to environ?) Rich.

...e skip the group if the client didn't request any metadata - * contexts. + * contexts, when doing SET (but an empty LIST is okay). */ assert (h->gflags & LIBNBD_HANDSHAKE_FLAG_FIXED_NEWSTYLE); nbd_internal_reset_size_and_flags (h); - if (!h->structured_replies || - nbd_internal_string_list_length (h->request_meta_contexts) == 0) { - SET_NEXT_STATE (%^OPT_GO.START); - return 0; + if (h->opt_current == NBD_OPT_LIST_META_CONTEXT) { + assert (h->opt_mode); + assert (h->structured_replies); + assert (CALLBACK_IS_NOT_NULL (h->opt_cb.fn.context)); + opt = h->op...

...ORE; SET_NEXT_STATE (%SEND_EXPORTNAMELEN); } return 0; @@ -67,6 +69,7 @@ case 0: h->wbuf = h->export_name; h->wlen = strlen (h->export_name); + h->wflags = MSG_MORE; SET_NEXT_STATE (%SEND_EXPORTNAME); } return 0; @@ -79,6 +82,7 @@ htobe32 (nbd_internal_string_list_length (h->request_meta_contexts)); h->wbuf = &h->sbuf; h->wlen = sizeof h->sbuf.nrqueries; + h->wflags = MSG_MORE; SET_NEXT_STATE (%SEND_NRQUERIES); } return 0; @@ -103,6 +107,7 @@ h->sbuf.len = htobe32 (strlen (query)); h->wbuf = &h->sbuf....

..._exit (127); + else + _exit (126); } /* Parent. */ diff --git a/lib/internal.h b/lib/internal.h index bdb0e83..31bc3d4 100644 --- a/lib/internal.h +++ b/lib/internal.h @@ -384,5 +384,7 @@ extern void nbd_internal_hexdump (const void *data, size_t len, FILE *fp); extern size_t nbd_internal_string_list_length (char **argv); extern char **nbd_internal_copy_string_list (char **argv); extern void nbd_internal_free_string_list (char **argv); +extern const char *nbd_internal_fork_safe_itoa (long v, char *buf, size_t len); +extern void nbd_internal_fork_safe_perror (const char *s); #endif /* LIBNBD_INTER...

...x"); + if (p0 == NULL) + goto err; + p1 = strdup ("LISTEN_FDS=1"); + if (p1 == NULL) + goto err; + + /* Copy the current environment. */ + env = nbd_internal_copy_string_list (environ); + if (env == NULL) + goto err; + + /* Reserve slots env[0] and env[1]. */ + len = nbd_internal_string_list_length (env); + vp = realloc (env, + sizeof (char *) * (len + 3 /* include final NULL entry */)); + if (vp == NULL) + goto err; + env = vp; + memmove (&env[2], &env[0], sizeof (char *) * (len + 1)); + + env[0] = p0; /* Code below assumes env[0] is LISTEN_PID. */ +...

...te); extern enum state_group nbd_internal_state_group_parent (enum state_group group); +#define set_state(h,next_state) ((h)->state) = (next_state) +#define get_state(h) ((h)->state) + /* utils.c */ extern void nbd_internal_hexdump (const void *data, size_t len, FILE *fp); extern size_t nbd_internal_string_list_length (char **argv); diff --git a/lib/is-state.c b/lib/is-state.c index 55d103b..51a2d47 100644 --- a/lib/is-state.c +++ b/lib/is-state.c @@ -104,40 +104,40 @@ nbd_internal_is_state_closed (enum state state) int nbd_unlocked_aio_is_created (struct nbd_handle *h) { - return nbd_internal_is_state_creat...

Appears to have a measurable benefit, see 3/3 for test results. Rich.

...PID or LISTEN_FDS instances. */ > + for (i = 2; env[i] != NULL; ++i) { > + if (strncmp (env[i], "LISTEN_PID=", 11) == 0 || > + strncmp (env[i], "LISTEN_FDS=", 11) == 0) { > + memmove (&env[i], &env[i+1], > + sizeof (char *) * (nbd_internal_string_list_length (&env[i]))); > + i--; > + } > + } Lots of O(N) traversals of the list, but this probably isn't our hot spot, and so probably not worth optimizing further. > +STATE_MACHINE { > + CONNECT_SA.START: > +#ifdef HAVE_EXECVPE > + size_t len; > + int s; > +...

In v2: ack'ed preliminary patches have been pushed, and I've added a lot of testsuite coverage as well as putting the new API to use in nbdinfo. Eric Blake (2): api: Add nbd_opt_list_meta_context info: List available meta-contexts lib/internal.h | 1 + generator/API.ml | 84 ++++++++-

...e; h->wlen = strlen (h->export_name); @@ -76,7 +76,7 @@ NEWSTYLE.OPT_SET_META_CONTEXT.SEND_EXPORTNAME: switch (send_from_wbuf (h)) { - case -1: SET_NEXT_STATE (%.DEAD); return -1; + case -1: SET_NEXT_STATE (%.DEAD); return 0; case 0: h->sbuf.nrqueries = htobe32 (nbd_internal_string_list_length (h->request_meta_contexts)); @@ -89,7 +89,7 @@ NEWSTYLE.OPT_SET_META_CONTEXT.SEND_NRQUERIES: switch (send_from_wbuf (h)) { - case -1: SET_NEXT_STATE (%.DEAD); return -1; + case -1: SET_NEXT_STATE (%.DEAD); return 0; case 0: h->querynum = 0; SET_NEXT_STATE (%PREPARE_NEXT_Q...

I'm still playing with ideas on how to split rstate from wstate (so that we can send a request without waiting for POLLIN to complete a pending reply), but this is some preliminary refactoring I found useful. I also fixed a couple of bugs while in the area (already pushed). There's a question of whether we want nbd_handle to be nearly 5k, or if we should instead keep it small and add one

I need to think about this patch series a bit more, but it does at least pass the tests. Rich.

This isn't quite finished because not all of the tests or examples have been updated, but it demonstrates an idea: Should we forget about the concept of having multiple connections managed under a single handle? In this patch there is a single ‘struct nbd_handle *’ which manages a single state machine and connection (and therefore no nbd_connection). To connect to a multi-conn server you must

Well, I'm not quite done (I still want to get nbdinfo to work on a single nbd connection for all cases when reading the heads of the file is not required), but I'm happy with patches 1-11, and 12-13 show where I'm headed for getting NBD_OPT_INFO to work. Posting now to see if some of the earlier patches are ready to commit while I continue working on the latter half. Eric Blake (13):

I still need to wire in the use of *_notify functions into nbdkit to prove whether it makes the code any faster or easier to maintain, but at least the added example shows one good use case for the new API. Eric Blake (6): api: Add nbd_aio_in_flight generator: Allow DEAD state actions to run generator: Allow Int64 in callbacks states: Prepare for aio notify callback api: Add new