search for: nbd_handle

...t library in userspace - * Copyright (C) 2013-2019 Red Hat Inc. + * Copyright (C) 2013-2020 Red Hat Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public @@ -48,12 +48,12 @@ nbd_unlocked_clear_debug_callback (struct nbd_handle *h) int nbd_unlocked_set_debug_callback (struct nbd_handle *h, - nbd_debug_callback debug_callback) + nbd_debug_callback *debug_callback) { /* This can't fail at the moment - see implementation above. */ nbd_unlocked_clea...

...rom the nbdkit server (which would in turn synchronously >request the data from VMware) and wait until nbdkit replies before >continuing. > >This converts the inner loop so it issues as many pread requests >asychronously to nbdkit as the server can handle (any extra are queued >in nbd_handle). The server will answer these in parallel and possibly >out of order. > >This results in somewhat better throughput (for me: 13 minutes down to >5 minutes for an "initial" run). Although unfortunately we are still >limited by VDDK's braindead threading model. >---...

As promised in my earlier thread on libnbd completion callback question. Eric Blake (2): generator: Refactor handling of closures in unlocked functions generator: Free closures on failure docs/libnbd.pod | 2 +- generator/C.ml | 48 +++++++++++------ generator/C.mli | 1 + lib/debug.c | 7 +-- lib/opt.c | 31 ++++++-----

I need to think about this patch series a bit more, but it does at least pass the tests. Rich.

...function parameters. This commit defines a new public typedef: typedef int (*nbd_debug_callback) (unsigned valid_flag, void *user_data, const char *context, const char *msg); and then uses the typedef like this: extern int nbd_set_debug_callback (struct nbd_handle *h, nbd_debug_callback debug_callback, void *debug_callback_user_data); (Previously typedefs were available, but they were written by hand and only used internally to the library.) This change necessitates that we uniquely...

..._t %s" n | Int n -> pr "int %s" n | Int64 n -> pr "int64_t %s" n @@ -3305,8 +3306,8 @@ let generate_include_libnbd_h () = pr "#define LIBNBD_HAVE_NBD_GET_ERRNO 1\n"; pr "\n"; pr "extern int nbd_add_close_callback (struct nbd_handle *h,\n"; - pr " void *user_data,\n"; - pr " nbd_close_callback cb);\n"; + pr " nbd_close_callback cb,\n"; + pr " void *u...

...\n" (permitted_state_text permitted_states); pr " return %s;\n" errcode; pr " }\n"; diff --git a/lib/connect.c b/lib/connect.c index 50ec58a..63d2234 100644 --- a/lib/connect.c +++ b/lib/connect.c @@ -38,16 +38,16 @@ static int error_unless_ready (struct nbd_handle *h) { - if (nbd_unlocked_aio_is_ready (h)) + if (nbd_internal_is_state_ready (h->state)) return 0; /* Why did it fail? */ - if (nbd_unlocked_aio_is_closed (h)) { + if (nbd_internal_is_state_closed (h->state)) { set_error (0, "connection is closed"); return -...

...le.am @@ -42,6 +42,7 @@ libnbd_la_SOURCES = \ flags.c \ handle.c \ internal.h \ + is-state.c \ nbd-protocol.h \ poll.c \ protocol.c \ diff --git a/lib/aio.c b/lib/aio.c index a129af2..38e0318 100644 --- a/lib/aio.c +++ b/lib/aio.c @@ -48,84 +48,6 @@ nbd_unlocked_aio_notify_write (struct nbd_handle *h) return nbd_internal_run (h, notify_write); } -/* NB: is_locked = false, may_set_error = false. */ -int -nbd_unlocked_aio_is_created (struct nbd_handle *h) -{ - return h->state == STATE_START; -} - -static int -is_connecting_group (enum state_group group) -{ - switch (group) { - case...

This is based on top of: https://github.com/nertpinx/v2v-conversion-host/commit/0bb2efdcacd975a2cae7380080991ac7fc238d2b The first 4 patches are fairly uncontroversial miscellaneous cleanups. Patch 5 is the interesting one. (Note it doesn't quite work yet, so it's for discussion only.) Patch 5 converts the inner loop to use asynchronous libnbd calls. performance improves quite a bit for

...st for each data block from the nbdkit server (which would in turn synchronously request the data from VMware) and wait until nbdkit replies before continuing. This converts the inner loop so it issues as many pread requests asychronously to nbdkit as the server can handle (any extra are queued in nbd_handle). The server will answer these in parallel and possibly out of order. This results in somewhat better throughput (for me: 13 minutes down to 5 minutes for an "initial" run). Although unfortunately we are still limited by VDDK's braindead threading model. --- wrapper/disk_sync.py |...

...ould in turn synchronously > >request the data from VMware) and wait until nbdkit replies before > >continuing. > > > >This converts the inner loop so it issues as many pread requests > >asychronously to nbdkit as the server can handle (any extra are queued > >in nbd_handle). The server will answer these in parallel and possibly > >out of order. > > > >This results in somewhat better throughput (for me: 13 minutes down to > >5 minutes for an "initial" run). Although unfortunately we are still > >limited by VDDK's braindead...

This works. I'm in the middle of testing whether there is any noticable benefit. Rich.

** INCOMPLETE ** This is the generator change as discussed on the list already. The Python and OCaml bindings are not yet done. It passes all [C only] tests and valgrind. Note that nbd_add_close_callback is inconsistent with other closure types because it passes the user_data parameter after the function. (This is not caused by the current patch, it was already inconsistent). We decided that

...pr " if (h->public_state != get_next_state (h))\n"; pr " h->public_state = get_next_state (h);\n"; diff --git a/lib/debug.c b/lib/debug.c index 1b503d9..b598ad3 100644 --- a/lib/debug.c +++ b/lib/debug.c @@ -54,6 +54,7 @@ nbd_unlocked_set_debug_callback (struct nbd_handle *h, nbd_unlocked_clear_debug_callback (h); h->debug_callback = *debug_callback; + SET_CALLBACK_TO_NULL (*debug_callback); return 0; } diff --git a/lib/opt.c b/lib/opt.c index 003ecf8..6ea8326 100644 --- a/lib/opt.c +++ b/lib/opt.c @@ -156,6 +156,7 @@ nbd_unlocked_opt_list (struct nb...

...ut diff --git a/info/nbdinfo.c b/info/nbdinfo.c index 49c97be..1aca548 100644 --- a/info/nbdinfo.c +++ b/info/nbdinfo.c @@ -37,7 +37,7 @@ static bool probe_content, content_flag, no_content_flag; static bool json_output = false; static bool size_only = false; -static void list_one_export (struct nbd_handle *nbd); +static void list_one_export (struct nbd_handle *nbd, const char *desc); static void list_all_exports (struct nbd_handle *nbd1, const char *uri); static void print_json_string (const char *); static char *get_content (struct nbd_handle *, int64_t size); @@ -243,7 +243,8 @@ main (int argc,...

...s changed, 60 insertions(+), 27 deletions(-) diff --git a/lib/internal.h b/lib/internal.h index 2a5147f..cde5dcd 100644 --- a/lib/internal.h +++ b/lib/internal.h @@ -432,7 +432,8 @@ extern const char *nbd_internal_name_of_nbd_cmd (uint16_t type); extern int64_t nbd_internal_command_common (struct nbd_handle *h, uint16_t flags, uint16_t type, uint64_t offset, uint64_t count, - void *data, struct command_cb *cb); + int count_err,...

...(h->state)\n"; + pr " switch (get_state (h))\n"; pr " {\n"; List.iter ( fun ({ parsed = { state_enum; events } }) -> @@ -2542,7 +2542,7 @@ let generate_lib_states_c () = pr "const char *\n"; pr "nbd_unlocked_connection_state (struct nbd_handle *h)\n"; pr "{\n"; - pr " switch (h->state)\n"; + pr " switch (get_state (h))\n"; pr " {\n"; List.iter ( fun ({ comment; parsed = { display_name; state_enum } }) -> @@ -2841,7 +2841,7 @@ let generate_lib_api_c () = pr &quot...

The API changes from: int nbd_add_close_callback (struct nbd_handle *h, nbd_close_callback cb, void *user_data); to: int nbd_add_close_callback (struct nbd_handle *h, void *user_data, nbd_close_callback cb); The second way is consistent with...

...2 100644 --- a/lib/internal.h +++ b/lib/internal.h @@ -72,6 +72,15 @@ struct export { char *description; }; +struct command_cb { + union { + nbd_extent_callback extent; + nbd_chunk_callback chunk; + nbd_list_callback list; + } fn; + nbd_completion_callback completion; +}; + struct nbd_handle { /* Unique name assigned to this handle for debug messages * (to avoid having to print actual pointers). @@ -102,10 +111,7 @@ struct nbd_handle { /* Option negotiation mode. */ bool opt_mode; uint8_t current_opt; - - /* Results of nbd_opt_list. */ - size_t nr_exports; - struct e...

An incremental improvement on top of listing exports. I still think it's worth experimenting with revisiting how our API for list mode should actually work [1] (so that we can reuse a single connection for both grabbing the list and finally using NBD_OPT_GO), but this change was easier to whip together while still thinking about that. [1]