search for: nr_cycles

...mbprint=aa:bb:cc:... \ > vm=moref=3 \ > file='[datastore1] Fedora 28/Fedora 28.vmdk' \ > --run 'time /var/tmp/threaded-reads $unixsocket' > > Source for threaded-reads is attached. > > (1) Existing nbdkit VDDK plugin. > > NR_MULTI_CONN = 1 > NR_CYCLES = 10000 > > Note this is making 10,000 pread requests. > > real 1m26.103s > user 0m0.283s > sys 0m0.571s > > (2) VDDK plugin patched to support SERIALIZE_REQUESTS. > > NR_MULTI_CONN = 1 > NR_CYCLES = 10000 > > Note this is making 10,000 pread request...

...rd='***' \ server='***' thumbprint=aa:bb:cc:... \ vm=moref=3 \ file='[datastore1] Fedora 28/Fedora 28.vmdk' \ --run 'time /var/tmp/threaded-reads $unixsocket' Source for threaded-reads is attached. (1) Existing nbdkit VDDK plugin. NR_MULTI_CONN = 1 NR_CYCLES = 10000 Note this is making 10,000 pread requests. real 1m26.103s user 0m0.283s sys 0m0.571s (2) VDDK plugin patched to support SERIALIZE_REQUESTS. NR_MULTI_CONN = 1 NR_CYCLES = 10000 Note this is making 10,000 pread requests. real 1m26.755s user 0m0.230s sys 0m0.539s (3) VDDK plugin same as...

Nir, BTW what are you using for performance testing? As far as I can tell it's not possible to make qemu-img convert use multi-conn when connecting to the source (which is going to be a problem if we want to use this stuff in virt-v2v). Instead I've hacked up a copy of this program from libnbd: https://github.com/libguestfs/libnbd/blob/master/examples/threaded-reads-and-writes.c so

...can + * have up to 16 requests in flight. + * + * Some servers do not support multiple requests in flight and may + * deadlock or even crash if this is larger than 1, but common NBD + * servers should be OK. + */ +#define MAX_IN_FLIGHT 16 + +/* Number of commands we issue (per thread). */ +#define NR_CYCLES 1000000 + +/* Reader thread. */ +struct reader_status { + size_t i; /* Thread index, 0 .. NR_MULTI_CONN-1 */ + int argc; /* Command line parameters. */ + char **argv; + int status; /* Return status. */ + unsigned requests; /*...

...s in flight may + * be up to NR_MULTI_CONN * MAX_IN_FLIGHT). See libnbd(3) section + * "Issuing multiple in-flight requests". + */ +#define MAX_IN_FLIGHT 64 + +/* The size of reads and writes. */ +#define BUFFER_SIZE (1024*1024) + +/* Number of commands we issue (per thread). */ +#define NR_CYCLES 1000000 + +/* Reader thread. */ +struct thread_status { + size_t i; /* Thread index, 0 .. NR_MULTI_CONN-1 */ + int argc; /* Command line parameters. */ + char **argv; + int status; /* Return status. */ + unsigned requests; /*...

This works, but there's no time saving and I'm still investigating whether it does what I think it does. Nevertheless I thought I would post it because it (probably) implements the idea I had last night outlined in: https://www.redhat.com/archives/libguestfs/2019-June/msg00010.html The meat of the change is patch 4. Patch 5 is an example which I would probably fold into patch 4 for

...to NR_MULTI_CONN * MAX_IN_FLIGHT). See libnbd(3) section + * "Issuing multiple in-flight requests". */ -#define MAX_IN_FLIGHT 16 +#define MAX_IN_FLIGHT 64 /* The size of reads and writes. */ #define BUFFER_SIZE (1024*1024) /* Number of commands we issue (per thread). */ -#define NR_CYCLES 100000 +#define NR_CYCLES 1000000 struct thread_status { size_t i; /* Thread index, 0 .. NR_MULTI_CONN-1 */ @@ -192,7 +188,6 @@ start_thread (void *arg) uint64_t handles[MAX_IN_FLIGHT]; size_t in_flight; /* counts number of requests in flight */ int dir, r,...

v1: https://www.redhat.com/archives/libguestfs/2019-June/msg00014.html I pushed a few bits which are uncontroversial. The main changes since v1 are: An extra patch removes the want_to_send / check for nbd_aio_is_ready in examples/threaded-reads-and-writes.c. This logic was wrong since commit 6af72b87 as was pointed out by Eric in his review. Comments and structure of

This isn't quite finished because not all of the tests or examples have been updated, but it demonstrates an idea: Should we forget about the concept of having multiple connections managed under a single handle? In this patch there is a single ‘struct nbd_handle *’ which manages a single state machine and connection (and therefore no nbd_connection). To connect to a multi-conn server you must