search for: nbd_request_full

...300,27 @@ nbd_request_raw (struct handle *h, uint16_t flags, uint16_t type, } /* Perform the request half of a transaction. On success, return the - non-negative fd for reading the reply; on error return -1. */ -static int + transaction; on error return NULL. */ +static struct transaction * nbd_request_full (struct handle *h, uint16_t flags, uint16_t type, uint64_t offset, uint32_t count, const void *req_buf, void *rep_buf, struct nbdkit_extents *extents) { int err; struct transaction *trans; - int fd; uint64_t cookie; trans = calloc (1, sizeof *tra...

...break; + ptr = &trans->next; + } + if (trans) + *ptr = trans->next; + nbd_unlock (h); + return trans; +} + /* Send a request, return 0 on success or -1 on write failure. */ static int nbd_request_raw (struct handle *h, uint32_t type, uint64_t offset, @@ -260,6 +280,8 @@ nbd_request_full (struct handle *h, uint32_t type, uint64_t offset, { int err; struct transaction *trans; + int fd; + uint64_t cookie; trans = calloc (1, sizeof *trans); if (!trans) { @@ -282,9 +304,14 @@ nbd_request_full (struct handle *h, uint32_t type, uint64_t offset, } trans->next = h-...

Tested via: term1$ qemu-nbd -k $PWD/sock -t -f raw -x foo junk --trace=nbd_\* term2$ ./nbdkit -f -v -e bar nbd socket=$PWD/sock export=foo term3$ qemu-io -t none -f raw nbd://localhost:10809/bar --trace=nbd_\* and checking the traces to see that 'w 0 1' vs. 'w -f 0 1' was able to influence whether the FUA flag showed up at the server in term1. Still to go: figure out how to

Updated based on other changes that have happened in the meantime: - rely more on cleanup.h (throughout) - split structured read for easier review (patch 2 and 3 were combined in v1) - rely on nbdkit not leaking a server's partial answer (patch 3) - add tests (patch 5) - other bug fixes I found while testing it - drop EOVERFLOW patch for now; it will be separate once upstream NBD protocol

I'm hoping to implement .extents for the nbd plugin; this is a prerequisite. I'm not sure about patch 3 - if we like it, I'll squash it to 2, if we don't, I think we are okay just dropping it. I'm also wondering if we have to worry about malicious plugins that don't populate the entire .pread buffer in an effort to get nbdkit to expose portions of the heap; my patch 7 loses

...nding request with type %d and cookie %#" PRIx64, type, + cookie); + return write_full (h->fd, &req, sizeof req); +} + +/* Perform the request half of a transaction. On success, return the + non-negative cookie to match to the reply; on error return -1. */ +static int +nbd_request_full (struct handle *h, uint32_t type, uint64_t offset, + uint32_t count, const void *req_buf, void *rep_buf) +{ + if (h->dead) + return nbd_mark_dead (h); + h->trans.buf = rep_buf; + h->trans.count = rep_buf ? count : 0; + if (++h->trans.cookie > INT_MAX) + h-&...

...kit and --trace='nbd_*' on qemu-io to turn up the debugging messages, to actually watch the interactions). Patch 2 shows my thought process to switching things to fully parallel: patch 3/2 (still unwritten at the moment) would basically change a single transaction into a linked list, where nbd_request_full() adds another transaction to the list, nbd_read_raw() uses the cookie received from the server to look up a transaction to remove from the list, and where breaking the while(1) loop of nbd_reader() cleans up any remaining transactions still on the list. Eric Blake (2): nbd: Add new nbd forwardi...

...tion **ptr; struct transaction *trans; @@ -244,7 +248,7 @@ find_trans_by_cookie (struct handle *h, uint64_t cookie) break; ptr = &trans->next; } - if (trans) + if (trans && remove) *ptr = trans->next; nbd_unlock (h); return trans; @@ -303,6 +307,7 @@ nbd_request_full (struct handle *h, uint16_t flags, uint16_t type, } trans->buf = rep_buf; trans->count = rep_buf ? count : 0; + trans->offset = offset; nbd_lock (h); if (h->dead) { nbd_unlock (h); @@ -315,7 +320,7 @@ nbd_request_full (struct handle *h, uint16_t flags, uint16_t type...

With this, I am finally able to get the nbd plugin to do out-of-order responses to the client. Once this series goes in, we should be ready for Rich to cut a release. Eric Blake (4): nbd: Split reading into separate thread nbd: Protect writes with mutex nbd: Enable parallel handling tests: Test parallel nbd behavior plugins/nbd/nbd.c | 217

...243,34 @@ nbd_request_raw (struct handle *h, uint32_t type, uint64_t offset, } /* Perform the request half of a transaction. On success, return the - non-negative cookie to match to the reply; on error return -1. */ + non-negative fd for reading the reply; on error return -1. */ static int nbd_request_full (struct handle *h, uint32_t type, uint64_t offset, uint32_t count, const void *req_buf, void *rep_buf) { + int err; + if (h->dead) return nbd_mark_dead (h); + if (pipe (h->trans.u.fds)) { + nbdkit_error ("unable to create pipe: %m"); + /* Still in...

..., count, cookie); - r = write_full (h->fd, &req, sizeof req); - if (buf && !r) - r = write_full (h->fd, buf, count); - return r; -} - -/* Perform the request half of a transaction. On success, return the - transaction; on error return NULL. */ -static struct transaction * -nbd_request_full (struct handle *h, uint16_t flags, uint16_t type, - uint64_t offset, uint32_t count, const void *req_buf, - void *rep_buf, struct nbdkit_extents *extents) -{ - int err; - struct transaction *trans; - uint64_t cookie; - - trans = calloc (1, sizeof *trans); - if...

Patch 1 played with an early draft of Rich's Fedora 30 libnbd package: https://bugzilla.redhat.com/show_bug.cgi?id=1713767#c17 Note that comment 21 provides a newer package 0.1.1-1 with a different API; and that libnbd has more unreleased API changes in the pipeline (whether that will be called 0.2 or 0.1.2); so we'll have to tweak things based on what is actually available in distros.

libnbd-0.1.2-1 is now available in Fedora 29/30 updates-testing, although it was not compiled against libxml2 so it lacks uri support (I ended up testing patch 4 with a self-built libnbd). Diffs since v1 - rebase to master, bump from libnbd 0.1 to 0.1.2, add URI support, better timing results Still not done - patch 5 needs associated tests Eric Blake (5): nbd: Check for libnbd nbd:

libnbd-0.1.4-1 is now available in Fedora 29/30 updates testing. Diffs since v2 - rebase to master, bump from libnbd 0.1.2 to 0.1.3+, add tests to TLS usage which flushed out the need to turn relative pathnames into absolute, doc tweaks Now that the testsuite covers TLS and libnbd has been fixed to provide the things I found lacking when developing v2, I'm leaning towards pushing this on

A combination of the work that both Rich and I have been doing lately, where filters use only the new API with flags on every command that the client can send over the wire (we can then add support for more flags in nbdkit without having to add new callbacks, as NBD adds more flags upstream). Eric Blake (4): protocol: Split flags from cmd field in requests backend: Pass flags argument through

When converting from a single transaction to a linked list, I forgot to free the storage for each member of the list. Reported-by: Richard W.M. Jones <rjones at redhat.com> Fixes: 7f5bb9bf13f041ea7702bda557d9dd668bc3423a Signed-off-by: Eric Blake <eblake at redhat.com> --- I'm still not sure why 'make check' passes while 'make check-valgrind' fails for

...EB996: nbd_reader (nbd.c:373) ==18076== by 0x55DC55A: start_thread (in /usr/lib64/libpthread-2.26.9000.so) ==18076== by 0x58E85AE: clone (in /usr/lib64/libc-2.26.9000.so) ==18076== Block was alloc'd at ==18076== at 0x4C2FA1E: calloc (vg_replace_malloc.c:711) ==18076== by 0x77EBBAF: nbd_request_full (nbd.c:264) ==18076== by 0x77EBD98: nbd_pread (nbd.c:602) ==18076== by 0x405986: handle_request (connections.c:884) ==18076== by 0x405986: recv_request_send_reply (connections.c:1061) ==18076== by 0x405AE7: connection_worker (connections.c:200) ==18076== by 0x55DC55A: start_thread (i...

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]