similar to: [nbdkit PATCH] protocol: Trivially implement NBD_CMD_FLAG_DF

On Thu, Mar 28, 2019 at 10:44:42PM -0500, Eric Blake wrote: > The DF flag is only available to clients that negotiated structured > replies, and even then, the spec does not require that it be > implemented. However, since our current implementation can't fragment > NBD_CMD_READ replies, we trivially implement the flag (by ignoring > it); we don't even have to pass it on to

On Fri, Mar 29, 2019 at 12:43:33PM +0000, Richard W.M. Jones wrote: >On Thu, Mar 28, 2019 at 10:44:42PM -0500, Eric Blake wrote: >> The DF flag is only available to clients that negotiated structured >> replies, and even then, the spec does not require that it be >> implemented. However, since our current implementation can't fragment >> NBD_CMD_READ replies, we

Qemu was able to demonstrate that knowing whether a zero operation is fast is useful when copying from one image to another: there is a choice between bulk pre-zeroing and then revisiting the data sections (fewer transactions, but depends on the zeroing to be fast), vs. visiting every portion of the disk only once (more transactions, but no time lost to duplicated I/O due to slow zeroes). As

Eventually nbdkit will install this as a public header file and we will be able to consume it from $includedir. In the mean time synchronize with the copy in nbdkit. --- lib/nbd-protocol.h | 57 +++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 29 deletions(-) diff --git a/lib/nbd-protocol.h b/lib/nbd-protocol.h index 699aa22..7df411a 100644 ---

This commit implements the core NBD protocol for the "base:allocation" Block Status replies. --- server/internal.h | 7 + server/protocol.h | 17 +- server/protocol-handshake-newstyle.c | 79 ++++++++- server/protocol.c | 248 +++++++++++++++++++++++++-- 4 files changed, 335 insertions(+), 16 deletions(-) diff --git

This is a cover letter to a series of patches being proposed in tandem to four different projects: - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag - qemu: Implement the flag for both clients and server - libnbd: Implement the flag for clients - nbdkit: Implement the flag for servers, including the nbd passthrough client If you want to test the patches together, I've pushed a

This commit implements the core NBD protocol for the "base:allocation" Block Status replies. --- server/internal.h | 7 ++ server/protocol.h | 15 +++ server/protocol-handshake-newstyle.c | 81 ++++++++++++++- server/protocol.c | 141 ++++++++++++++++++++++++--- 4 files changed, 229 insertions(+), 15 deletions(-) diff --git

This is about the simplest implementation of NBD Structured Replies (SR) that we can do right now. It accepts NBD_OPT_STRUCTURED_REPLIES when negotiated by the client, but only sends back the simplest possible SR when required to by NBD_CMD_READ. The rest of the time it will send back simple replies as before. We do not modify the plugin API so plugins are unable to send complex SRs. Also we

v2 was here: https://lists.debian.org/nbd/2022/11/msg00030.html The bulk of the changes since then are: - forbid NBD_OPT_EXPORT_NAME once extended headers are negotiated (Wouter) - consistently use 'maximum payload', rather than a haphazard mix of 'maximum block payload' (Vladimir) At this point, I want to make sure we are happy with the spec before re-posting patches for

Similar to the previous patch in caching size, we want to avoid calling into the plugin more than once per connection on any of the flag determination callbacks. The following script demonstrates the speedup, where we avoid repeated calls into a slow can_fua. Pre-patch: $ cat script case "$1" in get_size) echo 1m;; can_fua) sleep 1; echo native;; can_write | can_zero |

Add a new negotiation feature where the client and server agree to use larger packet headers on every packet sent during transmission phase. This has two purposes: first, it makes it possible to perform operations like trim, write zeroes, and block status on more than 2^32 bytes in a single command. For NBD_CMD_READ, replies are still implicitly capped by the maximum block payload limits

Use a simple bash script to generate the protostrings.c functions. Remove the extern decls from the nbd-protocol.h file which were used previously in the generation of this file. They have been moved to a new internal header called "protostrings.h". --- common/protocol/Makefile.am | 8 ++- ...tostrings.sed => generate-protostrings.sh} | 56 +++++++++++--------

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

On 9/24/19 4:07 PM, Richard W.M. Jones wrote: > Eventually nbdkit will install this as a public header file and we > will be able to consume it from $includedir. In the mean time > synchronize with the copy in nbdkit. > --- > lib/nbd-protocol.h | 57 +++++++++++++++++++++++----------------------- > 1 file changed, 28 insertions(+), 29 deletions(-) > > /* Newstyle

We should have only one NBD protocol file. Let's make nbdkit's version the canonical one, and use it in libnbd. Rich.

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

I want to post this but mainly for discussion and early review. It's not safe for these patches to all go upstream yet (because not all filters have been checked/adjusted), but if any patches were to go upstream then probably 1 & 2 only are safe. File, VDDK, memory and data plugins all work, although I have only done minimal testing on them. The current tests, such as they are, all

I'm hoping to implement .extents for the nbd plugin; this is a prerequisite. I'm not sure about patch 3 - if we like it, I'll squash it to 2, if we don't, I think we are okay just dropping it. I'm also wondering if we have to worry about malicious plugins that don't populate the entire .pread buffer in an effort to get nbdkit to expose portions of the heap; my patch 7 loses

Similar to the recent --mask-handshake command line added to nbdkit to test client fallbacks to crippled servers, it can be worth testing server fallbacks to crippled clients. And just as we have exposed whether the client will request structured replies, we can also expose whether the client will understand various handshake flags from the NBD protocol. Of course, we default to supporting all

The code handling the NBD protocol was located in the same file as the code handling connections, for not really any reason except historical. This is quite a large code movement which splits out the protocol code into four new files: server/protocol-handshake.c initial handshake server/protocol-handshake-newstyle.c " " (newstyle)