search for: nbd_cmd_flag_fast_zero

...a zeroed buffer. Add a protocol flag and corresponding transmission advertisement flag to make it easier for clients to inform the server of their intent. If the server advertises NBD_FLAG_SEND_FAST_ZERO, then it promises two things: to perform a fallback to write when the client does not request NBD_CMD_FLAG_FAST_ZERO (so that the client benefits from the lower network overhead); and to fail quickly with ENOTSUP, preferably without modifying the export, if the client requested the flag but the server cannot write zeroes more efficiently than a normal write (so that the client is not penalized with the time of wr...

...dd a protocol flag and corresponding transmission advertisement flag > to make it easier for clients to inform the server of their intent. If > the server advertises NBD_FLAG_SEND_FAST_ZERO, then it promises two > things: to perform a fallback to write when the client does not > request NBD_CMD_FLAG_FAST_ZERO (so that the client benefits from the > lower network overhead); and to fail quickly with ENOTSUP if the > client requested the flag but the server cannot write zeroes more > efficiently than a normal write (so that the client is not penalized > with the time of writing data areas of th...

...perform the fallback to send NBD_CMD_WRITE with a >> zeroed buffer. > > How are you going to finally use it in qemu-img convert? It's already in use there (in fact, the cover letter shows actual timing examples of how qemu-img's use of BDRV_REQ_NO_FALLBACK, which translates to NBD_CMD_FLAG_FAST_ZERO, observably affects timing). > Ok, we have a loop > of sending write-zero requests. And on first ENOTSUP we'll assume that there > is no benefit to continue? But what if actually server returns ENOTSUP only > once when we have 1000 iterations? Seems we should still do zeroing if we...

On Fri, Aug 23, 2019 at 09:34:26AM -0500, Eric Blake wrote: > +- bit 4, `NBD_CMD_FLAG_FAST_ZERO`; valid during > + `NBD_CMD_WRITE_ZEROES`. If set, but the server cannot perform the > + write zeroes any faster than it would for an equivalent > + `NBD_CMD_WRITE`, One way of fulfilling the letter of this requirement but not its spirit could be to have background writes; that is, the...

...we've managed to get a slightly better error message across than normal. I don't see that as a real show-stopper. But if it bothers you, note that in nbdkit, I actually coded things up to refuse to send NBD_EOVERFLOW unless NBD_CMD_FLAG_DF was set, and to refuse to send NBD_ENOTSUP unless NBD_CMD_FLAG_FAST_ZERO was set. I could copy that behavior here, if we want qemu to be that much stricter as a server. (Note to self: also check if, when using structured replies to send error messages, in addition to the code, if the string contains strerror(errno) from BEFORE the mapping, rather than after we've l...

...a zeroed buffer. Add a protocol flag and corresponding transmission advertisement flag to make it easier for clients to inform the server of their intent. If the server advertises NBD_FLAG_SEND_FAST_ZERO, then it promises two things: to perform a fallback to write when the client does not request NBD_CMD_FLAG_FAST_ZERO (so that the client benefits from the lower network overhead); and to fail quickly with ENOTSUP if the client requested the flag but the server cannot write zeroes more efficiently than a normal write (so that the client is not penalized with the time of writing data areas of the disk twice). Note...

...; > > > > I will not push this without both: > > - a positive review (for example, we may decide that burning another > > NBD_FLAG_* is undesirable, and that we should instead have some sort > > of NBD_OPT_ handshake for determining when the server supports > > NBD_CMD_FLAG_FAST_ZERO) >From an implementation point of view I prefer simple flags over having to implement a brand new option. We can always work out how to extend the flags field if we run out of flags. For example, by implementing NBD_OPT_INFO2 with a much bigger flags field. > > - a reference client and...

...operation, so less important to hand to the kernel during transmission phase, especially if the kernel can ever learn to utilize NBD_CMD_BLOCK_STATUS) is indeed different from knowing if probing for fast zeroing is supported (where it is easy to conceive of the kernel needing to know if it can send NBD_CMD_FLAG_FAST_ZERO). So we may still want to use NBD_OPT_* to get the initial zero extension, but NBD_FLAG to advertise the fast zero extension. On the other hand, it's also worth thinking about which extensions are easy for servers to implement - NBD_FLAG_INIT_ZEROES and NBD_FLAG_SEND_FAST_ZERO are orthogonal...

On 8/23/19 1:48 PM, Wouter Verhelst wrote: > On Fri, Aug 23, 2019 at 09:34:26AM -0500, Eric Blake wrote: >> +- bit 4, `NBD_CMD_FLAG_FAST_ZERO`; valid during >> + `NBD_CMD_WRITE_ZEROES`. If set, but the server cannot perform the >> + write zeroes any faster than it would for an equivalent >> + `NBD_CMD_WRITE`, > > One way of fulfilling the letter of this requirement but not its spirit > could be to have back...

So I had a think about this. Isn't this easier/better solved by lifting the 32 bit restriction on the size of certain non-data requests (ie. NBD_CMD_BLOCK_STATUS, NBD_CMD_TRIM, NBD_CMD_WRITE_ZEROES). The client can then query the disk efficiently to see if it starts as zeroes, and can decide on the basis of that whether it needs to "infill" zeroes as it goes along, or can ignore

...35db07b3..76c50bb8 100644 --- a/docs/nbdkit-protocol.pod +++ b/docs/nbdkit-protocol.pod @@ -173,6 +173,17 @@ This protocol extension allows a client to inform the server about intent to access a portion of the export, to allow the server an opportunity to cache things appropriately. +=item C<NBD_CMD_FLAG_FAST_ZERO> + +Supported in nbdkit E<ge> 1.13.9. + +This protocol extension allows a server to advertise that it can rank +all zero requests as fast or slow, at which point the client can make +fast zero requests which fail immediately with C<ENOTSUP> if the +request is no faster than a counter...

This is a cover letter to a series of patches being proposed in tandem to four different projects: - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag - qemu: Implement the flag for both clients and server - libnbd: Implement the flag for clients - nbdkit: Implement the flag for servers, including the nbd passthrough client If you want to test the patches together, I've pushed a 'fast-zero' branch to each of: https://rep...

...200, Wouter Verhelst wrote: > On Mon, Nov 14, 2022 at 04:46:54PM -0600, Eric Blake wrote: > > #### Simple reply message > > > > @@ -1232,6 +1235,19 @@ The field has the following format: > > will be faster than a regular write). Clients MUST NOT set the > > `NBD_CMD_FLAG_FAST_ZERO` request flag unless this transmission flag > > is set. > > +- bit 12, `NBD_FLAG_BLOCK_STATUS_PAYLOAD`: Indicates that the server > > + understands the use of the `NBD_CMD_FLAG_PAYLOAD_LEN` flag to > > + `NBD_CMD_BLOCK_STATUS` to allow the client to request that the &gt...

On Mon, Apr 22, 2019 at 07:50:22PM -0500, Eric Blake wrote: > Previously, we were squashing EOVERFLOW into EINVAL; continue to do so > at points in the protocol where the client may not be expecting > EOVERFLOW. > > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > server/protocol.c | 16 ++++++++++------ > 1 file changed, 10 insertions(+), 6 deletions(-) >

...So the question at hand is whether I should patch the NBD spec to > recommend that a server SHOULD NOT send EOVERFLOW except in response to > NBD_CMD_READ when the NBD_CMD_FLAG_DF bit is set (similar to my proposed > wording that a server SHOULD NOT send ENOTSUP except in response to > NBD_CMD_FLAG_FAST_ZERO). I think we can say that, but we should definitely also say that a client should treat unknown errors in a particular way. Possibly "abort the connection and give up", but something. -- To the thief who stole my anti-depressants: I hope you're happy -- seen somewhere on the Int...

...HOLE (1<<1) -#define NBD_CMD_FLAG_DF (1<<2) -#define NBD_CMD_FLAG_REQ_ONE (1<<3) -*/ +#define NBD_CMD_FLAG_FUA (1<<0) +#define NBD_CMD_FLAG_NO_HOLE (1<<1) +#define NBD_CMD_FLAG_DF (1<<2) +#define NBD_CMD_FLAG_REQ_ONE (1<<3) +#define NBD_CMD_FLAG_FAST_ZERO (1<<4) /* NBD error codes. */ #define NBD_SUCCESS 0 -- 2.23.0

See the cross-post cover letter for details: https://www.redhat.com/archives/libguestfs/2019-August/msg00322.html Eric Blake (1): protocol: Add NBD_CMD_FLAG_FAST_ZERO doc/proto.md | 50 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 1 deletion(-) -- 2.21.0

...redhat.com/archives/libguestfs/2019-August/msg00322.html Based-on: https://lists.gnu.org/archive/html/qemu-devel/2019-08/msg04805.html [Andrey Shinkevich block: workaround for unaligned byte range in fallocate()] Eric Blake (5): nbd: Improve per-export flag handling in server nbd: Prepare for NBD_CMD_FLAG_FAST_ZERO nbd: Implement client use of NBD FAST_ZERO nbd: Implement server use of NBD FAST_ZERO nbd: Tolerate more errors to structured reply request docs/interop/nbd.txt | 3 +- include/block/nbd.h | 6 +++- block/nbd.c | 7 +++++ blockdev-nbd.c | 3 +- nbd/client.c | 39...

..._FLAG_DF (1<<2) > -#define NBD_CMD_FLAG_REQ_ONE (1<<3) > -*/ > +#define NBD_CMD_FLAG_FUA (1<<0) > +#define NBD_CMD_FLAG_NO_HOLE (1<<1) > +#define NBD_CMD_FLAG_DF (1<<2) > +#define NBD_CMD_FLAG_REQ_ONE (1<<3) > +#define NBD_CMD_FLAG_FAST_ZERO (1<<4) Another case where having compile-time assertions that our generator matches protocol constants is probably worthwhile. ACK to the plan for synchronizing. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org

...ire back to EINVAL. So the question at hand is whether I should patch the NBD spec to recommend that a server SHOULD NOT send EOVERFLOW except in response to NBD_CMD_READ when the NBD_CMD_FLAG_DF bit is set (similar to my proposed wording that a server SHOULD NOT send ENOTSUP except in response to NBD_CMD_FLAG_FAST_ZERO). -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org