search for: nbdkit_flag_fast_zero

...t32_t count, uint64_t offset, uint32_t flags, int r; r = b_conn->b->zero (b_conn->b, b_conn->conn, count, offset, flags, err); - if (r == -1) - assert (*err && *err != ENOTSUP && *err != EOPNOTSUPP); + if (r == -1) { + assert (*err); + if (!(flags & NBDKIT_FLAG_FAST_ZERO)) + assert (*err != ENOTSUP && *err != EOPNOTSUPP); + } return r; } @@ -586,6 +589,15 @@ filter_can_zero (struct backend *b, struct connection *conn) return f->backend.next->can_zero (f->backend.next, conn); } +static int +filter_can_fast_zero (struct backend *b,...

...zero requests. If this +support is not advertised, a client cannot attempt fast zero requests, +and has no way to tell if writing zeroes offers any speedups compared +to using C<.pwrite> (other than compressed network traffic). If +support is advertised, then C<.zero> will have +C<NBDKIT_FLAG_FAST_ZERO> set when the client has requested a fast +zero, in which case the plugin must fail with C<ENOTSUP> or +C<EOPNOTSUPP> up front if the request would not offer any benefits +over C<.pwrite>. Advertising support for fast zero requests does not +require that writing zeroes be fast...

This is a cover letter to a series of patches being proposed in tandem to four different projects: - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag - qemu: Implement the flag for both clients and server - libnbd: Implement the flag for clients - nbdkit: Implement the flag for servers, including the nbd passthrough client If you want to test the patches together, I've pushed a

...dIntConstant (m, "FLAG_MAY_TRIM", NBDKIT_FLAG_MAY_TRIM); > + PyModule_AddIntConstant (m, "FLAG_FUA", NBDKIT_FLAG_FUA); > + PyModule_AddIntConstant (m, "FLAG_REQ_ONE", NBDKIT_FLAG_REQ_ONE); > + PyModule_AddIntConstant (m, "FLAG_FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); > + > + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); > + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); > + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); > + > + PyModule_AddIntConstant...

...Y_TRIM", NBDKIT_FLAG_MAY_TRIM); > > > + PyModule_AddIntConstant (m, "FLAG_FUA", NBDKIT_FLAG_FUA); > > > + PyModule_AddIntConstant (m, "FLAG_REQ_ONE", NBDKIT_FLAG_REQ_ONE); > > > + PyModule_AddIntConstant (m, "FLAG_FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); > > > + > > > + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); > > > + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); > > > + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE);...

...> *err = EROFS; > r = -1; > @@ -312,6 +329,8 @@ retry_zero (struct nbdkit_next_ops *next_ops, void *nxdata, > *err = EROFS; > return -1; > } > + if (! valid_range (next_ops, nxdata, count, offset, true, err)) > + return -1; > if (flags & NBDKIT_FLAG_FAST_ZERO && > next_ops->can_fast_zero (nxdata) != 1) { > *err = EOPNOTSUPP; > @@ -347,6 +366,8 @@ retry_extents (struct nbdkit_next_ops *next_ops, void *nxdata, > size_t i; > > again: > + if (! valid_range (next_ops, nxdata, count, offset, false, err))...

...le plugin yet; there, I probably need to do more testing and/or kernel source code reading to learn whether to mark fallocate() as potentially slow, as well as to definitely mark ioctl(BLKZEROOUT) as definitely slow. That will be a followup patch. Eric Blake (3): server: Add internal support for NBDKIT_FLAG_FAST_ZERO filters: Add .can_fast_zero hook plugins: Add .can_fast_zero hook docs/nbdkit-filter.pod | 27 ++++-- docs/nbdkit-plugin.pod | 74 +++++++++++--- docs/nbdkit-protocol.pod | 11 +++ filters/delay/nbdkit-delay-filter.pod | 15 ++- filters/l...

...if (next_ops->can_trim (nxdata) != 1) { *err = EROFS; r = -1; @@ -312,6 +329,8 @@ retry_zero (struct nbdkit_next_ops *next_ops, void *nxdata, *err = EROFS; return -1; } + if (! valid_range (next_ops, nxdata, count, offset, true, err)) + return -1; if (flags & NBDKIT_FLAG_FAST_ZERO && next_ops->can_fast_zero (nxdata) != 1) { *err = EOPNOTSUPP; @@ -347,6 +366,8 @@ retry_extents (struct nbdkit_next_ops *next_ops, void *nxdata, size_t i; again: + if (! valid_range (next_ops, nxdata, count, offset, false, err)) + return -1; if (next_ops-&g...

...+ PyModule_AddIntConstant (m, "FLAG_MAY_TRIM", NBDKIT_FLAG_MAY_TRIM); + PyModule_AddIntConstant (m, "FLAG_FUA", NBDKIT_FLAG_FUA); + PyModule_AddIntConstant (m, "FLAG_REQ_ONE", NBDKIT_FLAG_REQ_ONE); + PyModule_AddIntConstant (m, "FLAG_FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); + + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); + + PyModule_AddIntConstant (m, "CACHE_NONE",...

..., "FLAG_MAY_TRIM", NBDKIT_FLAG_MAY_TRIM); > > + PyModule_AddIntConstant (m, "FLAG_FUA", NBDKIT_FLAG_FUA); > > + PyModule_AddIntConstant (m, "FLAG_REQ_ONE", NBDKIT_FLAG_REQ_ONE); > > + PyModule_AddIntConstant (m, "FLAG_FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); > > + > > + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); > > + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); > > + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); > > + > &g...

On 3/18/20 8:21 PM, Eric Blake wrote: > In commit c306fa93ab and neighbors (v1.15.1), a concerted effort went > into caching the results of .can_FOO callbacks, with commit messages > demonstrating that a plugin with a slow callback should not have that > delay magnified multiple times. But nothing was added to the > testsuite at the time, and with the sh and eval plugins, we still

...;short write") + +def flush(h, flags): + os.fsync(h['fd']) diff --git a/server/plugins.c b/server/plugins.c index d4364cd2..ce4700a3 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; bool emulate = false; bool need_flush = false; + void *zero_buffer = NULL; + int buffer_size = MIN (MAX_REQUEST_SIZE, count); if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { flags &= ~NBDKIT_FLAG_FUA; @@ -669,19 +671,25 @@ plugin_zero (struct backend *b, void *...

...FUA); int r; @@ -566,7 +566,7 @@ backend_zero (struct backend *b, int *err) { GET_CONN; - struct b_conn_handle *h = &conn->handles[b->i]; + struct handle *h = get_handle (conn, b->i); bool fua = !!(flags & NBDKIT_FLAG_FUA); bool fast = !!(flags & NBDKIT_FLAG_FAST_ZERO); int r; @@ -601,7 +601,7 @@ backend_extents (struct backend *b, struct nbdkit_extents *extents, int *err) { GET_CONN; - struct b_conn_handle *h = &conn->handles[b->i]; + struct handle *h = get_handle (conn, b->i); int r; assert (h->handle &&am...

...;short write") + +def flush(h, flags): + os.fsync(h['fd']) diff --git a/server/plugins.c b/server/plugins.c index d4364cd2..ce4700a3 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; bool emulate = false; bool need_flush = false; + void *zero_buffer = NULL; + int buffer_size = MIN (MAX_REQUEST_SIZE, count); if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { flags &= ~NBDKIT_FLAG_FUA; @@ -669,19 +671,25 @@ plugin_zero (struct backend *b, void *...

...flags): > + os.fsync(h['fd']) > diff --git a/server/plugins.c b/server/plugins.c > index d4364cd2..ce4700a3 100644 > --- a/server/plugins.c > +++ b/server/plugins.c > @@ -631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, > bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; > bool emulate = false; > bool need_flush = false; > + void *zero_buffer = NULL; > + int buffer_size = MIN (MAX_REQUEST_SIZE, count); > > if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { > flags &= ~NBDKIT_FLAG_FUA; > @@ -669,19 +671,25 @...

I think this is my last round of patches for issues I identified with the retry filter. With this in place, it should be safe to interject another filter in between retry and the plugin. Eric Blake (5): retry: Don't call into closed plugin tests: Refactor test-retry-reopen-fail.sh tests: Enhance retry test to cover failed reopen server: Move prepare/finalize/close recursion to

As mentioned in my reviews, I wonder if we should make our python callbacks look a bit more Pythonic by having kwargs added for each new flag that we want to expose. The idea was first floated here: https://www.redhat.com/archives/libguestfs/2018-April/msg00108.html Note that with my proposal, there is no need for a python script to expose a global API_VERSION variable; new flags are added

...amp;err) == -1) + if (backend_cache (top, count, offset, 0, &err) == -1) return err; break; @@ -273,14 +273,14 @@ handle_request (uint16_t cmd, uint16_t flags, uint64_t offset, uint32_t count, f |= NBDKIT_FLAG_FUA; if (flags & NBD_CMD_FLAG_FAST_ZERO) f |= NBDKIT_FLAG_FAST_ZERO; - if (backend_zero (backend, count, offset, f, &err) == -1) + if (backend_zero (top, count, offset, f, &err) == -1) return err; break; case NBD_CMD_BLOCK_STATUS: if (flags & NBD_CMD_FLAG_REQ_ONE) f |= NBDKIT_FLAG_REQ_ONE; - if (backend_extents (bac...

I'm still working on another set of patches to have reopen call .finalize/.prepare (so that another filter can safely appear between retry and the plugin), but for tonight, these are the patches I think are ready to go. Eric Blake (4): retry: Handle can_fua and can_fast_zero changes tests: Test retry with different fua/fast-zero flags server: Close backends if a filter's .open fails

I still don't have .prepare/.finalize working cleanly across reopen, but did find a nasty bug where a botched assertion means we failed to notice reads beyond EOF in both the xz and retry filter. Refactoring backend.c will make .finalize work easier. Eric Blake (5): xz: Avoid reading beyond EOF retry: Check size before transactions tests: Test retry when get_size values change