search for: can_zero

...ient having to send zeroes over the wire). Furthermore, since we only have to honor FUA support on write requests, we do not need to advertise it on a readonly connection. Wire up two new backend callbacks, as well as exposing them to the filter interface (for now, we intentionally do not expose .can_zero to the plugin interface, and later patches will worry about exposing .can_fua as part of updating the plugin interface to support flags). This change breaks filter API; a filter compiled against an earlier nbdkit will not run against this version of nbdkit. We haven't promised ABI stability y...

Originally, I thought that since the plugin always emulates .zero with a fallback to .pwrite, we didn't need to expose the backend's .can_zero to plugins. But there is another consideration, as shown at least in the nbd plugin: a plugin may implement a .zero callback that depends on support from a remote endpoint. If the remote endpoint doesn't support zeroes, then the plugin HAS to fail .zero with EOPNOTSUPP to get the automatic fa...

This is a subset of the last half of the larger 9-patch series. The uncontroversial first half of that series is pushed, but here, I tried to reduce the size of the patches by splitting out some of the more complex changes, so that the rest of the changes remaining in the series are more mechanical. In turn, it forced me to write timing tests, which let me spot another spot where we are wasting

Fairly straightforward, as the code in plugins.c already takes care of necessary .pwrite fallbacks. Simplify similar can_FOO callbacks that were copy-and-pasted while implementing py_can_zero. Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/python/nbdkit-python-plugin.pod | 7 +++++ plugins/python/python.c | 51 +++++++++++++++++++-------------- tests/test.py | 4 +++ 3 files changed, 41 insertions(+), 21 deletions(-) diff -...

...insertions(+) diff --git a/info/nbdinfo.pod b/info/nbdinfo.pod index 9761551..d0d20a9 100644 --- a/info/nbdinfo.pod +++ b/info/nbdinfo.pod @@ -45,6 +45,9 @@ L<https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md>): can_multi_conn: true can_trim: true can_zero: true + block_size_minimum: 1 + block_size_preferred: 4096 + block_size_maximum: 33554432 To display only the size in bytes of the NBD export (useful for scripting) use the I<--size> parameter: @@ -73,6 +76,9 @@ the I<--json> parameter: "can_multi...

v1 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00099.html v2: - Just fixes the two problems noted in the review of the previous version. Rich.

...v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -165,63 +165,60 @@ def open(readonly): context = context ) + # The first request is to fetch the features of the server. + needs_auth = not params['rhv_direct'] + can_flush = False + can_trim = False + can_zero = False + + http.putrequest("OPTIONS", destination_url.path) + http.putheader("Authorization", transfer.signed_ticket) + http.endheaders() + + r = http.getresponse() + if r.status == 200: + # New imageio never needs authentication. + needs_auth = Fal...

...v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -165,63 +165,60 @@ def open(readonly): context = context ) + # The first request is to fetch the features of the server. + needs_auth = not params['rhv_direct'] + can_flush = False + can_trim = False + can_zero = False + + http.putrequest("OPTIONS", destination_url.path) + http.putheader("Authorization", transfer.signed_ticket) + http.endheaders() + + r = http.getresponse() + if r.status == 200: + # New imageio never needs authentication. + needs_auth = Fal...

...n more than once per connection on any of the flag determination callbacks. The following script demonstrates the speedup, where we avoid repeated calls into a slow can_fua. Pre-patch: $ cat script case "$1" in get_size) echo 1m;; can_fua) sleep 1; echo native;; can_write | can_zero | pwrite | zero) ;; *) exit 2 ;; esac $ /bin/time -f %e ./nbdkit --filter=blocksize sh script maxlen=4k \ --run 'qemu-io -f raw -c "w -z -f 0 16k" $nbd' wrote 16384/16384 bytes at offset 0 16 KiB, 1 ops; 0:00:05.07 (3.157 KiB/sec and 0.1973 ops/sec) 6.14 there are six execu...

These two patches add support for using a Unix domain socket to directly access imageio in the case where imageio is running on the conversion host (usually that means virt-v2v is running on the RHV node and something else -- eg. CFME scripts -- arranges that the RHV node is the same one running imageio). Conversions in the normal case are not affected - they happen over TCP as usual. This was

...ad-plugin.py > @@ -165,63 +165,60 @@ def open(readonly): > context = context > ) > > + # The first request is to fetch the features of the server. > + needs_auth = not params['rhv_direct'] > + can_flush = False > + can_trim = False > + can_zero = False > + > + http.putrequest("OPTIONS", destination_url.path) > + http.putheader("Authorization", transfer.signed_ticket) + http.endheaders() > + > + r = http.getresponse() > We should read the response data here to make sure we consume the ent...

...nc. + * Copyright (C) 2018-2020 Red Hat Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are @@ -193,29 +193,42 @@ sh_preconnect (int readonly) } } +struct sh_handle { + int can_flush; + int can_zero; + char *h; +}; + void * sh_open (int readonly) { const char *method = "open"; const char *script = get_script (method); - char *h = NULL; size_t hlen; const char *args[] = { script, method, readonly ? "true" : "false", nbdkit_export...

...yond the end of an older filter's smaller struct, there are no sizing limits in the other direction to prevent a newer filter to know that it cannot call beyond the end of our smaller nbdkit_next_ops. Eric Blake (3): connections: Don't advertise TRIM on readonly connection filter: Add .can_zero/.can_fua overrides filters: Add nozero filter TODO | 2 +- configure.ac | 3 +- docs/nbdkit-filter.pod | 30 +++++++++++- docs/nbdkit.pod | 1 + filters/Makefile.am | 1...

On Thu, Mar 19, 2020 at 07:09:32AM -0500, Eric Blake wrote: > Thinking about this more, maybe the real problem is that all > language bindings that have to wrap scripts (OCaml and Rust are > exceptions as they directly call into the C code; but lua, perl, > python, ruby, tcl, and sh all fit into the category I'm describing) > MUST define .can_FOO at the C level because they

...se) + +def can_multi_conn (h): + return cfg.get ('can_multi_conn', False) + +def can_write (h): + return cfg.get ('can_write', True) + +def can_flush(h): + return cfg.get ('can_flush', False) + +def can_trim(h): + return cfg.get ('can_trim', False) + +def can_zero(h): + return cfg.get ('can_zero', False) + +def can_fast_zero(h): + return cfg.get ('can_fast_zero', False) + +def can_fua(h): + fua = cfg.get ('can_fua', "none") + if fua == "none": + return nbdkit.FUA_NONE + elif fua == "emula...

...e filter is currently doing into something that nbdkit does from any one backend calling into another with different sizing constraints. However, that is not done in this series. My pending work on fast zeroes was easy enough to rebase on top of this (in fact, I wrote this because I noticed that .can_zero was being called more than necessary in implementing .can_fast_zero). Eric Blake (9): server: Fewer dereferences in filter server: Consolidate common backend tasks into new backend.c server: Create new backend_* functions for central handling server: Rework storage of per-backend handle...

On 3/18/20 8:21 PM, Eric Blake wrote: > In commit c306fa93ab and neighbors (v1.15.1), a concerted effort went > into caching the results of .can_FOO callbacks, with commit messages > demonstrating that a plugin with a slow callback should not have that > delay magnified multiple times. But nothing was added to the > testsuite at the time, and with the sh and eval plugins, we still

...gt; is also used), so returning false merely +serves as a way to avoid complicating the C<.zero> callback to have to +fail with C<ENOTSUP> or C<EOPNOTSUPP> on the connections where it will +never be more efficient than using C<.pwrite> up front. If there is an error, C<.can_zero> should call C<nbdkit_error> with an error message and return C<-1>. -This callback is not required. If omitted, then nbdkit always tries -C<.zero> first if it is present, and gracefully falls back to -C<.pwrite> if C<.zero> was absent or failed with C<ENOTSUP...

When I added .can_FOO caching in 1.16, I missed the case that the sh plugin itself was calling .can_flush twice in some situations (in order to default .can_fua). Then right after, I regressed it to call .can_zero twice (in order to default .can_fast_zero). I also missed that .thread_model could use better caching, because at the time, I did not add testsuite coverage. Fix that now. Eric Blake (2): sh, eval: Cache .can_zero and .can_flush server: Better caching of .thread_model server/internal.h...

...9;can_multi_conn', False) > + > +def can_write (h): > + return cfg.get ('can_write', True) > + > +def can_flush(h): > + return cfg.get ('can_flush', False) > + > +def can_trim(h): > + return cfg.get ('can_trim', False) > + > +def can_zero(h): > + return cfg.get ('can_zero', False) > + > +def can_fast_zero(h): > + return cfg.get ('can_fast_zero', False) > + > +def can_fua(h): > + fua = cfg.get ('can_fua', "none") > + if fua == "none": > + return...