search for: can_flush

v1 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00099.html v2: - Just fixes the two problems noted in the review of the previous version. Rich.

We want some testsuite coverage of handling non-1 .can_flush values. The only in-tree plugin with this property is nbd-standalone, but it doesn't get frequently tested, not to mention that the next patch will change it to work with older nbdkit. Signed-off-by: Eric Blake <eblake@redhat.com> --- work in progress: currently compiles but fails durin...

https://aur.archlinux.org/packages/nbdkit/#comment-733981 reports that tests-parallel-nbd.sh hangs. Looking at the log file from the test reveals: nbdkit: backend.c:523: backend_flush: Assertion `h->can_flush == 1' failed. I've attached the full log. If you want to see other logs from the run (probably not relevant) then download https://svenne.dk/nbdkit-20200315/nbdkit-logs-20200315.tar.gz I believe the assertion comes from calling flush when can_flush previously returned false, but also I d...

...n.py index 7c5084efd..419008517 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -165,63 +165,60 @@ def open(readonly): context = context ) + # The first request is to fetch the features of the server. + needs_auth = not params['rhv_direct'] + can_flush = False + can_trim = False + can_zero = False + + http.putrequest("OPTIONS", destination_url.path) + http.putheader("Authorization", transfer.signed_ticket) + http.endheaders() + + r = http.getresponse() + if r.status == 200: + # New imageio never nee...

...n.py index 7c5084efd..5be426897 100644 --- a/v2v/rhv-upload-plugin.py +++ b/v2v/rhv-upload-plugin.py @@ -165,63 +165,60 @@ def open(readonly): context = context ) + # The first request is to fetch the features of the server. + needs_auth = not params['rhv_direct'] + can_flush = False + can_trim = False + can_zero = False + + http.putrequest("OPTIONS", destination_url.path) + http.putheader("Authorization", transfer.signed_ticket) + http.endheaders() + + r = http.getresponse() + if r.status == 200: + # New imageio never nee...

...garbage collected. > def get_size(h): > # return the size of the disk > > -=item C<can_write> > +=item C<is_rotational> > > (Optional) > > - def can_write(h): > + def is_rotational(h): > # return a boolean > > -=item C<can_flush> > +=item C<can_write> > > (Optional) > > - def can_flush(h): > + def can_write(h): > # return a boolean > > -=item C<is_rotational> > +=item C<can_flush> > > (Optional) > > - def is_rotational(h): > + def can...

These two patches add support for using a Unix domain socket to directly access imageio in the case where imageio is running on the conversion host (usually that means virt-v2v is running on the RHV node and something else -- eg. CFME scripts -- arranges that the RHV node is the same one running imageio). Conversions in the normal case are not affected - they happen over TCP as usual. This was

...s/python/nbdkit-python-plugin.pod @@ -184,25 +184,25 @@ contents will be garbage collected. def get_size(h): # return the size of the disk -=item C<can_write> +=item C<is_rotational> (Optional) - def can_write(h): + def is_rotational(h): # return a boolean -=item C<can_flush> +=item C<can_write> (Optional) - def can_flush(h): + def can_write(h): # return a boolean -=item C<is_rotational> +=item C<can_flush> (Optional) - def is_rotational(h): + def can_flush(h): # return a boolean =item C<can_trim> diff --git a/plugins/p...

...o ensure we don't regress again. Note that nbdkit does not yet cache .thread_model; that will be addressed in the next patch. Furthermore, we end up duplicating the caching that nbdkit itself does, because it would be a layering violation for us to have enough information to call into backend_can_flush(). Fixes: 05c5eed6f2 Fixes: 8490694d08 Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/sh/methods.c | 102 +++++++++++++++++++++++++++---------------- tests/test-eflags.sh | 44 +++++++++++++++++-- 2 files changed, 105 insertions(+), 41 deletions(-) diff --git a/plugins/sh/metho...

...--- a/v2v/rhv-upload-plugin.py > +++ b/v2v/rhv-upload-plugin.py > @@ -165,63 +165,60 @@ def open(readonly): > context = context > ) > > + # The first request is to fetch the features of the server. > + needs_auth = not params['rhv_direct'] > + can_flush = False > + can_trim = False > + can_zero = False > + > + http.putrequest("OPTIONS", destination_url.path) > + http.putheader("Authorization", transfer.signed_ticket) + http.endheaders() > + > + r = http.getresponse() > We should read...

...again. > > Note that nbdkit does not yet cache .thread_model; that will be > addressed in the next patch. Furthermore, we end up duplicating the > caching that nbdkit itself does, because it would be a layering > violation for us to have enough information to call into > backend_can_flush(). > > Fixes: 05c5eed6f2 > Fixes: 8490694d08 > Signed-off-by: Eric Blake <eblake@redhat.com> > --- > @@ -457,8 +473,14 @@ int > sh_can_flush (void *handle) > { > const char *method = "can_flush"; > - const char *script = get_script (method);...

...const void *buf, uint32_t count, uint64_t offset, + uint32_t flags); This intercepts the plugin C<.pwrite> method and can be used to modify data written by the plugin. +At this time, flags may include C<NBDKIT_FLAG_FUA> on input based on +the result of C<.can_flush>. In turn, the filter may only pass +C<NBDKIT_FLAG_FUA> on to C<next_ops->pwrite> if C<next_ops->can_flush> +returned true. + +This function will not be called if C<.can_write> returned false; in +turn, the filter should not call C<next_ops->pwrite> if C&l...

On Thu, Mar 19, 2020 at 07:09:32AM -0500, Eric Blake wrote: > Thinking about this more, maybe the real problem is that all > language bindings that have to wrap scripts (OCaml and Rust are > exceptions as they directly call into the C code; but lua, perl, > python, ruby, tcl, and sh all fit into the category I'm describing) > MUST define .can_FOO at the C level because they

On Fri, Jan 19, 2018 at 10:45:51AM -0600, Eric Blake wrote: > I'm wondering if we're missing documentation here (and/or in the plugins > document) that if .can_write returns true, the plugin must supply a > .pwrite; likewise for .can_trim implying a .trim, and .can_flush > implying a .flush. It's a strange one. For example if a plugin returns .can_pwrite == 1 but doesn't implement .pwrite then the NBD connection will appear to be writable, but any attempt to write will return EROFS. On the other hand (a) plugins don't usually implement can_write b...

...n (struct nbdkit_next_ops *next_ops, + void *nxdata, void *handle); + +This intercepts the plugin C<.export_description> method and can be +used to read or modify the export description that the NBD client +will see. + =head2 C<.can_write> =head2 C<.can_flush> diff --git a/docs/nbdkit-plugin.pod b/docs/nbdkit-plugin.pod index f1476032..62feae47 100644 --- a/docs/nbdkit-plugin.pod +++ b/docs/nbdkit-plugin.pod @@ -825,6 +825,26 @@ to get the size (in bytes) of the block device being exported. The returned size must be E<ge> 0. If there is an er...

...fferent filters, but neither filter name was shown). By printing the filter name in the debug message we improve the output: nbdkit: pattern[1]: debug: truncate: prepare nbdkit: pattern[1]: debug: map: prepare Another example of the improved output is: nbdkit: pattern[1]: debug: truncate: can_flush # filter nbdkit: pattern[1]: debug: map: can_flush # filter nbdkit: pattern[1]: debug: can_flush # plugin --- src/filters.c | 48 ++++++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/src/filters.c b/src/filt...

...n, int sockin, int sockout); diff --git a/server/connections.c b/server/connections.c index 0d1bd74..0a89315 100644 --- a/server/connections.c +++ b/server/connections.c @@ -78,13 +78,13 @@ struct connection { uint32_t cflags; uint64_t exportsize; uint16_t eflags; - int readonly; - int can_flush; - int is_rotational; - int can_trim; - int can_zero; - int can_fua; - int using_tls; + bool readonly; + bool can_flush; + bool is_rotational; + bool can_trim; + bool can_zero; + bool can_fua; + bool using_tls; int sockin, sockout; connection_recv_function recv; @@ -419,7 +419,7...

v2 was here: https://www.redhat.com/archives/libguestfs/2018-June/msg00109.html v3: - Added/fixed all suggestions from Nir in previous review. Q: I wasn't sure if we still need the "UnsupportedError" class so I left it in. Q: Does the Unix socket always have the same name? What happens if there's more than one transfer happening? I tested this both ways, and it worked both

...6_t eflags = NBD_FLAG_HAS_FLAGS; + int fl; + + fl = plugin_can_write (conn); + if (fl == -1) + return -1; + if (readonly || !fl) { + eflags |= NBD_FLAG_READ_ONLY; + conn->readonly = 1; + } + if (!conn->readonly) { + eflags |= NBD_FLAG_SEND_WRITE_ZEROES; + } + + fl = plugin_can_flush (conn); + if (fl == -1) + return -1; + if (fl) { + eflags |= NBD_FLAG_SEND_FLUSH | NBD_FLAG_SEND_FUA; + conn->can_flush = 1; + } + + fl = plugin_is_rotational (conn); + if (fl == -1) + return -1; + if (fl) { + eflags |= NBD_FLAG_ROTATIONAL; + conn->is_rotational = 1; +...

...f open(readonly): transfer = transfers_service.add( types.ImageTransfer( disk = types.Disk(id = disk.id), + host = types.Host(id = host.id) if host else None, inactivity_timeout = 3600, ) ) @@ -170,6 +191,7 @@ def open(readonly): can_flush = False can_trim = False can_zero = False + unix_socket = None http.putrequest("OPTIONS", destination_url.path) http.putheader("Authorization", transfer.signed_ticket) @@ -184,6 +206,7 @@ def open(readonly): can_flush = "flush" in j[...