search for: can_extent

...++++++++++++++++++++++++- 8 files changed, 275 insertions(+), 5 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..ada49dd 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

...Makefile.am | 1 + 10 files changed, 381 insertions(+), 5 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..8bfd151 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

This tentative commit implements extents/can_extents, roughly as discussed in the previous thread here: https://www.redhat.com/archives/libguestfs/2019-March/msg00017.html I can't say that I'm a big fan of having the plugin allocate an extents array. There are no other plugin callbacks currently where we require the plugin to allocate com...

...Makefile.am | 1 + 10 files changed, 425 insertions(+), 5 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..6e81e0c 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

...f7..d7b6033f 100644 > --- a/plugins/python/nbdkit-python-plugin.pod > +++ b/plugins/python/nbdkit-python-plugin.pod > @@ -271,6 +271,13 @@ contents will be garbage collected. > # return nbdkit.CACHE_NONE or nbdkit.CACHE_EMULATE > # or nbdkit.CACHE_NATIVE > > +=item C<can_extents> > + > +(Optional) > + > + def can_extents(h): > + # return a boolean > + > =item C<pread> > > (Required) > @@ -365,6 +372,14 @@ indicated range. > If the cache operation fails, your function should throw an exception, > optionally using C<nbdk...

I'm not sure which version we're up to now. Anyway I believe this addresses all the points that Eric raised in: https://www.redhat.com/archives/libguestfs/2019-March/msg00038.html https://www.redhat.com/archives/libguestfs/2019-March/msg00040.html In particular: - default state of extents_map is all allocated disk - support hole + non-zero - you can now iterate with bounds -

On Mon, Mar 25, 2019 at 05:01:39PM +0000, Richard W.M. Jones wrote: > > > +=head2 C<.can_extents> [...] > There is definitely a case for removing can_extents completely, and > relying on the fallback. Would this affect language bindings? I > think no because in the language bindings we can also emulate the > .extents callback. > > Associated with this change would be t...

...it-python-plugin.pod index b20f51f7..d7b6033f 100644 --- a/plugins/python/nbdkit-python-plugin.pod +++ b/plugins/python/nbdkit-python-plugin.pod @@ -271,6 +271,13 @@ contents will be garbage collected. # return nbdkit.CACHE_NONE or nbdkit.CACHE_EMULATE # or nbdkit.CACHE_NATIVE +=item C<can_extents> + +(Optional) + + def can_extents(h): + # return a boolean + =item C<pread> (Required) @@ -365,6 +372,14 @@ indicated range. If the cache operation fails, your function should throw an exception, optionally using C<nbdkit.set_error> first. +=item C<extents> + +(Opti...

Second version based on nbdkit_extent* calls, as discussed here: https://www.redhat.com/archives/libguestfs/2019-March/msg00033.html Note in particular there is some subtlety about how filters would work in this implementation. See docs/nbdkit-filter.pod for the details. Rich.

...bdkit.syms | 5 ++ 11 files changed, 528 insertions(+), 4 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..6b22f21 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

...bdkit.syms | 5 ++ 12 files changed, 527 insertions(+), 4 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..1711351 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

...nbdkit.syms | 5 + 11 files changed, 547 insertions(+), 4 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..9e51c68 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

...probing lseek(SEEK_HOLE) to learn if extents are supported; a slight tweak to remember if that result is EOF tells us if we are sparse, and a similar lseek(SEEK_DATA) returning ENXIO tells us if the entire file is a hole, with at most two lseek calls during open (rather than one lseek for each of .can_extents, .init_sparse, and .init_zero). The split plugin is similar to file. For partitioning and linuxdisk, we know we are exposing a sparse image. For other file-based plugins, like ext2, we do not yet expose .extents so it is not worth trying to expose .init_sparse or .init_zero. A successful test d...

On 3/12/19 6:20 AM, Richard W.M. Jones wrote: > This tentative commit implements extents/can_extents, roughly as > discussed in the previous thread here: > > https://www.redhat.com/archives/libguestfs/2019-March/msg00017.html > > I can't say that I'm a big fan of having the plugin allocate an > extents array. There are no other plugin callbacks currently where we >...

...gt; +you would pass in C<offset> as this parameter. > > Well, the offset that the plugin should start at (which might not be > your C<offset>, if your filter is adjusting the plugin's offsets) Very true, I've adjusted the text and the example. > > +=head2 C<.can_extents> > > + > > + int can_extents (void *handle); > > + > > +This is called during the option negotiation phase to find out if the > > +plugin supports detecting allocated (non-sparse) regions of the disk > > +with the C<.extents> callback. > > + >...

...or nbdkit.CACHE_EMULATE > # or nbdkit.CACHE_NATIVE We expose various constants through 'import nbdkit', so... > +++ b/plugins/python/python.c > @@ -1020,6 +1020,79 @@ py_can_cache (void *handle) > return NBDKIT_CACHE_NONE; > } > > +static int > +py_can_extents (void *handle) > +{ > + ACQUIRE_PYTHON_GIL_FOR_CURRENT_SCOPE; > + return boolean_callback (handle, "can_extents", "extents"); > +} > + > +static int > +py_extents (void *handle, uint32_t count, uint64_t offset, > + uint32_t flags, struct nbd...

...kefile.am | 18 +- 12 files changed, 1111 insertions(+), 6 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index dc9a262..7b25c8e 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -350,6 +350,8 @@ calls. =head2 C<.can_zero> +=head2 C<.can_extents> + =head2 C<.can_fua> =head2 C<.can_multi_conn> @@ -365,6 +367,8 @@ calls. void *handle); int (*can_zero) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + int (*can_extents) (struct nbdkit_next_ops *next_ops, void *nxdata,...

...bdkit-plugin.pod > @@ -555,6 +555,20 @@ This callback is not required. If omitted, then nbdkit always tries > C<.zero> first if it is present, and gracefully falls back to > C<.pwrite> if C<.zero> was absent or failed with C<EOPNOTSUPP>. > > +=head2 C<.can_extents> > + > + int can_extents (void *handle); > + > +This is called during the option negotiation phase to find out if the > +plugin supports detecting allocated (non-sparse) regions of the disk > +with the C<.extents> callback. > + > +If there is an error, C<.can_ex...

...t64_t offset, > > + uint32_t flags, struct nbdkit_extents *extents); > > + > > +During the data serving phase, this callback is used to detect > > +allocated, sparse and zeroed regions of the disk. > > + > > +This function will not be called if C<.can_extents> returned false. > > +nbdkit's default behaviour in this case is to treat the whole virtual > > +disk as if it was allocated. > > + > > +The callback should detect and return the list of extents overlapping > > +the range C<[offset...offset+count-1]>. Th...

In can_extents we test whether the VDDK function VixDiskLib_QueryAllocatedBlocks works before declaring that extents are supported. This VDDK function can fail in several ways but most notable if the server doesn't support querying extents. In this case VDDK raises an error message through the error_functi...