search for: nbdkit_extent

.../server/extents.c @@ -42,6 +42,7 @@ #include <assert.h> #include "minmax.h" +#include "vector.h" #include "internal.h" @@ -51,9 +52,11 @@ */ #define MAX_EXTENTS (1 * 1024 * 1024) +/* Appendable list of extents. */ +DEFINE_VECTOR_TYPE(extents, struct nbdkit_extent); + struct nbdkit_extents { - struct nbdkit_extent *extents; - size_t nr_extents, allocated; + extents extents; uint64_t start, end; /* end is one byte beyond the end of the range */ @@ -92,8 +95,7 @@ nbdkit_extents_new (uint64_t start, uint64_t end) nbdkit_error ("nbdkit_exten...

...C<EOPNOTSUPP> (while plugins have automatic fallback to C<.pwrite>, filters do not). +=head2 C<.extents> + + int (*extents) (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, uint32_t count, uint64_t offset, uint32_t flags, + struct nbdkit_extents *extents, + int *err); + +This intercepts the plugin C<.extents> method and can be used to +modify extent requests. + +This function will not be called if C<.can_extents> returned false; in +turn, the filter should not call C<next_ops-E<gt>extents> if +C<...

...C<EOPNOTSUPP> (while plugins have automatic fallback to C<.pwrite>, filters do not). +=head2 C<.extents> + + int (*extents) (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, uint32_t count, uint64_t offset, uint32_t flags, + struct nbdkit_extents *extents, + int *err); + +This intercepts the plugin C<.extents> method and can be used to +modify extent requests. + +This function will not be called if C<.can_extents> returned false; in +turn, the filter should not call C<next_ops-E<gt>extents> if +C<...

...C<EOPNOTSUPP> (while plugins have automatic fallback to C<.pwrite>, filters do not). +=head2 C<.extents> + + int (*extents) (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle, uint32_t count, uint64_t offset, uint32_t flags, + struct nbdkit_extents *extents, + int *err); + +This intercepts the plugin C<.extents> method and can be used to +modify extent requests. + +This function will not be called if C<.can_extents> returned false; in +turn, the filter should not call C<next_ops-E<gt>extents> if +C<...

...some more review: > +++ b/docs/nbdkit-filter.pod > +It is also possible for filters to transform the extents list received > +back from the layer below. Without error checking it would look like > +this: > + > + myfilter_extents (...) > + { > + size_t i; > + struct nbdkit_extents *extents2; > + struct nbdkit_extent e; > + > + extents2 = nbdkit_extents_new (offset); > + next_ops->extents (nxdata, count, offset, flags, extents2, err); > + for (i = 0; i < nbdkit_extents_size (extents2); ++i) { > + e = nbdkit_get_extent (extents2, i); > +...

...lude <sys/socket.h> +#include <nbdkit-compat.h> #include <nbdkit-version.h> +#if defined(NBDKIT_INTERNAL) || !defined(WINDOWS_COMPAT) +#include <sys/socket.h> +#endif + #ifdef __cplusplus extern "C" { #endif @@ -76,6 +80,7 @@ extern "C" { #define NBDKIT_EXTENT_HOLE (1<<0) /* Same as NBD_STATE_HOLE */ #define NBDKIT_EXTENT_ZERO (1<<1) /* Same as NBD_STATE_ZERO */ +#if defined(NBDKIT_INTERNAL) || !defined(WINDOWS_COMPAT) extern void nbdkit_error (const char *msg, ...) ATTRIBUTE_FORMAT_PRINTF (1, 2); extern void nbdkit_verror (const c...

...TTRIBUTE_FORMAT_PRINTF(fmtpos, argpos) #endif +#define NBDKIT_DLLEXPORT + #define NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS 0 #define NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS 1 #define NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS 2 @@ -76,49 +78,63 @@ extern "C" { #define NBDKIT_EXTENT_HOLE (1<<0) /* Same as NBD_STATE_HOLE */ #define NBDKIT_EXTENT_ZERO (1<<1) /* Same as NBD_STATE_ZERO */ -extern void nbdkit_error (const char *msg, ...) ATTRIBUTE_FORMAT_PRINTF (1, 2); -extern void nbdkit_verror (const char *msg, va_list args) +extern NBDKIT_DLLEXPORT void nbdk...

Not sure what version we're up to, but this reimplements extents using the new simpler structure described in this thread: https://www.redhat.com/archives/libguestfs/2019-March/msg00077.html I also fixed most of the things that Eric pointed out in the previous review, although I need to go back over his replies and check I've got everything. This needs a bit more testing. However the

I pushed a few of the msys2 patches upstream. I changed the way that $(SHARED_LDFLAGS) works so it's more to my liking, and the others were pushed unchanged. Three patches remain which I'm posting on the mailing list for proper review. Rich.

...diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index 510781e1..066ca1c7 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -719,6 +719,24 @@ Returns the number of extents in the list. Returns a copy of the C<i>'th extent. +=head3 Enforcing alignment of an nbdkit_extents list + +A convenience function is provided to filters only which makes it +easier to ensure that the client only encounters aligned extents. + + int nbdkit_extents_aligned (struct nbdkit_next_ops *next_ops, + nbdkit_backend *nxdata, + uint32_...

...ARALLEL + +/* This lock protects the global state. */ +static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; + +/* The real size of the underlying plugin. */ +static uint64_t size; + +/* Cached extents from the last extents () call and its start and end for the + sake of simplicity. */ +struct nbdkit_extents *cache_extents; +static uint64_t cache_start; +static uint64_t cache_end; + +static void +cacheextents_unload (void) +{ + nbdkit_extents_free (cache_extents); +} + +static int +cacheextents_add (struct nbdkit_extents *extents) +{ + size_t i = 0; + + for (i = 0; i < nbdkit_extents_count (cach...

Now that cleanup.h is in common code, we can use it in our filters. The first round focuses just on places that called nbdkit_extents_free(), as all three callers had multiple exit paths that definitely benefit from the macro. Signed-off-by: Eric Blake <eblake@redhat.com> --- filters/offset/offset.c | 13 +++++-------- filters/partition/partition.c | 12 ++++-------- filters/truncate/truncate.c | 12 ++++--------...

I'm not sure exactly which version we're up to, but let's say it's version 4. I'm a lot happier with this version: - all filters have been reviewed and changed where I think that's necessary - can_extents is properly defined and implemented now - NBD protocol is followed - I believe it addresses all previous review points where possible The "only" thing

On Sat, Mar 23, 2019 at 11:25:06AM -0500, Eric Blake wrote: > On 3/20/19 5:11 PM, Richard W.M. Jones wrote: > > + myfilter_extents (...) > > + { > > + size_t i; > > + struct nbdkit_extents *extents2; > > + struct nbdkit_extent e; > > + > > + extents2 = nbdkit_extents_new (offset); > > + next_ops->extents (nxdata, count, offset, flags, extents2, err); > > + for (i = 0; i < nbdkit_extents_size (extents2); ++i) { > > + e = nbdkit_ge...

...I hate the filter name, so if anyone has a better idea, feel free to > suggest (or rename) it. Basically it's fine, however it could really do with having a test. I have no particular comment about the name. A few more things inline below. > +static int > +cacheextents_add (struct nbdkit_extents *extents) > +{ > + size_t i = 0; > + > + for (i = 0; i < nbdkit_extents_count (cache_extents); i++) { > + struct nbdkit_extent ex = nbdkit_get_extent (cache_extents, i); > + if (nbdkit_add_extent (extents, ex.offset, ex.length, ex.type) < 0) This really only return...

...n't change our minds on the interface in patch 1, then this looks correct. > +/* Extents. */ > +static int > +offset_extents (struct nbdkit_next_ops *next_ops, void *nxdata, > + void *handle, uint32_t count, uint64_t offs, uint32_t flags, > + struct nbdkit_extents *extents, int *err) > +{ > + size_t i; > + struct nbdkit_extents *extents2; > + struct nbdkit_extent e; > + > + extents2 = nbdkit_extents_new (offs + offset); > + if (extents2 == NULL) { > + *err = errno; > + return -1; > + } Ouch - nbdkit_extents_new() r...

...ze and swab filters want to return aligned extents to the client. I'm posting this as a snapshot of my work in progress on how I plan to get there (it's not quite working yet, but I'm done for today and wanted to at least document my ideas). I might also add a convenience function for nbdkit_extents_offset, since we have a number of filters that repeat the same code for translating all extents from the plugin by an offset. Eric Blake (3): vector: Add VECT_remove extents: Add nbdkit_extents_aligned() RFC swab: Re-enable .extents docs/nbdkit-filter.pod | 18 +++++++++ include/nbdki...

...lude "cleanup.h" + +#define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL + +/* This lock protects the global state. */ +static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; + +/* Cached extents from the last extents () call and its start and end for the + sake of simplicity. */ +struct nbdkit_extents *cache_extents; +static uint64_t cache_start; +static uint64_t cache_end; + +static void +cacheextents_unload (void) +{ + nbdkit_extents_free (cache_extents); +} + +static int +cacheextents_add (struct nbdkit_extents *extents, int *err) +{ + size_t i = 0; + + for (i = 0; i < nbdkit_extents_c...

This has already been pushed upstream. I am simply posting these here so we have a reference in the mailing list in case we find bugs later (as I'm sure we will - it's a complex patch series). Great thanks to Eric Blake for tireless review on this one. It also seems to have identified a few minor bugs in qemu along the way. Rich.

...0) bytes allocated at offset 0 bytes (0x0) nbdkit: sh[1]: error: extents: plugin must return at least one extent qemu-io: Failed to get allocation status: Invalid argument $ nbdkit -U - --filter=offset sh script offset=640k \ --run 'qemu-io -r -f raw -c map $nbd' nbdkit: sh[1]: error: nbdkit_extents_new: start (655360) >= end (393216) qemu-io: Failed to get allocation status: Invalid argument Fixes: 1a5e2d9c, 624abb36 Signed-off-by: Eric Blake <eblake@redhat.com> --- At least we're catching this now before 1.14. docs/nbdkit-filter.pod | 2 +- filters/offset/offset.c...