search for: next_op

After a retry, if the second connection has fewer permissions than the first, but we blindly call next_ops->FOO, we end up triggering an assertion failure in backend.c. This is particularly noticeable when the force_readonly flag is in effect, as that makes it much easier for there to be fewer permissions than before. Add testsuite coverage of pwrite to demonstrate. Signed-off-by: Eric Blake <...

...retry.c > index 840d7383..cf8f5246 100644 > --- a/filters/retry/retry.c > +++ b/filters/retry/retry.c > @@ -148,6 +148,17 @@ struct retry_data { > int delay; /* Seconds to wait before retrying. */ > }; > > +static bool > +valid_range (struct nbdkit_next_ops *next_ops, void *nxdata, > + uint32_t count, uint64_t offset, bool is_write, int *err) > +{ > + if ((int64_t) offset + count > next_ops->get_size (nxdata)) { > + *err = is_write ? ENOSPC : EIO; > + return false; > + } > + return true; > +} > +...

...-git a/filters/retry/retry.c b/filters/retry/retry.c index 840d7383..cf8f5246 100644 --- a/filters/retry/retry.c +++ b/filters/retry/retry.c @@ -148,6 +148,17 @@ struct retry_data { int delay; /* Seconds to wait before retrying. */ }; +static bool +valid_range (struct nbdkit_next_ops *next_ops, void *nxdata, + uint32_t count, uint64_t offset, bool is_write, int *err) +{ + if ((int64_t) offset + count > next_ops->get_size (nxdata)) { + *err = is_write ? ENOSPC : EIO; + return false; + } + return true; +} + static bool do_retry (struct retry_handle *...

...--------- filters/error/Makefile.am | 5 ++++- 8 files changed, 40 insertions(+), 48 deletions(-) diff --git a/filters/cache/cache.c b/filters/cache/cache.c index 6a9966e..b3fef42 100644 --- a/filters/cache/cache.c +++ b/filters/cache/cache.c @@ -209,9 +209,8 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_debug ("cache: underlying file size: %" PRIi64, size); - pthread_mutex_lock (&lock); + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock); r = blk_set_size (size); - pthread_mutex_unlock (&lock); if (r == -1) return -1; @@ -266,9 +265,1...

Based on Rich's review of my v1 that touched only cache.c, I have now tried to bring all three filters with alignment rounding in line with one another. There is definitely room for future improvements once we teach nbdkit to let filters and plugins advertise block sizes, but I'm hoping to get NBD_CMD_CACHE implemented first. Eric Blake (2): blocksize: Process requests in linear order

...r to use. But with filters in the mix, it is very difficult for a filter to know what error was set by the plugin (particularly since nbdkit_set_error() has no public counterpart for reading the thread-local storage). What's more, if a filter does any non-trivial processing after calling into next_ops, it is very probable that errno might be corrupted, which would affect the error returned by a plugin that relied on errno but not the error stored in thread-local storage. Better is to change the backend interface to just pass the direct error value, by moving the decoding of thread-local vs. er...

...s changed, 96 insertions(+), 52 deletions(-) diff --git a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index 55dfab1..5fed7ca 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -131,7 +131,12 @@ C<nbdkit_next_config_complete>, C<nbdkit_next_preconnect>, C<nbdkit_next_open>) and a structure called C<struct nbdkit_next_ops>. These abstract the next plugin or filter in the chain. There is also an opaque pointer C<nxdata> which must be passed along when calling -these functions. +these functions. The value of C<nxdata> passed to C<.open>...

...order to handle parallel requests safely, this lock must be held + * when calling any blk_* functions. + */ +static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; enum cache_mode cache_mode = CACHE_MODE_WRITEBACK; bool cache_on_read = false; @@ -132,6 +138,7 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) { int64_t size; + int r; size = next_ops->get_size (nxdata); if (size == -1) @@ -139,7 +146,10 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_debug ("cache: underlying file size: %" PRIi...

...ave the same stability +guarantee, and nbdkit may refuse to use a filter that was compiled +against a different version rather than risk misbehavior. =head1 C<nbdkit-filter.h> @@ -320,11 +325,15 @@ error message and return C<-1>. =head2 C<.pread> int (*pread) (struct nbdkit_next_ops *next_ops, void *nxdata, - void *handle, void *buf, uint32_t count, uint64_t offset); + void *handle, void *buf, uint32_t count, uint64_t offset, + uint32_t flags); This intercepts the plugin C<.pread> method and can be used to read or modify data...

...rn f->filter.preconnect (next_preconnect, &nxdata, readonly); + return f->filter.preconnect (next_preconnect, b->next, readonly); else return b->next->preconnect (b->next, readonly); } @@ -216,201 +205,181 @@ plugin_magic_config_key (struct backend *b) static int next_open (void *nxdata, int readonly) { - struct b_h *b_h = nxdata; + struct backend *b_next = nxdata; - return backend_open (b_h->b, readonly); + return backend_open (b_next, readonly); } static void * filter_open (struct backend *b, int readonly) { struct backend_filter *f = container...

With our recent cleanups to nxdata, the only remaining difference between functions like backend_open() and next_open() was the signature (one used void*, the other struct backend *); the API is compatible. All of our filters are in-tree, and we don't promise API/ABI stability, but it is still a lot of files to touch, so the simplest solution to avoid the redundant hop through wrapper functions is to change...

.../filters/ext2/ext2.c index 75ac2c4c..7ad4a005 100644 --- a/filters/ext2/ext2.c +++ b/filters/ext2/ext2.c @@ -294,6 +294,25 @@ static int ext2_thread_model (void) return NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS; } +/* Description. */ +static const char * +ext2_export_description (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle) +{ + struct handle *h = handle; + const char *fname = file ?: h->exportname; + const char *slash = fname[0] == '/' ? "" : "/"; + const char *base = next_ops->export_description (nxdata); + CLEAN...

...following conditions are @@ -58,6 +58,8 @@ #include "cache.h" #include "blk.h" #include "reclaim.h" +#include "isaligned.h" +#include "minmax.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -231,43 +233,68 @@ cache_pread (struct nbdkit_next_ops *next_ops, void *nxdata, uint32_t flags, int *err) { CLEANUP_FREE uint8_t *block = NULL; + uint64_t blknum, blkoffs; + int r; assert (!flags); - block = malloc (blksize); - if (block == NULL) { - *err = errno; - nbdkit_error ("malloc: %m"); - return -1;...

...re permitted provided that the following conditions are @@ -58,6 +58,7 @@ #include "cache.h" #include "blk.h" #include "reclaim.h" +#include "isaligned.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -233,11 +234,13 @@ cache_pread (struct nbdkit_next_ops *next_ops, void *nxdata, CLEANUP_FREE uint8_t *block = NULL; assert (!flags); - block = malloc (blksize); - if (block == NULL) { - *err = errno; - nbdkit_error ("malloc: %m"); - return -1; + if (!IS_ALIGNED (count | offset, blksize)) { + block = malloc (blksize); +...

The truncate filter tried to be careful to lock access to setting or reading the real_size variable learned from calling next_ops->get_size, in anticipation of not behaving incorrectly if the NBD protocol makes dynamic resize supported, and where the global variable could serve as a cache rather than having to always call next_ops->get_size to recompute things. However, nbdkit-plugin.pod already documents that .get_siz...

Here's what I'm currently playing with; I'm not ready to commit anything until I rebase my FUA work on top of this, as I only want to break filter ABI once between releases. Eric Blake (2): backend: Rework internal/filter error return semantics filters: Add log filter TODO | 2 - docs/nbdkit-filter.pod | 84 +++++++-- docs/nbdkit.pod

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

...the captive nbdkit to exit was overkill), and four new patches. The tests are intentionally separate, to allow rearranging the order of the series to see the failures being fixed. Eric Blake (6): server: Propagate unexpected nbdkit failure with --run tests: Enhance captive test retry: Check next_ops->can_FOO on retry tests: Test for retry-readonly behavior retry: Avoid assertion during retried extents tests: Test retry after partial extents server/captive.c | 47 +++++++++++---- filters/retry/retry.c | 80 ++++++++++++++++++++++-- tests/Makefile.am |...

...100644 > --- a/filters/ext2/ext2.c > +++ b/filters/ext2/ext2.c > @@ -294,6 +294,25 @@ static int ext2_thread_model (void) > return NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS; > } > > +/* Description. */ > +static const char * > +ext2_export_description (struct nbdkit_next_ops *next_ops, void *nxdata, > + void *handle) > +{ > + struct handle *h = handle; > + const char *fname = file ?: h->exportname; > + const char *slash = fname[0] == '/' ? "" : "/"; > + const char *base = next_ops->export_...

...a/docs/nbdkit-filter.pod b/docs/nbdkit-filter.pod index 55dfab1..0f81684 100644 --- a/docs/nbdkit-filter.pod +++ b/docs/nbdkit-filter.pod @@ -413,6 +413,10 @@ cached value. =head2 C<.can_cache> +=head2 C<.init_sparse> + +=head2 C<.init_zero> + int (*can_write) (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); int (*can_flush) (struct nbdkit_next_ops *next_ops, void *nxdata, @@ -434,6 +438,10 @@ cached value. void *handle); int (*can_cache) (struct nbdkit_next_ops *next_ops, void *nxdata, void *h...