search for: truncate_get_size

...9;s > because we are storing 'size' and 'real_size' as globals rather than > per-connection values in the handle, and we're worrying about parallel > connections where we don't want one thread reading inconsistent values > while another thread is in the middle of truncate_get_size()? At any > rate, as long as we don't have dynamic resize in the NBD protocol, > size can't change within the confines of a single connection. And even > if we DO assume that the underlying plugin reports different sizes for > different connections, our use of a global does not...

...nyway. I guess it's because we are storing 'size' and 'real_size' as globals rather than per-connection values in the handle, and we're worrying about parallel connections where we don't want one thread reading inconsistent values while another thread is in the middle of truncate_get_size()? At any rate, as long as we don't have dynamic resize in the NBD protocol, size can't change within the confines of a single connection. And even if we DO assume that the underlying plugin reports different sizes for different connections, our use of a global does not play well (if client...

...nst char *value, unsigned *ret) { @@ -121,51 +112,90 @@ truncate_config (nbdkit_next_config *next, void *nxdata, "round-up=<N> Round up to next multiple of N.\n" \ "round-down=<N> Round down to multiple of N." -static int64_t truncate_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); +/* Per-connection state. Until the NBD protocol gains dynamic resize + * support, each connection remembers the size of the underlying + * plugin at open (even if that size differs between connections + * because the plugin tracks ext...

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

While working on adding assertions to pthread_mutex_lock calls, I noticed that the truncate filter's use of mutex didn't really protect us, and isn't really necessary. Cleaning that up also spotted a couple of other potential cleanups. Eric Blake (4): filters: Drop useless .open callbacks truncate: Fix corruption when plugin changes per-connection size truncate: Test for safe

...lue); +} + +#define truncate_config_help \ + "truncate=<SIZE> The new size.\n" \ + "round-up=<N> Round up to next multiple of N.\n" \ + "round-down=<N> Round down to multiple of N." + +static int64_t truncate_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + +/* In prepare, force a call to get_size which sets the real_size & size + * globals. + */ +static int +truncate_prepare (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle) +{ + int64_t r; + + r =...

...lue); +} + +#define truncate_config_help \ + "truncate=<SIZE> The new size.\n" \ + "round-up=<N> Round up to next multiple of N.\n" \ + "round-down=<N> Round down to multiple of N." + +static int64_t truncate_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle); + +/* In prepare, force a call to get_size which sets the real_size & size + * globals. + */ +static int +truncate_prepare (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle) +{ + int64_t r; + + r =...

Since we're using ({ .. }) gcc/clang extension, let's rewrite the rounding.h change too. Rich.

...okay, though. > +++ b/filters/truncate/truncate.c > @@ -46,6 +46,7 @@ > > #include "ispowerof2.h" > #include "iszero.h" > +#include "rounding.h" > > #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL > > @@ -157,9 +158,9 @@ truncate_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, > if (truncate_size >= 0) > size = truncate_size; > if (round_up > 0) > - size = (size + round_up - 1) & ~(round_up - 1); > + size = ROUND_UP (size, round_up); > if (round_down > 0) > - siz...

...ncate.c index 185f6cc..b95432a 100644 --- a/filters/truncate/truncate.c +++ b/filters/truncate/truncate.c @@ -46,6 +46,7 @@ #include "ispowerof2.h" #include "iszero.h" +#include "rounding.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -157,9 +158,9 @@ truncate_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, if (truncate_size >= 0) size = truncate_size; if (round_up > 0) - size = (size + round_up - 1) & ~(round_up - 1); + size = ROUND_UP (size, round_up); if (round_down > 0) - size &= ~(round_down - 1); + size =...

...ncate.c index 185f6cc..b95432a 100644 --- a/filters/truncate/truncate.c +++ b/filters/truncate/truncate.c @@ -46,6 +46,7 @@ #include "ispowerof2.h" #include "iszero.h" +#include "rounding.h" #define THREAD_MODEL NBDKIT_THREAD_MODEL_PARALLEL @@ -157,9 +158,9 @@ truncate_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, if (truncate_size >= 0) size = truncate_size; if (round_up > 0) - size = (size + round_up - 1) & ~(round_up - 1); + size = ROUND_UP (size, round_up); if (round_down > 0) - size &= ~(round_down - 1); + size =...

I have dropped the map filter from this series for now while I try to get it working. However I think the truncate filter is in a good shape. This incorporates all feedback from Eric's review. Also there are three small fixes to the filter code, all revealed when I was testing using multiple filters which we'd not done much of before. Rich.

This patch series proposes two new filters. * truncate: This can truncate, extend, round up or round down the size of a plugin/device. A typical usage is to fix the qemu problem that it can only handle devices which are a multiple of 512-bytes: nbdkit --filter=truncate random size=500 round-up=512 This will serve a virtual device with size 512 bytes. Reading from the last 12 bytes will

...t; h->size) + return 1; + return next_ops->init_sparse (nxdata); +} + /* Read data. */ static int truncate_pread (struct nbdkit_next_ops *next_ops, void *nxdata, @@ -426,6 +438,7 @@ static struct nbdkit_filter filter = { .prepare = truncate_prepare, .get_size = truncate_get_size, .can_fast_zero = truncate_can_fast_zero, + .init_sparse = truncate_init_sparse, .pread = truncate_pread, .pwrite = truncate_pwrite, .trim = truncate_trim, diff --git a/include/nbdkit-filter.h b/include/nbdkit-filter.h index 3500317..d327bf...

With these patches, a majority of tests pass. The notable things which are still broken: - Because FreeBSD links /home -> /usr/home, $(pwd) gives a different result from realpath(2). Therefore some tests which implicitly rely on (eg) a plugin which calls nbdkit_realpath internally and then checking that path against $(pwd) fail. - Shebangs (#!) don't seem to work the same way

nbdkit partitioning boot.img swap.img root.img ... creates a virtual disk by adding a partition table. In ancient times Xen used to do this. Rich.

The partitioning plugin patch is the same (except for rebasing). However I have changed the first two patches based on feedback received. In particular this fixes a very serious bug found by Eric Blake in the current truncate filter. Rich.

This is a cover letter to a series of patches being proposed in tandem to four different projects: - nbd: Document a new NBD_CMD_FLAG_FAST_ZERO command flag - qemu: Implement the flag for both clients and server - libnbd: Implement the flag for clients - nbdkit: Implement the flag for servers, including the nbd passthrough client If you want to test the patches together, I've pushed a

I will be following up to this email with four separate threads each addressed to the appropriate single list, with proposed changes to: - the NBD protocol - qemu: both server and client - libnbd: client - nbdkit: server The feature in question adds a new optional NBD_INFO_ packet to the NBD_OPT_GO portion of handshake, adding up to 16 bits of information that the server can advertise to the