search for: cow_get_size

...nderlying plugin. We aren't validating that filter calls to next_ops are within bounds; and even if the plugin tolerates the past-EOF read, we aren't guaranteeing that the client will always read 0 bytes in the past-EOF tail. Several ideas of fixing it, each with some drawbacks: + in cache/cow_get_size(), truncate the plugin's size down to blksize prior to calling blk_set_size() (renders the plugin's tail unusable) + reject serving images that aren't already aligned to blksize (avoids missing bytes or worrying about past-EOF slop, but can be mean, unless...) + document that for unalig...

Eric, you'll probably find the design "interesting" ... It does work, for me at least. Rich.

...a non-NULL + * pointer that we can return. + */ + static int handle; + + /* Always pass readonly=1 to the underlying plugin. */ + if (next (nxdata, 1) == -1) + return NULL; + + return &handle; +} + +/* Get the file size and ensure the overlay is the correct size. */ +static int64_t +cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, + void *handle) +{ + int64_t size; + + size = next_ops->get_size (nxdata); + if (size == -1) + return -1; + + if (ftruncate (fd, size) == -1) + return -1; + + nbdkit_debug ("cow: underlying file size: %" PRIi64, s...

...80,7 +81,9 @@ cow_open (nbdkit_next_open *next, void *nxdata, int readonly) return NBDKIT_HANDLE_NOT_NEEDED; } -/* Get the file size and ensure the overlay is the correct size. */ +/* Get the file size; round it down to overlay granularity before + * setting overlay size. + */ static int64_t cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) @@ -93,6 +96,7 @@ cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; nbdkit_debug ("cow: underlying file size: %" PRIi64, size); + size = ROUND_DOWN (size, BLKSIZE); ACQUIRE_L...

...a, r = cache_get_size (next_ops, nxdata, handle); if (r < 0) return -1; - /* TODO: cache per-connection FUA mode? */ return 0; } diff --git a/filters/cow/cow.c b/filters/cow/cow.c index 9d91d432..e4330bf3 100644 --- a/filters/cow/cow.c +++ b/filters/cow/cow.c @@ -127,7 +127,7 @@ cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, return size; } -/* Force an early call to cow_get_size, consequently truncating the +/* Force early calls to populate nbdkit's cache, and truncate the * overlay to the correct size. */ static int @@ -137,7 +137,14 @@ cow_prepare (struct...

When using a filter that rounds up to alignment boundaries, the tail bytes of the plugin are difficult to access correctly. Rather than duplicating lots of code already in the truncate filter, it's easier to just make the other filters default to rounding down and add doc links on how to round up instead. Eric Blake (2): blocksize: Lift restriction against 0-size file cache, cow: Round

...bug ("cow: bitmap resized to %" PRIu64 " bytes", new_bm_size); + + if (ftruncate (fd, new_size) == -1) { + nbdkit_error ("ftruncate: %m"); + return -1; + } + + return 0; +} + /* Get the file size and ensure the overlay is the correct size. */ static int64_t cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, @@ -158,11 +186,11 @@ cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, if (size == -1) return -1; - if (ftruncate (fd, size) == -1) - return -1; - nbdkit_debug ("cow: underlying file size: %" PRIi64, size); +...

...ch_dirty_block (flush_dirty_block, &data); + } /* Now issue a flush request to the underlying storage. */ if (next_ops->flush (nxdata, 0, diff --git a/filters/cow/cow.c b/filters/cow/cow.c index 04ec44f..35ad718 100644 --- a/filters/cow/cow.c +++ b/filters/cow/cow.c @@ -92,9 +92,8 @@ cow_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_debug ("cow: underlying file size: %" PRIi64, size); - pthread_mutex_lock (&lock); + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock); r = blk_set_size (size); - pthread_mutex_unlock (&lock); if (r == -1) return...

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

It's easy to use the sh script to demonstrate that nbdkit is inefficiently calling into .get_size, .can_fua, and friends more than necessary. We've also commented on the list in the past that it would be nice to ensure that when filters call into next_ops, they are not violating constraints (as we've have to fix several bugs in the past where we did not have such checking to protect

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack. Eric Blake (24): server: Internal hooks for implementing