search for: cache_get_size

...ata) return next (nxdata); } -static void * -cache_open (nbdkit_next_open *next, void *nxdata, int readonly) -{ - if (next (nxdata, readonly) == -1) - return NULL; - - return NBDKIT_HANDLE_NOT_NEEDED; -} - /* Get the file size and ensure the cache is the correct size. */ static int64_t cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, @@ -476,7 +467,6 @@ static struct nbdkit_filter filter = { .unload = cache_unload, .config = cache_config, .config_complete = cache_config_complete, - .open = cache_open, .prepare = cache_pr...

...THREAD_MODEL_PARALLEL + +/* In order to handle parallel requests safely, this lock must be held + * when calling any blk_* functions. + */ +static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; enum cache_mode cache_mode = CACHE_MODE_WRITEBACK; bool cache_on_read = false; @@ -132,6 +138,7 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) { int64_t size; + int r; size = next_ops->get_size (nxdata); if (size == -1) @@ -139,7 +146,10 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_debug ("cache: underlying...

This adds a cache filter, which works like the COW filter in reverse. For realistic use it needs a bit more work, especially to add limits on the size of the cache, a more sensible cache replacement policy, and perhaps some kind of background worker to write dirty blocks out. Rich.

...@ -188,10 +189,12 @@ cache_config_complete (nbdkit_next_config_complete *next, void *nxdata) return next (nxdata); } -/* Get the file size and ensure the cache is the correct size. */ +/* Get the file size; round it down to cache granularity before + * setting cache size. + */ static int64_t cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, - void *handle) + void *handle) { int64_t size; int r; @@ -201,6 +204,7 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; nbdkit_debug ("cache: underlying file size: %"...

When using a filter that rounds up to alignment boundaries, the tail bytes of the plugin are difficult to access correctly. Rather than duplicating lots of code already in the truncate filter, it's easier to just make the other filters default to rounding down and add doc links on how to round up instead. Eric Blake (2): blocksize: Lift restriction against 0-size file cache, cow: Round

While working on adding assertions to pthread_mutex_lock calls, I noticed that the truncate filter's use of mutex didn't really protect us, and isn't really necessary. Cleaning that up also spotted a couple of other potential cleanups. Eric Blake (4): filters: Drop useless .open callbacks truncate: Fix corruption when plugin changes per-connection size truncate: Test for safe

...stats.c | 18 ++++++------------ filters/error/Makefile.am | 5 ++++- 8 files changed, 40 insertions(+), 48 deletions(-) diff --git a/filters/cache/cache.c b/filters/cache/cache.c index 6a9966e..b3fef42 100644 --- a/filters/cache/cache.c +++ b/filters/cache/cache.c @@ -209,9 +209,8 @@ cache_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_debug ("cache: underlying file size: %" PRIi64, size); - pthread_mutex_lock (&lock); + ACQUIRE_LOCK_FOR_CURRENT_SCOPE (&lock); r = blk_set_size (size); - pthread_mutex_unlock (&lock); if (r == -1) retur...

This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.

...@@ static struct nbdkit_filter filter = { .load = cache_load, .unload = cache_unload, .config = cache_config, + .config_complete = cache_config_complete, .open = cache_open, .prepare = cache_prepare, .get_size = cache_get_size, diff --git a/filters/cache/lru.c b/filters/cache/lru.c index 60cf379..8159040 100644 --- a/filters/cache/lru.c +++ b/filters/cache/lru.c @@ -109,8 +109,11 @@ lru_set_size (uint64_t new_size) if (bitmap_resize (&bm[1], new_size) == -1) return -1; - /* XXX Choose this better. */ - N...

...@@ static struct nbdkit_filter filter = { .load = cache_load, .unload = cache_unload, .config = cache_config, + .config_complete = cache_config_complete, .open = cache_open, .prepare = cache_prepare, .get_size = cache_get_size, diff --git a/filters/cache/lru.c b/filters/cache/lru.c index 60cf379..8159040 100644 --- a/filters/cache/lru.c +++ b/filters/cache/lru.c @@ -109,8 +109,11 @@ lru_set_size (uint64_t new_size) if (bitmap_resize (&bm[1], new_size) == -1) return -1; - /* XXX Choose this better. */ - N...

...@@ static struct nbdkit_filter filter = { .load = cache_load, .unload = cache_unload, .config = cache_config, + .config_complete = cache_config_complete, .open = cache_open, .prepare = cache_prepare, .get_size = cache_get_size, diff --git a/filters/cache/lru.c b/filters/cache/lru.c index 60cf379..2f379d1 100644 --- a/filters/cache/lru.c +++ b/filters/cache/lru.c @@ -69,9 +69,10 @@ * * To do this we keep two bitmaps. * - * When a block is accessed, we set the corresponding bit in bm[0] and - * increment c0 (c0 count...

...size_get_size (struct nbdkit_next_ops *next_ops, void *nxdata, diff --git a/filters/cache/cache.c b/filters/cache/cache.c index b5dbccd2..e5f18d9b 100644 --- a/filters/cache/cache.c +++ b/filters/cache/cache.c @@ -239,7 +239,6 @@ cache_prepare (struct nbdkit_next_ops *next_ops, void *nxdata, r = cache_get_size (next_ops, nxdata, handle); if (r < 0) return -1; - /* TODO: cache per-connection FUA mode? */ return 0; } diff --git a/filters/cow/cow.c b/filters/cow/cow.c index 9d91d432..e4330bf3 100644 --- a/filters/cow/cow.c +++ b/filters/cow/cow.c @@ -127,7 +127,7 @@ cow_get_size (struct nbd...

I do have a question about whether patch 2 is right, or whether I've exposed a bigger problem in the truncate (and possibly other) filter, but the rest seem fairly straightforward. Eric Blake (4): server: Check for pthread lock failures truncate: Factor out reading real_size under mutex plugins: Check for mutex failures filters: Check for mutex failures filters/cache/cache.c

...@@ static struct nbdkit_filter filter = { .load = cache_load, .unload = cache_unload, .config = cache_config, + .config_complete = cache_config_complete, .open = cache_open, .prepare = cache_prepare, .get_size = cache_get_size, diff --git a/filters/cache/lru.c b/filters/cache/lru.c index c828968..2f379d1 100644 --- a/filters/cache/lru.c +++ b/filters/cache/lru.c @@ -110,8 +110,11 @@ lru_set_size (uint64_t new_size) if (bitmap_resize (&bm[1], new_size) == -1) return -1; - /* XXX Choose this better. */ - N...

Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.

...@@ static struct nbdkit_filter filter = { .load = cache_load, .unload = cache_unload, .config = cache_config, + .config_complete = cache_config_complete, .open = cache_open, .prepare = cache_prepare, .get_size = cache_get_size, diff --git a/filters/cache/lru.c b/filters/cache/lru.c index 9e20408..90352b5 100644 --- a/filters/cache/lru.c +++ b/filters/cache/lru.c @@ -90,8 +90,8 @@ static unsigned N = 100; void lru_init (void) { - bitmap_init (&bm[0], BLKSIZE, 1 /* bits per block */); - bitmap_init (&bm[1], BL...

v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I

These are essentially identical to what was previously posted as patches 6/9 through 9/9 here: https://www.redhat.com/archives/libguestfs/2018-December/msg00145.html except that it has been rebased onto the current git master and retested thoroughly. Rich.

Since v1: - rework .can_cache to be tri-state, with default of no advertisement (ripple effect through other patches) - add a lot more patches in order to round out filter support And in the meantime, Rich pushed NBD_CMD_CACHE support into libnbd, so in theory we now have a way to test cache commands through the entire stack. Eric Blake (24): server: Internal hooks for implementing