search for: bm_entry

Displaying 20 results from an estimated 24 matches for "bm_entry".

2018 Dec 01
0
[PATCH nbdkit] common: Move shared bitmap code to a common library.
...cow/cow.c. */ #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS @@ -75,10 +75,7 @@ static int fd = -1; * 10 = <unused> * 11 = block cached and dirty */ -static uint8_t *bitmap; - -/* Size of the bitmap in bytes. */ -static size_t bm_size; +static struct bitmap bm; enum bm_entry { BLOCK_NOT_CACHED = 0, @@ -93,9 +90,7 @@ static enum cache_mode { CACHE_MODE_UNSAFE, } cache_mode = CACHE_MODE_WRITEBACK; -static int -cache_flush (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle, - uint32_t flags, int *err); +static int cache_flush (struct nbdkit...
2018 Dec 02
0
[PATCH nbdkit v2] common: Move shared bitmap code to a common library.
...cow/cow.c. */ #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS @@ -75,10 +75,7 @@ static int fd = -1; * 10 = <unused> * 11 = block cached and dirty */ -static uint8_t *bitmap; - -/* Size of the bitmap in bytes. */ -static size_t bm_size; +static struct bitmap bm; enum bm_entry { BLOCK_NOT_CACHED = 0, @@ -93,9 +90,7 @@ static enum cache_mode { CACHE_MODE_UNSAFE, } cache_mode = CACHE_MODE_WRITEBACK; -static int -cache_flush (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle, - uint32_t flags, int *err); +static int cache_flush (struct nbdkit...
2018 Dec 03
0
[PATCH nbdkit v3] common: Move shared bitmap code to a common library.
...cow/cow.c. */ #define THREAD_MODEL NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS @@ -75,10 +75,7 @@ static int fd = -1; * 10 = <unused> * 11 = block cached and dirty */ -static uint8_t *bitmap; - -/* Size of the bitmap in bytes. */ -static size_t bm_size; +static struct bitmap bm; enum bm_entry { BLOCK_NOT_CACHED = 0, @@ -93,9 +90,7 @@ static enum cache_mode { CACHE_MODE_UNSAFE, } cache_mode = CACHE_MODE_WRITEBACK; -static int -cache_flush (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle, - uint32_t flags, int *err); +static int cache_flush (struct nbdkit...
2018 Dec 01
2
[PATCH nbdkit] common: Move shared bitmap code to a common library.
I have some patches I'm working on to fix the cache filter. However this is a prelude. It should be simply pure refactoring. All tests pass still. Rich.
2018 Dec 03
3
[PATCH nbdkit v3] common: Move shared bitmap code to a common library.
v2: https://www.redhat.com/archives/libguestfs/2018-December/msg00039.html v2 -> v3: - Fix all the issues raised in Eric's review. - Precompute some numbers to make the calculations easier. - Calculations now use bitshifts and masks in preference to division and modulo. - Clear existing bits before setting (which fixes a bug in the cache filter). Rich.
2018 Dec 02
2
[PATCH nbdkit v2] common: Move shared bitmap code to a common library.
This is exactly the same as v1: https://www.redhat.com/archives/libguestfs/2018-December/msg00004.html except that it now frees the bitmap on unload (which the old code did not - there was always a memory leak). Rich.
2018 Jan 22
1
[PATCH nbdkit] filters: Add caching filter.
This adds a cache filter, which works like the COW filter in reverse. For realistic use it needs a bit more work, especially to add limits on the size of the cache, a more sensible cache replacement policy, and perhaps some kind of background worker to write dirty blocks out. Rich.
2018 Dec 28
12
[PATCH nbdkit 0/9] cache: Implement cache-max-size and method of reclaiming space from the cache.
This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.
2018 Dec 28
0
[PATCH nbdkit 9/9] cache: Implement cache-max-size and method of reclaiming space from the cache.
...index b256446..4a7f65f 100644 --- a/filters/cache/blk.c +++ b/filters/cache/blk.c @@ -46,6 +46,8 @@ #include <unistd.h> #include <fcntl.h> #include <errno.h> +#include <sys/types.h> +#include <sys/stat.h> #include <nbdkit-filter.h> @@ -74,6 +76,31 @@ enum bm_entry { BLOCK_DIRTY = 3, }; +#ifdef HAVE_CACHE_RECLAIM +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU b...
2018 Jan 26
2
[PATCH nbdkit] filters: cache, cow: Handle bitmap overflow on 32 bit architectures.
...--git a/filters/cache/cache.c b/filters/cache/cache.c index 7410f0d..9473f2c 100644 --- a/filters/cache/cache.c +++ b/filters/cache/cache.c @@ -72,7 +72,7 @@ static int fd = -1; static uint8_t *bitmap; /* Size of the bitmap in bytes. */ -static uint64_t bm_size; +static size_t bm_size; enum bm_entry { BLOCK_NOT_CACHED = 0, @@ -167,7 +167,14 @@ blk_set_size (uint64_t new_size) { uint8_t *new_bm; const size_t old_bm_size = bm_size; - size_t new_bm_size = DIV_ROUND_UP (new_size, BLKSIZE*8/2); + uint64_t new_bm_size_u64 = DIV_ROUND_UP (new_size, BLKSIZE*8/2); + size_t new_bm_size; + +...
2019 Jan 01
0
[PATCH nbdkit v2 4/4] cache: Implement cache-max-size and method of reclaiming space from the cache.
...index b256446..6f67a7f 100644 --- a/filters/cache/blk.c +++ b/filters/cache/blk.c @@ -46,6 +46,8 @@ #include <unistd.h> #include <fcntl.h> #include <errno.h> +#include <sys/types.h> +#include <sys/stat.h> #include <nbdkit-filter.h> @@ -74,6 +76,31 @@ enum bm_entry { BLOCK_DIRTY = 3, }; +#ifdef HAVE_CACHE_RECLAIM +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU b...
2019 Jan 03
0
[PATCH nbdkit v3 2/2] cache: Implement cache-max-size and method of reclaiming space from the cache.
...index b256446..294f0cb 100644 --- a/filters/cache/blk.c +++ b/filters/cache/blk.c @@ -46,6 +46,8 @@ #include <unistd.h> #include <fcntl.h> #include <errno.h> +#include <sys/types.h> +#include <sys/stat.h> #include <nbdkit-filter.h> @@ -74,6 +76,34 @@ enum bm_entry { BLOCK_DIRTY = 3, }; +#ifdef HAVE_CACHE_RECLAIM +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU b...
2019 Jan 03
0
[PATCH nbdkit v4 2/2] cache: Implement cache-max-size and method of reclaiming space from the cache.
...-- a/filters/cache/blk.c +++ b/filters/cache/blk.c @@ -54,6 +54,7 @@ #include "cache.h" #include "blk.h" #include "lru.h" +#include "reclaim.h" /* The cache. */ static int fd = -1; @@ -69,7 +70,7 @@ static int fd = -1; static struct bitmap bm; enum bm_entry { - BLOCK_NOT_CACHED = 0, + BLOCK_NOT_CACHED = 0, /* assumed to be zero by reclaim code */ BLOCK_CLEAN = 1, BLOCK_DIRTY = 3, }; @@ -146,6 +147,8 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, off_t offset = blknum * BLKSIZE; enum bm_entry state = bitmap_get_blk (&bm...
2019 Jan 03
3
[PATCH nbdkit v3 0/2] cache: Implement cache-max-size and method of reclaiming space from the cache.
Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.
2019 Jan 04
0
[PATCH nbdkit v5 3/3] cache: Implement cache-max-size and cache space reclaim.
...t; #include "bitmap.h" +#include "minmax.h" #include "cache.h" #include "blk.h" #include "lru.h" +#include "reclaim.h" /* The cache. */ static int fd = -1; @@ -69,7 +72,7 @@ static int fd = -1; static struct bitmap bm; enum bm_entry { - BLOCK_NOT_CACHED = 0, + BLOCK_NOT_CACHED = 0, /* assumed to be zero by reclaim code */ BLOCK_CLEAN = 1, BLOCK_DIRTY = 3, }; @@ -80,10 +83,7 @@ blk_init (void) const char *tmpdir; size_t len; char *template; - - lru_init (); - - bitmap_init (&bm, BLKSIZE, 2 /* bits per bl...
2019 Jan 01
7
[PATCH nbdkit v2 0/4] cache: Implement cache-max-size etc.
These are essentially identical to what was previously posted as patches 6/9 through 9/9 here: https://www.redhat.com/archives/libguestfs/2018-December/msg00145.html except that it has been rebased onto the current git master and retested thoroughly. Rich.
2019 Jan 03
2
Re: [PATCH nbdkit v2 4/4] cache: Implement cache-max-size and method of reclaiming space from the cache.
...t; + Should the man page mention that max cache size can only be enforced with kernel support? Do we want to go further and probe whether FALLOC_FL_PUNCH_HOLE works on $TMPDIR? (Even if the kernel supports punching holes, there are some filesystems that don't). > @@ -74,6 +76,31 @@ enum bm_entry { > BLOCK_DIRTY = 3, > }; > > +#ifdef HAVE_CACHE_RECLAIM > +/* If we are currently reclaiming blocks from the cache. > + * > + * The state machine starts in the NOT_RECLAIMING state. When the > + * size of the cache exceeds the high threshold, we move to > + * RECL...
2019 Jan 03
4
[PATCH nbdkit v4 0/2] cache: Implement cache-max-size and method of
v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.
2019 Jan 04
5
[PATCH nbdkit v5 3/3] cache: Implement cache-max-size and cache space reclaim.
v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I
2018 Feb 01
0
[nbdkit PATCH v2 1/3] backend: Rework internal/filter error return semantics
...-1) { + r = blk_writeback (next_ops, nxdata, blknum, block); + if (r) { free (block); - return -1; + return r; } count -= n; @@ -463,7 +473,7 @@ cache_flush (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) uint64_t i, j; uint64_t blknum; enum bm_entry state; - unsigned errors = 0; + int error = 0, r; if (cache_mode == CACHE_MODE_UNSAFE) return 0; @@ -494,10 +504,10 @@ cache_flush (struct nbdkit_next_ops *next_ops, void *nxdata, void *handle) /* Perform a read + writethrough which will read from the * cache and w...