search for: not_reclaiming

...<errno.h> +#include <sys/types.h> +#include <sys/stat.h> #include <nbdkit-filter.h> @@ -74,6 +76,31 @@ enum bm_entry { BLOCK_DIRTY = 3, }; +#ifdef HAVE_CACHE_RECLAIM +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to + * RECLAIMING_ANY (reclaiming any blocks). + * + * If at any time the size of the cache goes below the low threshold + * we move back to the NOT_RECL...

...<errno.h> +#include <sys/types.h> +#include <sys/stat.h> #include <nbdkit-filter.h> @@ -74,6 +76,31 @@ enum bm_entry { BLOCK_DIRTY = 3, }; +#ifdef HAVE_CACHE_RECLAIM +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to + * RECLAIMING_ANY (reclaiming any blocks). + * + * If at any time the size of the cache goes below the low threshold + * we move back to the NOT_RECL...

...<errno.h> +#include <sys/types.h> +#include <sys/stat.h> #include <nbdkit-filter.h> @@ -74,6 +76,34 @@ enum bm_entry { BLOCK_DIRTY = 3, }; +#ifdef HAVE_CACHE_RECLAIM +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to + * RECLAIMING_ANY (reclaiming any blocks). + * + * If at any time the size of the cache goes below the low threshold + * we move back to the NOT_RECL...

...h" +#include "reclaim.h" +#include "lru.h" + +#ifndef HAVE_CACHE_RECLAIM + +void +reclaim (int fd, struct bitmap *bm) +{ + /* nothing */ +} + +#else /* HAVE_CACHE_RECLAIM */ + +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to + * RECLAIMING_ANY (reclaiming any blocks). + * + * If at any time the size of the cache goes below the low threshold + * we move back to the NOT_RECL...

v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.

Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.

...h" +#include "reclaim.h" +#include "lru.h" + +#ifndef HAVE_CACHE_RECLAIM + +void +reclaim (int fd, struct bitmap *bm) +{ + /* nothing */ +} + +#else /* HAVE_CACHE_RECLAIM */ + +/* If we are currently reclaiming blocks from the cache. + * + * The state machine starts in the NOT_RECLAIMING state. When the + * size of the cache exceeds the high threshold, we move to + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to + * RECLAIMING_ANY (reclaiming any blocks). + * + * If at any time the size of the cache goes below the low threshold + * we move back to the NOT_RECL...

...el supports punching holes, there are some filesystems that don't). > @@ -74,6 +76,31 @@ enum bm_entry { > BLOCK_DIRTY = 3, > }; > > +#ifdef HAVE_CACHE_RECLAIM > +/* If we are currently reclaiming blocks from the cache. > + * > + * The state machine starts in the NOT_RECLAIMING state. When the > + * size of the cache exceeds the high threshold, we move to > + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to > + * RECLAIMING_ANY (reclaiming any blocks). > + * > + * If at any time the size of the cache goes below the low threshold > +...

These are essentially identical to what was previously posted as patches 6/9 through 9/9 here: https://www.redhat.com/archives/libguestfs/2018-December/msg00145.html except that it has been rebased onto the current git master and retested thoroughly. Rich.

...sub-hole regions that you can't reclaim anything? Or do you need the cache module to use a larger block size by default, or even add a parameter to control the block size? > + > +/* If we are currently reclaiming blocks from the cache. > + * > + * The state machine starts in the NOT_RECLAIMING state. When the > + * size of the cache exceeds the high threshold, we move to > + * RECLAIMING_LRU. Once we have exhausted all LRU blocks, we move to > + * RECLAIMING_ANY (reclaiming any blocks). > + * > + * If at any time the size of the cache goes below the low threshold > +...

This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I