search for: bitmap_init

...46 100644 --- a/filters/cache/blk.c +++ b/filters/cache/blk.c @@ -53,6 +53,7 @@ #include "cache.h" #include "blk.h" +#include "lru.h" /* The cache. */ static int fd = -1; @@ -80,6 +81,8 @@ blk_init (void) size_t len; char *template; + lru_init (); + bitmap_init (&bm, BLKSIZE, 2 /* bits per block */); tmpdir = getenv ("TMPDIR"); @@ -115,6 +118,8 @@ blk_free (void) close (fd); bitmap_free (&bm); + + lru_free (); } int @@ -128,6 +133,9 @@ blk_set_size (uint64_t new_size) return -1; } + if (lru_set_size (new_si...

...46 100644 --- a/filters/cache/blk.c +++ b/filters/cache/blk.c @@ -53,6 +53,7 @@ #include "cache.h" #include "blk.h" +#include "lru.h" /* The cache. */ static int fd = -1; @@ -80,6 +81,8 @@ blk_init (void) size_t len; char *template; + lru_init (); + bitmap_init (&bm, BLKSIZE, 2 /* bits per block */); tmpdir = getenv ("TMPDIR"); @@ -115,6 +118,8 @@ blk_free (void) close (fd); bitmap_free (&bm); + + lru_free (); } int @@ -128,6 +133,9 @@ blk_set_size (uint64_t new_size) return -1; } + if (lru_set_size (new_si...

...-1; static struct bitmap bm; enum bm_entry { - BLOCK_NOT_CACHED = 0, + BLOCK_NOT_CACHED = 0, /* assumed to be zero by reclaim code */ BLOCK_CLEAN = 1, BLOCK_DIRTY = 3, }; @@ -80,10 +83,7 @@ blk_init (void) const char *tmpdir; size_t len; char *template; - - lru_init (); - - bitmap_init (&bm, BLKSIZE, 2 /* bits per block */); + struct statvfs statvfs; tmpdir = getenv ("TMPDIR"); if (!tmpdir) @@ -108,6 +108,24 @@ blk_init (void) unlink (template); + /* Choose the block size. + * + * A 4K block size means that we need 64 MB of memory to store the +...

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I

This is exactly the same as v1: https://www.redhat.com/archives/libguestfs/2018-December/msg00004.html except that it now frees the bitmap on unload (which the old code did not - there was always a memory leak). Rich.

...structure. */ +struct bitmap { + unsigned blksize; /* Block size. */ + unsigned bpb; /* Bits per block (1, 2, 4, 8 only). */ + + uint8_t *bitmap; /* The bitmap. */ + size_t size; /* Size of bitmap in bytes. */ +}; + +static inline void +bitmap_init (struct bitmap *bm, unsigned blksize, unsigned bpb) +{ + assert (is_power_of_2 (blksize)); + assert (bpb >= 1); + assert (bpb <= 8); + assert (is_power_of_2 (bpb)); /* Only 1, 2, 4, 8 allowed. */ + + bm->blksize = blksize; + bm->bpb = bpb; + + bm->bitmap = NULL; + bm->size...

...structure. */ +struct bitmap { + unsigned blksize; /* Block size. */ + unsigned bpb; /* Bits per block (1, 2, 4, 8 only). */ + + uint8_t *bitmap; /* The bitmap. */ + size_t size; /* Size of bitmap in bytes. */ +}; + +static inline void +bitmap_init (struct bitmap *bm, unsigned blksize, unsigned bpb) +{ + assert (is_power_of_2 (blksize)); + assert (bpb >= 1); + assert (bpb <= 8); + assert (is_power_of_2 (bpb)); /* Only 1, 2, 4, 8 allowed. */ + + bm->blksize = blksize; + bm->bpb = bpb; + + bm->bitmap = NULL; + bm->size...

...b + 1 0 8 + 2 1 4 + 4 2 2 + 8 3 1 + */ + unsigned bitshift, ibpb; + + uint8_t *bitmap; /* The bitmap. */ + size_t size; /* Size of bitmap in bytes. */ +}; + +static inline void +bitmap_init (struct bitmap *bm, unsigned blksize, unsigned bpb) +{ + assert (is_power_of_2 (blksize)); + bm->blksize = blksize; + + /* bpb can be 1, 2, 4 or 8 only. */ + bm->bpb = bpb; + bm->ibpb = 8/bpb; + switch (bpb) { + case 1: bm->bitshift = 0; break; + case 2: bm->bitshift = 1; bre...

I have some patches I'm working on to fix the cache filter. However this is a prelude. It should be simply pure refactoring. All tests pass still. Rich.

v2: https://www.redhat.com/archives/libguestfs/2018-December/msg00039.html v2 -> v3: - Fix all the issues raised in Eric's review. - Precompute some numbers to make the calculations easier. - Calculations now use bitshifts and masks in preference to division and modulo. - Clear existing bits before setting (which fixes a bug in the cache filter). Rich.

Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.

...ned blksize; /* Block size. */ > + unsigned bpb; /* Bits per block (1, 2, 4, 8 only). */ > + > + uint8_t *bitmap; /* The bitmap. */ > + size_t size; /* Size of bitmap in bytes. */ > +}; > + > +static inline void > +bitmap_init (struct bitmap *bm, unsigned blksize, unsigned bpb) > +{ > + assert (is_power_of_2 (blksize)); > + assert (bpb >= 1); > + assert (bpb <= 8); > + assert (is_power_of_2 (bpb)); /* Only 1, 2, 4, 8 allowed. */ > + > + bm->blksize = blksize; No validation that blksize...

While working on can_FOO caching, at one point I got confused by whether 'readonly' meant the global set by -r or a local passed to .open(). A quick attempt to compile with -Wshadow found several other confusing points; this series gets us to the point that we can add -Wshadow to builds with --enable-gcc-warnings. Eric Blake (10): server: Avoid -Wshadow warnings guestfs: Avoid

v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.

These are essentially identical to what was previously posted as patches 6/9 through 9/9 here: https://www.redhat.com/archives/libguestfs/2018-December/msg00145.html except that it has been rebased onto the current git master and retested thoroughly. Rich.

This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.

...nt blks[] = + { + 1, 2, 3, 10, 12, + 90, 91, 92, 93, 94, 99, + 800, 801, 802, 803, + 902, 903, 905, 907, 911, 913, 917, 919, 923, 929, + 999 + }; + unsigned v, vexp; + size_t i, j; + + printf ("bpb = %d, blksize = %d\n", bpb, blksize); + fflush (stdout); + + bitmap_init (&bm, blksize, bpb); + if (bitmap_resize (&bm, nr_blocks * blksize) == -1) + exit (EXIT_FAILURE); + + /* Set some bits at known block numbers. */ + for (j = 0; j < sizeof blks / sizeof blks[0]; ++j) { + v = (j & 1) == 0 ? 1 : (1<<bpb) - 1; + bitmap_set_blk (&bm,...

Should we use attribute((nonnull)) at all? There's a very interesting history of this in libvirt -- try looking at commit eefb881 plus the commits referencing eefb881 -- but it does seem to work for me using recent GCC and Clang. I only did a few functions because annotating them gets old quickly... Rich.

...bitmap.h b/common/bitmap/bitmap.h index 0da650a..553b9d2 100644 --- a/common/bitmap/bitmap.h +++ b/common/bitmap/bitmap.h @@ -65,7 +65,7 @@ struct bitmap { size_t size; /* Size of bitmap in bytes. */ }; -static inline void +static inline void __attribute__((__nonnull__ (1))) bitmap_init (struct bitmap *bm, unsigned blksize, unsigned bpb) { assert (is_power_of_2 (blksize)); @@ -99,10 +99,11 @@ bitmap_free (struct bitmap *bm) /* Resize the bitmap to the virtual disk size in bytes. * Returns -1 on error, setting nbdkit_error. */ -extern int bitmap_resize (struct bitmap *bm,...

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which