search for: bitmap_set_blk

...mp;bm); + + lru_free (); } int @@ -128,6 +133,9 @@ blk_set_size (uint64_t new_size) return -1; } + if (lru_set_size (new_size) == -1) + return -1; + return 0; } @@ -163,6 +171,7 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; } bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); + lru_set_recently_accessed (blknum); } return 0; } @@ -172,6 +181,7 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_error ("pread: %m"); return -1; } + lru_set_recently_accessed (blknum); ret...

...mp;bm); + + lru_free (); } int @@ -128,6 +133,9 @@ blk_set_size (uint64_t new_size) return -1; } + if (lru_set_size (new_size) == -1) + return -1; + return 0; } @@ -163,6 +171,7 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; } bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); + lru_set_recently_accessed (blknum); } return 0; } @@ -172,6 +181,7 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_error ("pread: %m"); return -1; } + lru_set_recently_accessed (blknum); ret...

This is exactly the same as v1: https://www.redhat.com/archives/libguestfs/2018-December/msg00004.html except that it now frees the bitmap on unload (which the old code did not - there was always a memory leak). Rich.

...in bytes. */ +static inline unsigned +bitmap_get (const struct bitmap *bm, uint64_t offset, unsigned default_) +{ + return bitmap_get_blk (bm, offset / bm->blksize, default_); +} + +/* Set the bit(s) associated with the given block. + * If out of range, it is ignored. + */ +static inline void +bitmap_set_blk (const struct bitmap *bm, uint64_t blk, unsigned v) +{ + uint64_t blk_offset = blk / (8 / bm->bpb); + unsigned blk_bit = bm->bpb * (blk % (8 / bm->bpb)); + + if (blk_offset >= bm->size) { + nbdkit_debug ("bitmap_set: block number is out of range"); + return; + } +...

...in bytes. */ +static inline unsigned +bitmap_get (const struct bitmap *bm, uint64_t offset, unsigned default_) +{ + return bitmap_get_blk (bm, offset / bm->blksize, default_); +} + +/* Set the bit(s) associated with the given block. + * If out of range, it is ignored. + */ +static inline void +bitmap_set_blk (const struct bitmap *bm, uint64_t blk, unsigned v) +{ + uint64_t blk_offset = blk / (8 / bm->bpb); + unsigned blk_bit = bm->bpb * (blk % (8 / bm->bpb)); + + if (blk_offset >= bm->size) { + nbdkit_debug ("bitmap_set: block number is out of range"); + return; + } +...

...in bytes. */ +static inline unsigned +bitmap_get (const struct bitmap *bm, uint64_t offset, unsigned default_) +{ + return bitmap_get_blk (bm, offset / bm->blksize, default_); +} + +/* Set the bit(s) associated with the given block. + * If out of range, it is ignored. + */ +static inline void +bitmap_set_blk (const struct bitmap *bm, uint64_t blk, unsigned v) +{ + BITMAP_OFFSET_BIT_MASK (bm, blk); + + if (blk_offset >= bm->size) { + nbdkit_debug ("bitmap_set: block number is out of range"); + return; + } + + bm->bitmap[blk_offset] &= ~mask; + bm->bitmap[blk_offset]...

I have some patches I'm working on to fix the cache filter. However this is a prelude. It should be simply pure refactoring. All tests pass still. Rich.

v2: https://www.redhat.com/archives/libguestfs/2018-December/msg00039.html v2 -> v3: - Fix all the issues raised in Eric's review. - Precompute some numbers to make the calculations easier. - Calculations now use bitshifts and masks in preference to division and modulo. - Clear existing bits before setting (which fixes a bug in the cache filter). Rich.

...;< and >> instead of / and % aid the compiler, since we know we have powers of 2 based on initialization, but the compiler might not see it locally? > +/* Set the bit(s) associated with the given block. > + * If out of range, it is ignored. > + */ > +static inline void > +bitmap_set_blk (const struct bitmap *bm, uint64_t blk, unsigned v) > +{ > + uint64_t blk_offset = blk / (8 / bm->bpb); > + unsigned blk_bit = bm->bpb * (blk % (8 / bm->bpb)); > + > + if (blk_offset >= bm->size) { > + nbdkit_debug ("bitmap_set: block number is out of ran...

Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.

This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.

Should we use attribute((nonnull)) at all? There's a very interesting history of this in libvirt -- try looking at commit eefb881 plus the commits referencing eefb881 -- but it does seem to work for me using recent GCC and Clang. I only did a few functions because annotating them gets old quickly... Rich.

These are essentially identical to what was previously posted as patches 6/9 through 9/9 here: https://www.redhat.com/archives/libguestfs/2018-December/msg00145.html except that it has been rebased onto the current git master and retested thoroughly. Rich.

v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.

...bm, offset / bm->blksize, default_); @@ -148,7 +149,7 @@ bitmap_get (const struct bitmap *bm, uint64_t offset, unsigned default_) /* Set the bit(s) associated with the given block. * If out of range, it is ignored. */ -static inline void +static inline void __attribute__((__nonnull__ (1))) bitmap_set_blk (const struct bitmap *bm, uint64_t blk, unsigned v) { BITMAP_OFFSET_BIT_MASK (bm, blk); @@ -163,7 +164,7 @@ bitmap_set_blk (const struct bitmap *bm, uint64_t blk, unsigned v) } /* As above bit works with virtual disk offset in bytes. */ -static inline void +static inline void __attribute__(...

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which

...+ /* If the block is already set in the first bitmap, don't need to do > + * anything. > + */ > + if (bitmap_get_blk (&bm[0], blknum, false)) > + return; And this implements the latter (the most recent N blocks accessed, not the most recent N accesses). > + > + bitmap_set_blk (&bm[0], blknum, true); > + c0++; > + > + /* If we've reached N/2 then we need to swap over the bitmaps. */ > + if (c0 >= N/2) { > + struct bitmap tmp; > + > + tmp = bm[0]; > + bm[0] = bm[1]; > + bm[1] = tmp; > + c1 = c0; > + > + b...

...lock %" PRIu64 + " (offset %" PRIu64 ")", + blknum, (uint64_t) offset); + + if (pwrite (fd, block, BLKSIZE, offset) == -1) { + *err = errno; + nbdkit_error ("pwrite: %m"); + return -1; + } + bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); + } + return 0; + } + else { /* Read cache. */ if (pread (fd, block, BLKSIZE, offset) == -1) { *err = errno; nbdkit_error ("pread: %m"); diff --git a/tests/Makefile.am b/tests/Makefile.am index 79b723a..ccd5...

..., block, blksize, offset) == -1) { *err = errno; nbdkit_error ("pwrite: %m"); return -1; } - if (next_ops->pwrite (nxdata, block, BLKSIZE, offset, flags, err) == -1) + if (next_ops->pwrite (nxdata, block, blksize, offset, flags, err) == -1) return -1; bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); @@ -222,12 +244,14 @@ blk_write (struct nbdkit_next_ops *next_ops, void *nxdata, (cache_mode == CACHE_MODE_WRITEBACK && (flags & NBDKIT_FLAG_FUA))) return blk_writethrough (next_ops, nxdata, blknum, block, flags, err); - offset = blknum * B...