search for: blk_writethrough

...backend behavior by insisting on the opaque parameter having or not having any particular value, rather than just blindly passing it on through and letting the backend do the enforcing. I would use just __nonnull__ (1, 4, 5). > > /* Write to the cache and the plugin. */ > -extern int blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, uint64_t blknum, const uint8_t *block, uint32_t flags, int *err); > +extern int blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, > + uint64_t blknum, const uint8_t *block, > +...

This patch series enhances the cache filter in a few ways, primarily adding a "cache-on-read" feature (similar to qemu's copyonread); and adding the ability to limit the cache size and the antecedent of that which is having a method to reclaim cache blocks. As the cache is stored as a sparse temporary file, reclaiming cache blocks simply means punching holes in the temporary file.

v1 was here: https://www.redhat.com/archives/libguestfs/2019-January/msg00008.html In v2 I have provided two patches: The first patch extends attribute((nonnull)) to most internal functions, but not to the external API. The second patch uses a macro so that attribute((format)) is only used in the public API on GCC or Clang. At least in theory these headers could be used by a C compiler which

....errno_is_preserved = plugin_errno_is_preserved, .open = plugin_open, .prepare = plugin_prepare, .finalize = plugin_finalize, diff --git a/filters/cache/cache.c b/filters/cache/cache.c index 9473f2c..2ae6f01 100644 --- a/filters/cache/cache.c +++ b/filters/cache/cache.c @@ -292,6 +292,7 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, uint64_t blknum, const uint8_t *block) { off_t offset = blknum * BLKSIZE; + int r; nbdkit_debug ("cache: blk_writethrough block %" PRIu64 " (offset %" PRIu64 ")", @@ -302,8 +3...

This adds a cache filter, which works like the COW filter in reverse. For realistic use it needs a bit more work, especially to add limits on the size of the cache, a more sensible cache replacement policy, and perhaps some kind of background worker to write dirty blocks out. Rich.

..._entry state = blk_get_bitmap_entry (blknum); + enum bm_entry state = bitmap_get_blk (&bm, blknum, BLOCK_NOT_CACHED); nbdkit_debug ("cache: blk_read block %" PRIu64 " (offset %" PRIu64 ") is %s", blknum, (uint64_t) offset, @@ -326,7 +273,7 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, if (next_ops->pwrite (nxdata, block, BLKSIZE, offset, flags, err) == -1) return -1; - blk_set_bitmap_entry (blknum, BLOCK_CLEAN); + bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); return 0; } @@ -354,7 +301,7 @@ blk_writeback (str...

..._entry state = blk_get_bitmap_entry (blknum); + enum bm_entry state = bitmap_get_blk (&bm, blknum, BLOCK_NOT_CACHED); nbdkit_debug ("cache: blk_read block %" PRIu64 " (offset %" PRIu64 ") is %s", blknum, (uint64_t) offset, @@ -326,7 +275,7 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, if (next_ops->pwrite (nxdata, block, BLKSIZE, offset, flags, err) == -1) return -1; - blk_set_bitmap_entry (blknum, BLOCK_CLEAN); + bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); return 0; } @@ -354,7 +303,7 @@ blk_writeback (str...

..._entry state = blk_get_bitmap_entry (blknum); + enum bm_entry state = bitmap_get_blk (&bm, blknum, BLOCK_NOT_CACHED); nbdkit_debug ("cache: blk_read block %" PRIu64 " (offset %" PRIu64 ") is %s", blknum, (uint64_t) offset, @@ -326,7 +275,7 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, if (next_ops->pwrite (nxdata, block, BLKSIZE, offset, flags, err) == -1) return -1; - blk_set_bitmap_entry (blknum, BLOCK_CLEAN); + bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); return 0; } @@ -354,7 +303,7 @@ blk_writeback (str...

I have some patches I'm working on to fix the cache filter. However this is a prelude. It should be simply pure refactoring. All tests pass still. Rich.

...ops, void *nxdata, uint64_t blknum, uint8_t *block, int *err); +extern int blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, + uint64_t blknum, uint8_t *block, int *err) + __attribute__((__nonnull__ (1, 2, 4, 5))); /* Write to the cache and the plugin. */ -extern int blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, uint64_t blknum, const uint8_t *block, uint32_t flags, int *err); +extern int blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, + uint64_t blknum, const uint8_t *block, + uint32_...

v2: https://www.redhat.com/archives/libguestfs/2018-December/msg00039.html v2 -> v3: - Fix all the issues raised in Eric's review. - Precompute some numbers to make the calculations easier. - Calculations now use bitshifts and masks in preference to division and modulo. - Clear existing bits before setting (which fixes a bug in the cache filter). Rich.

This is exactly the same as v1: https://www.redhat.com/archives/libguestfs/2018-December/msg00004.html except that it now frees the bitmap on unload (which the old code did not - there was always a memory leak). Rich.

Here's what I'm currently playing with; I'm not ready to commit anything until I rebase my FUA work on top of this, as I only want to break filter ABI once between releases. Eric Blake (2): backend: Rework internal/filter error return semantics filters: Add log filter TODO | 2 - docs/nbdkit-filter.pod | 84 +++++++-- docs/nbdkit.pod

...lru_set_recently_accessed (blknum); } return 0; } @@ -172,6 +181,7 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_error ("pread: %m"); return -1; } + lru_set_recently_accessed (blknum); return 0; } } @@ -196,6 +206,7 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); + lru_set_recently_accessed (blknum); return 0; } @@ -222,6 +233,7 @@ blk_write (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; } bitmap_set_blk (&b...

...lru_set_recently_accessed (blknum); } return 0; } @@ -172,6 +181,7 @@ blk_read (struct nbdkit_next_ops *next_ops, void *nxdata, nbdkit_error ("pread: %m"); return -1; } + lru_set_recently_accessed (blknum); return 0; } } @@ -196,6 +206,7 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; bitmap_set_blk (&bm, blknum, BLOCK_CLEAN); + lru_set_recently_accessed (blknum); return 0; } @@ -222,6 +233,7 @@ blk_write (struct nbdkit_next_ops *next_ops, void *nxdata, return -1; } bitmap_set_blk (&b...

Patch 1 is the same as last time, except for a minor comment fix. Patch 2 should address everything that Eric mentioned in his review, and has been retested. Rich.

...s, void *nxdata, return 0; } else { /* Read cache. */ - if (pread (fd, block, BLKSIZE, offset) == -1) { + if (pread (fd, block, blksize, offset) == -1) { *err = errno; nbdkit_error ("pread: %m"); return -1; @@ -191,18 +211,20 @@ blk_writethrough (struct nbdkit_next_ops *next_ops, void *nxdata, uint64_t blknum, const uint8_t *block, uint32_t flags, int *err) { - off_t offset = blknum * BLKSIZE; + off_t offset = blknum * blksize; + + reclaim (fd, &bm); nbdkit_debug ("cache: writethrough...

v4: https://www.redhat.com/archives/libguestfs/2019-January/msg00032.html v5: - Now we set the block size at run time. I'd like to say that I was able to test this change, but unfortunately I couldn't find any easy way to create a filesystem on x86-64 with a block size > 4K. Ext4 doesn't support it at all, and XFS doesn't support block size > page size (and I

Since v1: add the blocksize filter, add testsuite coverage of the log filter, several fixes to the log filter based on what adding tests revealed I'm still working on FUA flag support patches on top of this; the patches should all be committed in the same release, as we want to minimize the number of releases that cause a filter ABI/API bump Eric Blake (3): backend: Rework internal/filter

v3 was broken by a bad rebase, so let's forget about that one. Compared to v2: - Patch 1 is the same except for a minor comment change. - Patch 2 splits the reclaim code into a separate file (filters/cache/reclaim.c) - Addressed Eric's comments from his review of v2. - Retested on Linux and FreeBSD.