search for: cacheext

...le overhead. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- v2: - Errors are set in _add and _fill functions - Return values are checked for only -1 - Added a test - Indentation fixed - Fixed access before lock configure.ac | 2 + filters/cacheextents/Makefile.am | 64 ++++++ filters/cacheextents/cacheextents.c | 196 ++++++++++++++++++ .../nbdkit-cacheextents-filter.pod | 47 +++++ tests/Makefile.am | 5 +- tests/test-cacheextents.sh | 56 +++++ 6 files ch...

...lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://people.redhat.com/~rjones/virt-df/ --xXmbgvnjoT4axfJE Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=patch diff --git a/filters/cacheextents/cacheextents.c b/filters/cacheextents/cacheextents.c index 01c0328..6cd22a7 100644 --- a/filters/cacheextents/cacheextents.c +++ b/filters/cacheextents/cacheextents.c @@ -50,9 +50,6 @@ /* This lock protects the global state. */ static pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; -/* Th...

...request and not the second >one (and even test that a subsequent write should kill the cache so >another extent request is seen by the plugin). > Good point with the write. The test checks that there is only one request to the plugin for extents before the log filter is applied after the cacheextents filter. If I remove the cacheextents one, the log will contain multiple requests for extents for which the grep will return 4 and the test will fail. Anyway for the write the sh plugin will help, so I'll convert it to that. >Rich. > >-- >Richard Jones, Virtualization Group,...

...s no space for improvement) this does not add any noticeable overhead. Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- Yes, I hate the filter name, so if anyone has a better idea, feel free to suggest (or rename) it. configure.ac | 2 + filters/cacheextents/Makefile.am | 64 ++++++ filters/cacheextents/cacheextents.c | 200 ++++++++++++++++++ .../nbdkit-cacheextents-filter.pod | 47 ++++ 4 files changed, 313 insertions(+) create mode 100644 filters/cacheextents/Makefile.am create mode 100644 filters/cacheexten...

...hate the wording of the comment explaining it, please suggest a reword (or just just fix it) - Tests are now using custom shell plugin logging all accesses and also qemu-io testing all other expected behaviour configure.ac | 2 + filters/cacheextents/Makefile.am | 64 ++++++ filters/cacheextents/cacheextents.c | 196 ++++++++++++++++++ .../nbdkit-cacheextents-filter.pod | 47 +++++ tests/Makefile.am | 4 + tests/test-cacheextents.sh | 110 ++++++++++ 6 file...

...e wording of the comment explaining it, please suggest a reword (or > just just fix it) I'll give it a shot. > > - Tests are now using custom shell plugin logging all accesses and also qemu-io > testing all other expected behaviour > > +++ b/filters/cacheextents/cacheextents.c > + > +/* Cached extents from the last extents () call and its start and end for the > + sake of simplicity. */ The prevailing style seems to be the use of leading ' *' on subsequent comment lines, and */ on its own line. > +struct nbdkit_extents *cache_e...

...st extents request). For 100G disk with no data on it, that > is one hole extent spanning the whole disk (where there is no space for > improvement) this does not add any noticeable overhead. > > Signed-off-by: Martin Kletzander <mkletzan@redhat.com> > --- > +++ b/filters/cacheextents/cacheextents.c > + > +static int > +cacheextents_extents (struct nbdkit_next_ops *next_ops, void *nxdata, > + void *handle, uint32_t count, uint64_t offset, uint32_t flags, > + struct nbdkit_extents *extents, > +...

...l index a8e4fd5ac78e..cc63f77ce825 100644 --- a/v2v/input_libvirt_vddk.ml +++ b/v2v/input_libvirt_vddk.ml @@ -284,6 +284,9 @@ object (self) add_arg "--selinux-label"; add_arg "system_u:object_r:svirt_socket_t:s0" ); + if shell_command "nbdkit --filter cacheextents memory 1M --run true 2>/dev/null" = 0 + then add_arg "--filter=cacheextents"; + (* Name of the plugin. Everything following is a plugin parameter. *) add_arg "vddk"; -- 2.23.0

...644 --- a/v2v/nbdkit.ml +++ b/v2v/nbdkit.ml @@ -169,6 +169,13 @@ let common_create ?bandwidth plugin_name plugin_args plugin_env = ) else [] in + (* Caching extents speeds up qemu-img, especially its consecutive + block_status requests with req_one=1. + *) + if probe_filter "cacheextents" then ( + add_arg "--filter"; add_arg "cacheextents" + ); + (* Retry filter (if it exists) can be used to get around brief * interruptions in service. It must be closest to the plugin. *) -- 2.24.0

...00G disk with no data on it, that >> is one hole extent spanning the whole disk (where there is no space for >> improvement) this does not add any noticeable overhead. >> >> Signed-off-by: Martin Kletzander <mkletzan@redhat.com> >> --- > >> +++ b/filters/cacheextents/cacheextents.c > > >> + >> +static int >> +cacheextents_extents (struct nbdkit_next_ops *next_ops, void *nxdata, >> + void *handle, uint32_t count, uint64_t offset, uint32_t flags, >> + struct nbdkit_extents *extents,...

...om> > --- > Yes, I hate the filter name, so if anyone has a better idea, feel free to > suggest (or rename) it. Basically it's fine, however it could really do with having a test. I have no particular comment about the name. A few more things inline below. > +static int > +cacheextents_add (struct nbdkit_extents *extents) > +{ > + size_t i = 0; > + > + for (i = 0; i < nbdkit_extents_count (cache_extents); i++) { > + struct nbdkit_extent ex = nbdkit_get_extent (cache_extents, i); > + if (nbdkit_add_extent (extents, ex.offset, ex.length, ex.type) &l...

On 5/20/19 8:23 AM, Martin Kletzander wrote: >>> +  if (cache_extents && >>> +      offset >= cache_start && offset < cache_end) { >>> +    nbdkit_debug ("cacheextents: returning from cache"); >>> +    return cacheextents_add (extents, err); >>> +  } >>> + >>> +  nbdkit_debug ("cacheextents: cache miss"); >>> +  int r = next_ops->extents (nxdata, count, offset, flags, extents, >>> err); &...

On 5/15/19 5:39 PM, Martin Kletzander wrote: > This filter caches the last result of the extents() call and offers a nice > speed-up for clients that only support req_on=1 in combination with plugins like > vddk, which has no overhead for returning information for multiple extents in > one call, but that call is very time-consuming. > > Quick test showed that on a fast

On 6/11/19 3:49 AM, Martin Kletzander wrote: > This filter caches the last result of the extents() call and offers a nice > speed-up for clients that only support req_one=1 in combination with plugins > like vddk, which has no overhead for returning information for multiple extents > in one call, but that call is very time-consuming. > > Quick test showed that on a fast

...ed bytes in ._api_version). docs/nbdkit-filter.pod | 7 ------- server/filters.c | 20 +++++++++++++++----- include/nbdkit-filter.h | 18 ++++++++++-------- filters/blocksize/blocksize.c | 1 - filters/cache/cache.c | 1 - filters/cacheextents/cacheextents.c | 1 - filters/cow/cow.c | 1 - filters/delay/delay.c | 1 - filters/error/error.c | 1 - filters/fua/fua.c | 1 - filters/log/log.c | 1 - filters/nocache/nocache.c | 1 - filters/...

....com> --- docs/nbdkit-loop.pod | 2 +- docs/nbdkit.pod | 4 ++-- plugins/eval/nbdkit-eval-plugin.pod | 6 +++--- plugins/sh/nbdkit-sh-plugin.pod | 2 +- plugins/sh/assemble.sh | 2 +- plugins/sh/example.sh | 7 +++---- tests/test-cacheextents.sh | 5 ++--- tests/test-eval-file.sh | 5 ++--- tests/test-eval.sh | 3 +-- tests/test-export-name.sh | 5 ++--- tests/test-parallel-sh.sh | 5 ++--- tests/test-readahead-test-plugin.sh | 2 +- tests/test-readahead.sh | 1 -...

...} + else { + free (bm->bitmap); + new_bitmap = NULL; } bm->bitmap = new_bitmap; bm->size = new_bm_size; diff --git a/tests/Makefile.am b/tests/Makefile.am index f54597b..9eec75e 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -50,6 +50,7 @@ EXTRA_DIST = \ test-cacheextents.sh \ test-captive.sh \ test-cow.sh \ + test-cow-null.sh \ test-cxx.sh \ test-data-7E.sh \ test-data-base64.sh \ @@ -960,6 +961,7 @@ TESTS += test-cacheextents.sh if HAVE_GUESTFISH TESTS += test-cow.sh endif HAVE_GUESTFISH +TESTS += test-cow-null.sh # delay filter tests. TESTS +...

--- configure.ac | 2 ++ filters/blocksize/Makefile.am | 1 + filters/cache/Makefile.am | 1 + filters/cacheextents/Makefile.am | 1 + filters/cow/Makefile.am | 1 + filters/ddrescue/Makefile.am | 1 + filters/delay/Makefile.am | 3 +++ filters/error/Makefile.am | 1 + filters/exitlast/Makefile.am | 3 +++ filters/ext2/Makefile.am | 3 ++- filters/extentlist/Make...

v1 was here: https://www.redhat.com/archives/libguestfs/2019-September/msg00100.html In v2 I've changed the patch so it avoids calling realloc at all in this case. The patch is a bit longer this way. But I don't see any other alternative if we are to avoid having a "realloc wrapper" of some kind that we use everywhere, which I guess we should avoid because it makes plugins

...cue/Makefile.am create mode 100644 filters/ddrescue/ddrescue.c create mode 100644 filters/ddrescue/nbdkit-ddrescue-filter.pod diff --git a/configure.ac b/configure.ac index 6c25226e..21e1013f 100644 --- a/configure.ac +++ b/configure.ac @@ -97,6 +97,7 @@ filters="\ cache \ cacheextents \ cow \ + ddrescue \ delay \ error \ exitlast \ @@ -1089,6 +1090,7 @@ AC_CONFIG_FILES([Makefile filters/cache/Makefile filters/cacheextents/Makefile filters/cow/Makefile + filters/dd...