similar to: [PATCH v2v v2] nbdkit: Use cacheextents if possible for vddk input

The tests assume probe_filter returns true if the filter is available (and not the other way around). Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com> --- v2v/nbdkit.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2v/nbdkit.ml b/v2v/nbdkit.ml index 77d2a506..00122bec 100644 --- a/v2v/nbdkit.ml +++ b/v2v/nbdkit.ml @@ -142,7 +142,7 @@ let common_create

It does not need configuring and speeds up the process of requesting extents if the client asks for them one by one (like qemu-img). Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- v2v/input_libvirt_vddk.ml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/v2v/input_libvirt_vddk.ml b/v2v/input_libvirt_vddk.ml index a8e4fd5ac78e..cc63f77ce825 100644 ---

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_on=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map` runs

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_on=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map` runs

This filter caches the last result of the extents() call and offers a nice speed-up for clients that only support req_one=1 in combination with plugins like vddk, which has no overhead for returning information for multiple extents in one call, but that call is very time-consuming. Quick test showed that on a fast connection and a sparsely allocated 16G disk with a OS installed `qemu-img map`

On Thu, May 16, 2019 at 08:05:18AM -0500, Eric Blake wrote: >On 5/15/19 5:39 PM, Martin Kletzander wrote: >> This filter caches the last result of the extents() call and offers a nice >> speed-up for clients that only support req_on=1 in combination with plugins like > >s/on/one/ > >> vddk, which has no overhead for returning information for multiple extents in

The readahead filter is a self-configuring filter that makes sequential reads faster when the plugin is slow (and all of the plugins we use here are always slow). I observed the behaviour of the readahead filter with our qcow2 overlay when converting a guest from a vCenter source. Even when doing random reads, qemu issues 64K reads which happen to also be the minimum request size of the

On Fri, Sep 20, 2019 at 10:28:18AM +0100, Richard W.M. Jones wrote: >The readahead filter is a self-configuring filter that makes >sequential reads faster when the plugin is slow (and all of the >plugins we use here are always slow). > >I observed the behaviour of the readahead filter with our qcow2 >overlay when converting a guest from a vCenter source. Even when >doing

Apply the attached patch on top of yours which contains miscellaneous fixes. If you use ‘./configure --enable-gcc-warnings’ then it will enable GCC warnings. The test doesn't really test anything. If extents are being cached then I have two ideas about how you could see that: Either (1) you could look at debug messages, the second extents call shouldn't be processed at the plugin layer.

On Thu, May 16, 2019 at 08:42:43AM +0100, Richard W.M. Jones wrote: > >Apply the attached patch on top of yours which contains miscellaneous >fixes. If you use ‘./configure --enable-gcc-warnings’ then it will >enable GCC warnings. > Thanks. Again, too much used to libvirt codebase where those are being enabled by default for builds from git. >The test doesn't really test

On 5/15/19 5:39 PM, Martin Kletzander wrote: > This filter caches the last result of the extents() call and offers a nice > speed-up for clients that only support req_on=1 in combination with plugins like s/on/one/ > vddk, which has no overhead for returning information for multiple extents in > one call, but that call is very time-consuming. > > Quick test showed that on a

On 6/11/19 3:49 AM, Martin Kletzander wrote: > This filter caches the last result of the extents() call and offers a nice > speed-up for clients that only support req_one=1 in combination with plugins > like vddk, which has no overhead for returning information for multiple extents > in one call, but that call is very time-consuming. > > Quick test showed that on a fast

On Wed, May 15, 2019 at 02:54:17PM +0200, Martin Kletzander wrote: > This filter caches the last result of the extents() call and offers a nice > speed-up for clients that only support req_on=1 in combination with plugins like > vddk, which has no overhead for returning information for multiple extents in > one call, but that call is very time-consuming. > > Quick test showed

On 5/20/19 8:23 AM, Martin Kletzander wrote: >>> +  if (cache_extents && >>> +      offset >= cache_start && offset < cache_end) { >>> +    nbdkit_debug ("cacheextents: returning from cache"); >>> +    return cacheextents_add (extents, err); >>> +  } >>> + >>> +  nbdkit_debug ("cacheextents: cache

This refactoring takes the nbdkit-specific code from the ‘-it vddk’ mode and puts it into a separate module. The idea is to reuse this module (in future commits) to support ssh, curl and rate limiting. This refactoring is not quite neutral because it moves some of the prechecking into the Nbdkit.create function. This means it will happen a little bit later in the cycle (in input#source instead

On 9/20/19 9:04 AM, Martin Kletzander wrote: > On Fri, Sep 20, 2019 at 10:28:18AM +0100, Richard W.M. Jones wrote: >> The readahead filter is a self-configuring filter that makes >> sequential reads faster when the plugin is slow (and all of the >> plugins we use here are always slow). >> >> @@ -133,9 +142,17 @@ let common_create plugin_name plugin_args >>

This is built on top of the following patch series: https://www.redhat.com/archives/libguestfs/2019-April/msg00054.html Rich.

v3 posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00200.html v4: - The first patch in the v3 series was just a trivial doc whitespace fix so I pushed it. - There's a new patch using the nbdkit-retry-filter. This is not actually upstream in nbdkit but we know enough about how it will work. - Rebased against master and reran the tests. Rich.

v2 was posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00115.html This also has links to earlier versions. v3: - The 01/11 patch in v2 included a bunch of unnecessary plus one necessary change to how input_password is passed around. I moved the necessary change into the final patch (implementing SSH password authentication) and dropped the rest. - The 01/11

Originally posted here: https://www.redhat.com/archives/libguestfs/2019-April/thread.html#00054 https://www.redhat.com/archives/libguestfs/2019-April/msg00076.html https://www.redhat.com/archives/libguestfs/2019-April/msg00126.html This is a rebase on top of current master branch with no other changes. The first patch in the old series was pushed a while back, and the last "TEMPORARY"