Libguestfs - Jun 2020 - [PATCH nbdkit] v2v: Disable readahead for VMware curl sources too (RHBZ#1848862).

I'm still testing this fix, so let's hold off the review for the
moment.

Also it may be better to specifically identify problematic servers
rather than disabling this for every curl source.  eg.  I suspect that
the problem is the Java server used by VCenter, so we might think
about only disabling readahead for that single case.

Rich.

This appears to be the cause of timeouts during the conversion step
where VMware VCenter server's Tomcat HTTPS server stops responding to
requests (or rather, responds only with 503 errors).  The server later
recovers and in fact because of the retry filter the conversion
usually succeeds, but I found that we can avoid the problem by
disabling readahead.
---
 v2v/nbdkit_sources.ml | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/v2v/nbdkit_sources.ml b/v2v/nbdkit_sources.ml
index f5e919116..7c177e358 100644
--- a/v2v/nbdkit_sources.ml
+++ b/v2v/nbdkit_sources.ml
@@ -99,12 +99,12 @@ let common_create ?bandwidth ?extra_debug ?extra_env
password
 
   (* Adding the readahead filter is always a win for our access
    * patterns.  If it doesn't exist don't worry.  However it
-   * breaks VMware servers (RHBZ#1832805).
+   * breaks VMware servers (RHBZ#1832805, RHBZ#1848862).
    *)
   let cmd -    if plugin_name <> "vddk" then
-      Nbdkit.add_filter_if_available cmd "readahead"
-    else cmd in
+    match plugin_name with
+    | "vddk" | "curl" -> cmd
+    | _ -> Nbdkit.add_filter_if_available cmd "readahead" in
 
   (* Caching extents speeds up qemu-img, especially its consecutive
    * block_status requests with req_one=1.
-- 
2.25.0

On 6/19/20 7:47 AM, Richard W.M. Jones wrote:
> This appears to be the cause of timeouts during the conversion step
> where VMware VCenter server's Tomcat HTTPS server stops responding to
> requests (or rather, responds only with 503 errors).  The server later
> recovers and in fact because of the retry filter the conversion
> usually succeeds, but I found that we can avoid the problem by
> disabling readahead.
> ---
>   v2v/nbdkit_sources.ml | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)
> 
Is it just a matter of readahead requesting too much data at once?  Is 
it time to start revisiting the idea of moving the core of the 
'blocksize' filter into the server itself, in order to make it easier 
for various plugins and filters to request min/max block limits, and 
where the common server code then stitches together read-modify-write or 
splits requests as needed to the next layer?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org