Libguestfs - Dec 2021 - [PATCH v2v] input: Disable multi-conn in VDDK input mode

I already pushed this commit, but I'm sending it to the list because
it reveals an interesting and unexpected side effect of
nbdkit-cow-filter (see commit message).

I wonder if we want to add a cow-multi-conn=false flag?

Rich.

The cow filter unconditionally enables multi-conn (because it is
safe).  However this causes an unintended consequence with the VDDK
plugin.  Multiple VDDK handles are opened (one per multi-conn
connection), and for some reason, possibly internal locking, they
conflict with each other.  This manifests itself as API calls taking
between 2 and 7 times longer to serve (especially QueryAllocatedBlocks
which seems to slow down most).

Avoid this by adding nbdkit-multi-conn-filter with
multi-conn-mode=disable on top which disables multi-conn
advertisement.

Virt-v2v -ic esx://... -it vddk -o null, before this change:

[   2.8] Opening the source
[  11.4] Inspecting the source
[  16.2] Checking for sufficient free disk space in the guest
[  16.2] Converting Windows 10 Enterprise to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  25.5] Mapping filesystem data to avoid copying unused and blank areas
[  67.3] Closing the overlay
[  68.0] Assigning disks to buses
[  68.0] Checking if the guest needs BIOS or UEFI to boot
[  68.0] Copying disk 1/1
? 100% [****************************************]
[ 416.4] Creating output metadata
[ 416.4] Finishing off

After this change:

[   2.8] Opening the source
[  12.6] Inspecting the source
[  18.7] Checking for sufficient free disk space in the guest
[  18.7] Converting Windows 10 Enterprise to run on KVM
virt-v2v: This guest has virtio drivers installed.
[  36.1] Mapping filesystem data to avoid copying unused and blank areas
[  77.0] Closing the overlay
[  77.9] Assigning disks to buses
[  77.9] Checking if the guest needs BIOS or UEFI to boot
[  77.9] Copying disk 1/1
? 100% [****************************************]
[ 332.3] Creating output metadata
[ 332.3] Finishing off
---
 input/nbdkit_vddk.ml | 22 +++++++++++++++++++++-
 1 file changed, 21 insertions(+), 1 deletion(-)

diff --git a/input/nbdkit_vddk.ml b/input/nbdkit_vddk.ml
index 79ec363bce..c2c3cceb93 100644
--- a/input/nbdkit_vddk.ml
+++ b/input/nbdkit_vddk.ml
@@ -157,10 +157,30 @@ See also the virt-v2v-input-vmware(1) manual.") libNN
   let cmd = Nbdkit.add_filter_if_available cmd "cacheextents" in
 
   (* IMPORTANT! Add the COW filter.  It must be furthest away
-   * except for the rate filter.
+   * except for the multi-conn and rate filters.
    *)
   let cmd = Nbdkit.add_filter cmd "cow" in
 
+  (* The cow filter unconditionally enables multi-conn (because it is
+   * safe).  However this causes an unintended consequence with the VDDK
+   * plugin.  Multiple VDDK handles are opened (one per multi-conn
+   * connection), and for some reason, possibly internal locking, they
+   * conflict with each other.  This manifests itself as API calls taking
+   * between 2 and 7 times longer to serve (especially QueryAllocatedBlocks
+   * which seems to slow down most).
+   *
+   * Avoid this by adding nbdkit-multi-conn-filter with
+   * multi-conn-mode=disable on top which disables multi-conn
+   * advertisement.
+   *)
+  let cmd +    if Nbdkit.probe_filter "multi-conn" then (
+      let cmd = Nbdkit.add_filter cmd "multi-conn" in
+      let cmd = Nbdkit.add_arg cmd "multi-conn-mode"
"disable" in
+      cmd
+    )
+    else cmd in
+
   (* If the filter supports it, enable cow-block-size (added in
    * nbdkit 1.27.6).  This helps to reduce fragmentated small
    * extent and read requests.
-- 
2.32.0