similar to: [PATCH 00/11] v2v: Change virt-v2v to use nbdkit for input in several modes.

v3 posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00200.html v4: - The first patch in the v3 series was just a trivial doc whitespace fix so I pushed it. - There's a new patch using the nbdkit-retry-filter. This is not actually upstream in nbdkit but we know enough about how it will work. - Rebased against master and reran the tests. Rich.

v2 was posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00115.html This also has links to earlier versions. v3: - The 01/11 patch in v2 included a bunch of unnecessary plus one necessary change to how input_password is passed around. I moved the necessary change into the final patch (implementing SSH password authentication) and dropped the rest. - The 01/11

Originally posted here: https://www.redhat.com/archives/libguestfs/2019-April/thread.html#00054 https://www.redhat.com/archives/libguestfs/2019-April/msg00076.html https://www.redhat.com/archives/libguestfs/2019-April/msg00126.html This is a rebase on top of current master branch with no other changes. The first patch in the old series was pushed a while back, and the last "TEMPORARY"

This is built on top of the following patch series: https://www.redhat.com/archives/libguestfs/2019-April/msg00054.html Rich.

Patch 1 was previously posted here: https://www.redhat.com/archives/libguestfs/2020-June/msg00086.html to handle this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1848862 I was able to observe this bug and for me at least disabling readahead seems to cure it. Patches 2 and 3 are simplifications, removing a now-undocumented feature of virt-v2v-copy-to-local and thus simplifying greatly the

Part 2 fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1842440 Actually this fix on its own should be sufficient, but probably we want the nbdkit fixes too. Note this uses actual OCaml 4.05 features! ("let open" and the Unix.tcgetattr functions). I checked that both features are available on RHEL 7's OCaml. Rich.

Use -ip instead of --password-file, and various refactorings. It strikes me that we should probably deprecate and eventually remove virt-v2v-copy-to-local. With the introduction of the new SSH and VDDK transports, and with RHEL 5 Xen becoming more irrelevant, it's no longer needed. Rich.

v6 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00126.html This makes a number of significant changes: - Input and output options now use a uniform set of -io and -oo parameters. - For -o rhv-upload, we use ‘-oo rhv-cafile=/tmp/ca.pem’ etc. The ‘--rhv*’ options have been dropped. - Rearranges the documentation. - As before includes (untested) support for zero,

The first patch was previously posted here: https://www.redhat.com/archives/libguestfs/2017-December/msg00018.html That patch hasn't changed except that I made the ‘input_transport’ variable type-safe. The second patch adds a significant new mode for liberating data from VMware: the ability to copy VMs over SSH directly from ESXi hypervisors. Although this requires enabling SSH access (a

Proposed small change to the command line of virt-v2v when specifying that you want VDDK mode. Rich.

This patch isn't quite complete (see ‘assert false’). Currently we have a bunch of ad hoc options like --vddk* and --vdsm* (and proposed to add --rhv*) to handle extra parameters for input and output modes/transports. This complicates the command line parsing and also the clarity of the command line (becauseit's not very obvious which options apply to which side of the conversion).

On Fri, Sep 20, 2019 at 10:28:18AM +0100, Richard W.M. Jones wrote: >The readahead filter is a self-configuring filter that makes >sequential reads faster when the plugin is slow (and all of the >plugins we use here are always slow). > >I observed the behaviour of the readahead filter with our qcow2 >overlay when converting a guest from a vCenter source. Even when >doing

Mainly minor fixes and code cleanups over the v4 patch. There are still several problems with this patch, but it is in a reviewable state, especially the Python code. Rich.

The first commit removes the --dcpath parameter, which just makes the following stuff simpler. Since libvirt has supported getting datacenterpath from VMware since Oct 2015, it's time to drop this hairy parameter. The rest is quite a complicated series of refactorings, followed by a very simple change to add handling of snapshots taken from old virt-v2v. Rich.

Previous versions: v3: https://www.redhat.com/archives/libguestfs/2018-March/msg00000.html v2: https://www.redhat.com/archives/libguestfs/2018-February/msg00177.html v1: https://www.redhat.com/archives/libguestfs/2018-February/msg00139.html This completely rethinks the approach taken by the previous patches. Instead of trying to involve qemu's curl driver, this uses a small Python 3

See earlier thread on libvir-list: https://www.redhat.com/archives/libvir-list/2015-September/thread.html#00201 Libvirt >= 1.2.20 supplies the correct dcPath parameter. If it is available in the libvirt XML, use it, otherwise fall back to the old method of trying to guess it from the vpx:// path. Patches 1, 2 and 4 are just refactoring around this change. Rich.

This refactoring takes the nbdkit-specific code from the ‘-it vddk’ mode and puts it into a separate module. The idea is to reuse this module (in future commits) to support ssh, curl and rate limiting. This refactoring is not quite neutral because it moves some of the prechecking into the Nbdkit.create function. This means it will happen a little bit later in the cycle (in input#source instead

This is the simplest solution to this problem. There are two other possible fixes I considered: Increase the documented limit (see http://libguestfs.org/virt-v2v-input-vmware.1.html#vddk:-esxi-nfc-service-memory-limits). However at the moment we know the current limit works through extensive testing (without readahead), plus I have no idea nor any way to test if larger limits are supported by

v5 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00032.html There is only a single patch in this version because the other patches went upstream. This patch adds the virt-v2v -o rhv-upload mode (https://bugzilla.redhat.com/show_bug.cgi?id=1557273). Compared to v5, this adds the ability to make zero, trim and flush requests to the oVirt imageio server

It turns out that RHEL 5 Xen conversions don't work if the source disk is located on a block device. See patch 1/4 for the gory details. This patch series proposes a new tool called virt-v2v-copy-to-local which essentially is a way to make new virt-v2v work like the old virt-v2v, ie. copy first, convert after. Of course this is very slow and would only be used as a last resort, but I