similar to: [PATCH v4 00/12] v2v: Change virt-v2v to use nbdkit for input in several modes.

Originally posted here: https://www.redhat.com/archives/libguestfs/2019-April/thread.html#00054 https://www.redhat.com/archives/libguestfs/2019-April/msg00076.html https://www.redhat.com/archives/libguestfs/2019-April/msg00126.html This is a rebase on top of current master branch with no other changes. The first patch in the old series was pushed a while back, and the last "TEMPORARY"

v2 was posted here: https://www.redhat.com/archives/libguestfs/2019-July/msg00115.html This also has links to earlier versions. v3: - The 01/11 patch in v2 included a bunch of unnecessary plus one necessary change to how input_password is passed around. I moved the necessary change into the final patch (implementing SSH password authentication) and dropped the rest. - The 01/11

This is built on top of the following patch series: https://www.redhat.com/archives/libguestfs/2019-April/msg00054.html Rich.

This series (except the last one) changes virt-v2v to use nbdkit for several input modes: -i vmx -it vddk: No change in functionality, as this already uses nbdkit-vddk-plugin, but the code is refactored for the other modes to use. -i libvirtxml: Use nbdkit-curl-plugin instead of qemu curl. vCenter: Use nbdkit-curl-plugin instead of qemu curl. xen: Use nbdkit-ssh-plugin instead of qemu

Part 2 fix for: https://bugzilla.redhat.com/show_bug.cgi?id=1842440 Actually this fix on its own should be sufficient, but probably we want the nbdkit fixes too. Note this uses actual OCaml 4.05 features! ("let open" and the Unix.tcgetattr functions). I checked that both features are available on RHEL 7's OCaml. Rich.

Patch 1 was previously posted here: https://www.redhat.com/archives/libguestfs/2020-June/msg00086.html to handle this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1848862 I was able to observe this bug and for me at least disabling readahead seems to cure it. Patches 2 and 3 are simplifications, removing a now-undocumented feature of virt-v2v-copy-to-local and thus simplifying greatly the

The first patch was previously posted here: https://www.redhat.com/archives/libguestfs/2017-December/msg00018.html That patch hasn't changed except that I made the ‘input_transport’ variable type-safe. The second patch adds a significant new mode for liberating data from VMware: the ability to copy VMs over SSH directly from ESXi hypervisors. Although this requires enabling SSH access (a

This isn't quite the full thing. I think that Pino is also working on replacing the ssh and scp commands in the v2v/input_vmx.ml file with libssh. Without those changes, -i vmx will still issue raw ssh and scp commands, which will use ssh-agent (or keyboard-interactive). The Xen input method doesn't use raw ssh and scp commands, so that one is OK. Rich.

Use -ip instead of --password-file, and various refactorings. It strikes me that we should probably deprecate and eventually remove virt-v2v-copy-to-local. With the introduction of the new SSH and VDDK transports, and with RHEL 5 Xen becoming more irrelevant, it's no longer needed. Rich.

On Fri, Sep 20, 2019 at 10:28:18AM +0100, Richard W.M. Jones wrote: >The readahead filter is a self-configuring filter that makes >sequential reads faster when the plugin is slow (and all of the >plugins we use here are always slow). > >I observed the behaviour of the readahead filter with our qcow2 >overlay when converting a guest from a vCenter source. Even when >doing

It does not need configuring and speeds up the process of requesting extents if the client asks for them one by one (like qemu-img). Signed-off-by: Martin Kletzander <mkletzan@redhat.com> --- Essentially a v2 of the following patch since now there is a probing function: https://www.redhat.com/archives/libguestfs/2019-September/msg00249.html v2v/nbdkit.ml | 7 +++++++ 1 file changed, 7

This is the simplest solution to this problem. There are two other possible fixes I considered: Increase the documented limit (see http://libguestfs.org/virt-v2v-input-vmware.1.html#vddk:-esxi-nfc-service-memory-limits). However at the moment we know the current limit works through extensive testing (without readahead), plus I have no idea nor any way to test if larger limits are supported by

Proposed small change to the command line of virt-v2v when specifying that you want VDDK mode. Rich.

This patch isn't quite complete (see ‘assert false’). Currently we have a bunch of ad hoc options like --vddk* and --vdsm* (and proposed to add --rhv*) to handle extra parameters for input and output modes/transports. This complicates the command line parsing and also the clarity of the command line (becauseit's not very obvious which options apply to which side of the conversion).

I'm still testing this fix, so let's hold off the review for the moment. Also it may be better to specifically identify problematic servers rather than disabling this for every curl source. eg. I suspect that the problem is the Java server used by VCenter, so we might think about only disabling readahead for that single case. Rich.

v6 was here: https://www.redhat.com/archives/libguestfs/2018-March/msg00126.html This makes a number of significant changes: - Input and output options now use a uniform set of -io and -oo parameters. - For -o rhv-upload, we use ‘-oo rhv-cafile=/tmp/ca.pem’ etc. The ‘--rhv*’ options have been dropped. - Rearranges the documentation. - As before includes (untested) support for zero,

This refactoring takes the nbdkit-specific code from the ‘-it vddk’ mode and puts it into a separate module. The idea is to reuse this module (in future commits) to support ssh, curl and rate limiting. This refactoring is not quite neutral because it moves some of the prechecking into the Nbdkit.create function. This means it will happen a little bit later in the cycle (in input#source instead

The tests assume probe_filter returns true if the filter is available (and not the other way around). Signed-off-by: Tomáš Golembiovský <tgolembi@redhat.com> --- v2v/nbdkit.ml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/v2v/nbdkit.ml b/v2v/nbdkit.ml index 77d2a506..00122bec 100644 --- a/v2v/nbdkit.ml +++ b/v2v/nbdkit.ml @@ -142,7 +142,7 @@ let common_create

On Friday, 8 December 2017 17:02:30 CET Richard W.M. Jones wrote: > This enhances the existing VMX input support allowing it to be > used over SSH to the ESXi server. > > The original command (for local .vmx files) was: > > $ virt-v2v -i vmx guest.vmx -o local -os /var/tmp > > Adding ‘-it ssh’ and using an SSH remote path gives the new syntax: > > $ virt-v2v \

Pass (again) around the libvirt URI string in the various input_libvirt subclasses so that input_libvirt#as_options does not need to connect to print the connection URI. As related change: pass input_conn as non-optional string parameter in classes that require one (all but input_libvirt_other, basically). This avoids the need for extra checks. --- v2v/input_libvirt.ml | 10