Libguestfs - Feb 2022 - [PATCH v2] v2v/v2v.ml: Use larger request size for -o rhv-upload

This change slowed things down (slightly) for me, although the change
is within the margin of error so it probably made no difference.

Before:

$ time ./run virt-v2v -i disk /var/tmp/fedora-35.qcow2 -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -op /tmp/ovirt-passwd -oo rhv-direct -os
ovirt-data -on test14 -of raw
[   0.0] Setting up the source: -i disk /var/tmp/fedora-35.qcow2
[   1.0] Opening the source
[   6.5] Inspecting the source
[  10.5] Checking for sufficient free disk space in the guest
[  10.5] Converting Fedora Linux 35 (Thirty Five) to run on KVM
virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
device "vda".  You may have to fix this entry manually after
conversion.
virt-v2v: This guest has virtio drivers installed.
[  57.0] Mapping filesystem data to avoid copying unused and blank areas
[  59.0] Closing the overlay
[  59.6] Assigning disks to buses
[  59.6] Checking if the guest needs BIOS or UEFI to boot
[  59.6] Setting up the destination: -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -os ovirt-data
[  79.3] Copying disk 1/1
? 100% [****************************************]
[  89.9] Creating output metadata
[  94.0] Finishing off

real	 1m34.213s
user	 0m6.585s
sys	 0m11.880s


After:

$ time ./run virt-v2v -i disk /var/tmp/fedora-35.qcow2 -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -op /tmp/ovirt-passwd -oo rhv-direct -os
ovirt-data -on test15 -of raw
[   0.0] Setting up the source: -i disk /var/tmp/fedora-35.qcow2
[   1.0] Opening the source
[   7.4] Inspecting the source
[  11.7] Checking for sufficient free disk space in the guest
[  11.7] Converting Fedora Linux 35 (Thirty Five) to run on KVM
virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
device "vda".  You may have to fix this entry manually after
conversion.
virt-v2v: This guest has virtio drivers installed.
[  59.6] Mapping filesystem data to avoid copying unused and blank areas
[  61.5] Closing the overlay
[  62.2] Assigning disks to buses
[  62.2] Checking if the guest needs BIOS or UEFI to boot
[  62.2] Setting up the destination: -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -os ovirt-data
[  81.6] Copying disk 1/1
? 100% [****************************************]
[  91.3] Creating output metadata
[  96.0] Finishing off

real	 1m36.275s
user	 0m4.700s
sys	 0m14.070s


Rich.


-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/

On Mon, Feb 14, 2022 at 11:56 AM Richard W.M. Jones <rjones at redhat.com>
wrote:
>
> This change slowed things down (slightly) for me, although the change
> is within the margin of error so it probably made no difference.
>
> Before:
...
> [  79.3] Copying disk 1/1
> ? 100% [****************************************]
> [  89.9] Creating output metadata
10.6 seconds...
> After:
...
> [  81.6] Copying disk 1/1
> ? 100% [****************************************]
> [  91.3] Creating output metadata
9.7 seconds - 9% speedup.

We cannot compare the total time since creating a disk can be take 4-16
seconds.

What kind of storage is this? Is this the local storage hack used as NFS?
You may get much better performance with local disk, hiding the delays in
writing to shared storage. But most oVirt users use NFS or GlsuterFS.

I tested on NFS, tuned to simulate a fast NFS server.

Testing such changes should be done on a real server with real storage.
I'll try to get a server in our scale lab to do more real testing.

Nir

On 02/14/22 10:56, Richard W.M. Jones wrote:
> This change slowed things down (slightly) for me, although the change
> is within the margin of error so it probably made no difference.
> 
> Before:
> 
> $ time ./run virt-v2v -i disk /var/tmp/fedora-35.qcow2 -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -op /tmp/ovirt-passwd -oo rhv-direct -os
ovirt-data -on test14 -of raw
> [   0.0] Setting up the source: -i disk /var/tmp/fedora-35.qcow2
> [   1.0] Opening the source
> [   6.5] Inspecting the source
> [  10.5] Checking for sufficient free disk space in the guest
> [  10.5] Converting Fedora Linux 35 (Thirty Five) to run on KVM
> virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
> device "vda".  You may have to fix this entry manually after
conversion.
> virt-v2v: This guest has virtio drivers installed.
> [  57.0] Mapping filesystem data to avoid copying unused and blank areas
> [  59.0] Closing the overlay
> [  59.6] Assigning disks to buses
> [  59.6] Checking if the guest needs BIOS or UEFI to boot
> [  59.6] Setting up the destination: -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -os ovirt-data
> [  79.3] Copying disk 1/1
> ? 100% [****************************************]
> [  89.9] Creating output metadata
> [  94.0] Finishing off
> 
> real	 1m34.213s
> user	 0m6.585s
> sys	 0m11.880s
> 
> 
> After:
> 
> $ time ./run virt-v2v -i disk /var/tmp/fedora-35.qcow2 -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -op /tmp/ovirt-passwd -oo rhv-direct -os
ovirt-data -on test15 -of raw
> [   0.0] Setting up the source: -i disk /var/tmp/fedora-35.qcow2
> [   1.0] Opening the source
> [   7.4] Inspecting the source
> [  11.7] Checking for sufficient free disk space in the guest
> [  11.7] Converting Fedora Linux 35 (Thirty Five) to run on KVM
> virt-v2v: warning: /files/boot/grub2/device.map/hd0 references unknown 
> device "vda".  You may have to fix this entry manually after
conversion.
> virt-v2v: This guest has virtio drivers installed.
> [  59.6] Mapping filesystem data to avoid copying unused and blank areas
> [  61.5] Closing the overlay
> [  62.2] Assigning disks to buses
> [  62.2] Checking if the guest needs BIOS or UEFI to boot
> [  62.2] Setting up the destination: -o rhv-upload -oc
https://ovirt4410/ovirt-engine/api -os ovirt-data
> [  81.6] Copying disk 1/1
> ? 100% [****************************************]
> [  91.3] Creating output metadata
> [  96.0] Finishing off
> 
> real	 1m36.275s
> user	 0m4.700s
> sys	 0m14.070s
My ACK on Nir's v2 patch basically means that I defer to you on its
review -- I don't have anything against it, but I understand it's
(perhaps a temporary) workaround until we find a more sustainable (and
likely much more complex) solution.

Thanks
Laszlo