Libguestfs - Jan 2022 - Virt-v2v performance benchmarking part 3

On 01/11/22 11:56, Richard W.M. Jones wrote:
> On Tue, Jan 11, 2022 at 08:07:39AM +0100, Laszlo Ersek wrote:
>> hmmmm, not necessarily; according to the manual, "qemu-img
convert" uses
>> (by default) 8 co-routines. There's also the -W flag ("out of
order
>> writes"), which I don't know if the original virt-v2v used.
> 
> I'm never sure how qemu coroutines map to threads.  I assume it's
not
> 1-1, and it's somehow connected to the iothread setting?
(I don't know the answer, but) this difference should not matter here I
believe; the QEMU co-routines submit async IO one way or another (IIRC),
so once those are in the kernel's hands, it shouldn't matter (from an IO
performance perspective) whether the userspace process waits on their
completions with heavy-weight threads, or co-routines or whatever. What
matters is that the kernel has multiple IO reqs in flight at the same time.

Thanks,
Laszlo
> 
> The -W flag was only used for -o rhv-upload and not for any other
> input or output method.  See write_out_of_order in the 1.44 code.
> 
> Rich.
>

On Wed, Jan 12, 2022 at 1:12 PM Laszlo Ersek <lersek at redhat.com>
wrote:
>
> On 01/11/22 11:56, Richard W.M. Jones wrote:
> > On Tue, Jan 11, 2022 at 08:07:39AM +0100, Laszlo Ersek wrote:
> >> hmmmm, not necessarily; according to the manual, "qemu-img
convert" uses
> >> (by default) 8 co-routines. There's also the -W flag
("out of order
> >> writes"), which I don't know if the original virt-v2v
used.
> >
> > I'm never sure how qemu coroutines map to threads.  I assume
it's not
> > 1-1, and it's somehow connected to the iothread setting?
>
> (I don't know the answer, but) this difference should not matter here I
> believe; the QEMU co-routines submit async IO one way or another (IIRC),
> so once those are in the kernel's hands, it shouldn't matter (from
an IO
> performance perspective) whether the userspace process waits on their
> completions with heavy-weight threads, or co-routines or whatever. What
> matters is that the kernel has multiple IO reqs in flight at the same time.
In practice this matters a lot. Using -W can be 6x times faster with qemu-img
convert.

See how I tested this here:
https://bugzilla.redhat.com/1511891#c57

Nir