Libguestfs - Jan 2022 - Virt-v2v performance benchmarking part 3

On Tue, Jan 11, 2022 at 08:07:39AM +0100, Laszlo Ersek
wrote:
> hmmmm, not necessarily; according to the manual, "qemu-img
convert" uses
> (by default) 8 co-routines. There's also the -W flag ("out of
order
> writes"), which I don't know if the original virt-v2v used.
I'm never sure how qemu coroutines map to threads.  I assume it's not
1-1, and it's somehow connected to the iothread setting?

The -W flag was only used for -o rhv-upload and not for any other
input or output method.  See write_out_of_order in the 1.44 code.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org

On 01/11/22 11:56, Richard W.M. Jones wrote:
> On Tue, Jan 11, 2022 at 08:07:39AM +0100, Laszlo Ersek wrote:
>> hmmmm, not necessarily; according to the manual, "qemu-img
convert" uses
>> (by default) 8 co-routines. There's also the -W flag ("out of
order
>> writes"), which I don't know if the original virt-v2v used.
> 
> I'm never sure how qemu coroutines map to threads.  I assume it's
not
> 1-1, and it's somehow connected to the iothread setting?
(I don't know the answer, but) this difference should not matter here I
believe; the QEMU co-routines submit async IO one way or another (IIRC),
so once those are in the kernel's hands, it shouldn't matter (from an IO
performance perspective) whether the userspace process waits on their
completions with heavy-weight threads, or co-routines or whatever. What
matters is that the kernel has multiple IO reqs in flight at the same time.

Thanks,
Laszlo
> 
> The -W flag was only used for -o rhv-upload and not for any other
> input or output method.  See write_out_of_order in the 1.44 code.
> 
> Rich.
>

On Tue, Jan 11, 2022 at 1:00 PM Richard W.M. Jones <rjones at redhat.com>
wrote:
>
> On Tue, Jan 11, 2022 at 08:07:39AM +0100, Laszlo Ersek wrote:
> > hmmmm, not necessarily; according to the manual, "qemu-img
convert" uses
> > (by default) 8 co-routines. There's also the -W flag ("out of
order
> > writes"), which I don't know if the original virt-v2v used.
>
> I'm never sure how qemu coroutines map to threads.  I assume it's
not
> 1-1, and it's somehow connected to the iothread setting?
In qemu-img and qemu-nbd, when using cache=none and aio=native there
is one thread submitting io via libaio.

When using aio=threads we can see multiple threads doing the I/O. I don't
know how qemu maps request to threads, but it seems that we have one
thread per inflight request, but never more than 16 (qmu-nbd never handle
more than 16 inflight requests per client).

I tested with "nbdcopy --connections 1 --requests ..." and we see the
same
number of threads in qemu-nbd plus one idle thread when requests < 16.

Nir