Richard W.M. Jones
2020-Sep-18 09:44 UTC
[Libguestfs] [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
So it didn't make any noticable difference in my test. I wonder if the test guest I'm using (Fedora 32 using dracut) doesn't use parallel compression? However I don't think it can cause a problem and it seems obvious that it could benefit some cases. Rich.
Richard W.M. Jones
2020-Sep-18 09:44 UTC
[Libguestfs] [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
This helps mkinitrd which runs pigz (parallel gzip). Thanks Jean-Louis Dupond for suggesting this change. --- common | 2 +- v2v/v2v.ml | 2 ++ 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/common b/common index c840f2e39..ea5278bba 160000 --- a/common +++ b/common @@ -1 +1 @@ -Subproject commit c840f2e39d0bb637a98b224c89f6df011e1d4414 +Subproject commit ea5278bba7800e3a6d9c233a0f5f53a9acf244f8 diff --git a/v2v/v2v.ml b/v2v/v2v.ml index 73edff2c4..7637b1a8c 100644 --- a/v2v/v2v.ml +++ b/v2v/v2v.ml @@ -88,6 +88,8 @@ let rec main () let g = open_guestfs ~identifier:"v2v" () in g#set_memsize (g#get_memsize () * 14 / 5); + (* Setting the number of vCPUs allows parallel mkinitrd. *) + g#set_smp (Sysconf.nr_processors_online ()); (* The network is only used by the unconfigure_vmware () function. *) g#set_network true; (match conversion_mode with -- 2.27.0
Daniel P. Berrangé
2020-Sep-18 09:52 UTC
Re: [Libguestfs] [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
On Fri, Sep 18, 2020 at 10:44:03AM +0100, Richard W.M. Jones wrote:> So it didn't make any noticable difference in my test. I wonder if > the test guest I'm using (Fedora 32 using dracut) doesn't use parallel > compression?Do you do anything special to optimize storage ? If the thing using parallel CPUs in the guest is doing I/O you'd likely want to tune storage at same time. In current QEMU git master there's a change[1] that sets viort-blk/virtio-scsi num_queues to match SMP count, so the guest OS get fully parallel I/O from meach guest CPU. [1] a4eef0711b2cf7a7476c3e2c202a414b68a1baa0> However I don't think it can cause a problem and it seems obvious that > it could benefit some cases.What's the default RAM size ? On hosts with very large numbers of CPUs (100's) you might need to scale up RAM size too so you keep a sane RAM-per-CPU ratio. Or default to host CPUs, but cap it at say 32. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Richard W.M. Jones
2020-Sep-18 10:22 UTC
Re: [Libguestfs] [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
On Fri, Sep 18, 2020 at 10:52:58AM +0100, Daniel P. Berrangé wrote:> On Fri, Sep 18, 2020 at 10:44:03AM +0100, Richard W.M. Jones wrote: > > So it didn't make any noticable difference in my test. I wonder if > > the test guest I'm using (Fedora 32 using dracut) doesn't use parallel > > compression? > > Do you do anything special to optimize storage ? If the thing using > parallel CPUs in the guest is doing I/O you'd likely want to tune > storage at same time. In current QEMU git master there's a change[1] > that sets viort-blk/virtio-scsi num_queues to match SMP count, > so the guest OS get fully parallel I/O from meach guest CPU. > > [1] a4eef0711b2cf7a7476c3e2c202a414b68a1baa0I'm not using a new enough qemu to have that and it doesn't appear in a released version, but I'll test again once we have a new enough version.> > However I don't think it can cause a problem and it seems obvious that > > it could benefit some cases. > > What's the default RAM size ? On hosts with very large numbers of CPUs > (100's) you might need to scale up RAM size too so you keep a sane > RAM-per-CPU ratio. Or default to host CPUs, but cap it at say 32.I'll cap the number of CPUs in this patch. Thanks, Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW
Pino Toscano
2020-Sep-18 12:03 UTC
Re: [Libguestfs] [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
On Friday, 18 September 2020 11:44:04 CEST Richard W.M. Jones wrote:> let g = open_guestfs ~identifier:"v2v" () in > g#set_memsize (g#get_memsize () * 14 / 5); > + (* Setting the number of vCPUs allows parallel mkinitrd. *) > + g#set_smp (Sysconf.nr_processors_online ());IMHO this is not a good idea, for few reasons: a) it unconditionally uses all the available CPUs with no way to change it to any other value b) virt-v2v is run also on systems that are not specifically dedicated as conversion servers, for example the oVirt hosts (that run the oVirt VMs); in this context, suddently using more CPUs than requested may saturate the system resources IMHO the right approach is to add a --smp N option like available in other libguestfs tools; this way, the user can decide how many resources virt-v2v can use. -- Pino Toscano
Maybe Matching Threads
- [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
- Re: [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
- Re: [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
- Re: [PATCH v2v] v2v: Set the number of vCPUs to same as host number of pCPUs.
- [PATCH common] mlutils: Simple wrapper around sysconf (_SC_NPROCESSORS_ONLN).