This series of patches implements the following ~ [1] Memory allocation schemes for VMs on NUMA platforms : The specific allocation allocation strategies available as configuration parameters are - * CONFINE - Confine the VM memory to a single NUMA node. [config] strategy = "confine" * STRIPE - Stripe the VM memory across a specified number of nodes. [config] strategy = "stripe" vnodes = <num> stripesz = <in pages> * SPLIT - Split the VM memory across a specified number of nodes to construct virtual nodes, which are then exposed to the VM. For now, we require the number of vnodes and number of vcpus to be powers of 2 (for symmetric distribution), as opposed to using multiples. [config] strategy = "split" vnodes = <num> * AUTO - Choose a scheme automatically, based on memory distribution across the nodes. The strategy attempts CONFINE and STRIPE(by dividing memory in equal parts) in that order. If both fail, then it reverts to the existing non-numa allocation. * No Configuration - No change from existing behaviour. [2] HVM NUMA guests : If the user specifies "split" strategy, we expose the virtual nodes to the HVM (SRAT/SLIT). [3] Disable migration : For now, the allocation information is not preserved across migration. We set disable_migrate for documentation, although it doesn't really stop one from migrating VMs(?). -Dulloor Signed-off-by: Dulloor <dulloor@gmail.com> -- xen/common/memory.c | 2 + xen/common/page_alloc.c | 9 + xen/include/public/memory.h | 3 + xen/include/xen/mm.h | 2 + tools/libxc/xenctrl.h | 19 + tools/libxl/libxl.h | 1 + tools/libxl/xl_cmdimpl.c | 50 +- xen/include/public/dom_numa.h | 71 ++ tools/libxc/Makefile | 1 + tools/libxc/xc_cpumap.c | 88 +++ tools/libxc/xc_cpumap.h | 113 +++ tools/libxc/Makefile | 1 + tools/libxc/xc_dom_numa.c | 909 +++++++++++++++++++++++++++++++ tools/libxc/xc_dom_numa.h | 49 + tools/libxc/xc_hvm_build.c | 320 ++++++----- tools/libxc/ia64/xc_ia64_hvm_build.c | 1 + tools/libxc/xc_dom_numa.c | 2 +- tools/libxc/xc_dom_numa.h | 2 +- tools/libxc/xc_hvm_build.c | 298 +++++++--- tools/libxc/xenguest.h | 1 + tools/libxl/libxl_dom.c | 5 +- tools/python/xen/lowlevel/xc/xc.c | 2 +- tools/firmware/hvmloader/acpi/acpi2_0.h | 64 ++ tools/firmware/hvmloader/acpi/build.c | 118 ++++ tools/libxc/xc_hvm_build.c | 49 +- xen/include/public/hvm/hvm_info_table.h | 8 +- tools/libxc/xc_hvm_build.c | 2 + 27 files changed, 1939 insertions(+), 251 deletions(-) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Sorry for the delay. I have been busy with other things.
Summary of the patches :
In this patch series, we implement the following ~
[1] Memory allocation schemes for VMs on NUMA platforms : The specific
allocation allocation strategies available as configuration parameters are -
* CONFINE - Confine the VM memory to a single NUMA node.
[config]
strategy = "confine"
* STRIPE - Stripe the VM memory across a specified number of nodes.
[config]
strategy = "stripe"
vnodes = <num>
stripesz = <in pages>
* SPLIT - Split the VM memory across a specified number of nodes
to construct virtual nodes, which are then exposed to the VM.
For now, we require the number of vnodes and number of vcpus to
be powers of 2 (for symmetric distribution), as opposed to using
multiples.
[config]
strategy = "split"
vnodes = <num>
* AUTO - Choose a scheme automatically, based on memory distribution
across the nodes. The strategy attempts CONFINE and STRIPE(by
dividing memory in equal parts) in that order. If both fail, then
it reverts to the existing non-numa allocation.
[config]
strategy = "auto"
* No Configuration - No change from existing behaviour.
[2] HVM NUMA guests : If the user specifies "split" strategy, we
expose the
virtual nodes to the HVM (SRAT/SLIT).
[3] Disable migration : For now, the allocation information is not preserved
across migration, so we just disable migration. We will address this in the next
patch series.
[4] PoD (Populate on Demand) : For now, PoD is disabled internally if a NUMA
allocation strategy is specified and applied to a VM. We will address
this in the
next patch series.
Changes from previous version :
[1] The guest interface structure has been modified per Keir''s
suggestions.
Most changes from previous version are due to this.
[2] Cleaned up debug code in setup_guest (spotted by George).
-Dulloor
Signed-off-by: Dulloor <dulloor@gmail.com>
--
tools/firmware/hvmloader/acpi/acpi2_0.h | 64 ++++++
tools/firmware/hvmloader/acpi/build.c | 122 ++++++++++++
tools/libxc/Makefile | 2 +
tools/libxc/ia64/xc_ia64_hvm_build.c | 1 +
tools/libxc/xc_cpumap.c | 88 +++++++++
tools/libxc/xc_cpumap.h | 113 +++++++++++
tools/libxc/xc_dom_numa.c | 901
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
tools/libxc/xc_dom_numa.h | 73 +++++++
tools/libxc/xc_hvm_build.c | 574
++++++++++++++++++++++++++++++++++++++++------------------
tools/libxc/xenctrl.h | 19 +
tools/libxc/xenguest.h | 1 +
tools/libxl/libxl.h | 1 +
tools/libxl/libxl_dom.c | 1 +
tools/libxl/xl_cmdimpl.c | 44 ++++
tools/python/xen/lowlevel/xc/xc.c | 2 +-
xen/include/public/arch-x86/dom_numa.h | 91 +++++++++
xen/include/public/dom_numa.h | 33 +++
xen/include/public/hvm/hvm_info_table.h | 10 +-
18 files changed, 1954 insertions(+), 186 deletions(-)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Aug-02 16:16 UTC
Re: [xen-devel][vNUMA v2][PATCH 0/8] VM memory mgmt for NUMA
On Sun, Aug 01, 2010 at 03:00:31PM -0700, Dulloor wrote:> Sorry for the delay. I have been busy with other things.Np. Can you CC these patches in the future to Andre? His email is Andre Przywara <andre.przywara@amd.com> In the meantime, I am CC-ing him here.> > > Summary of the patches : > In this patch series, we implement the following ~ > > [1] Memory allocation schemes for VMs on NUMA platforms : The specific > allocation allocation strategies available as configuration parameters are - > > * CONFINE - Confine the VM memory to a single NUMA node. > [config] > strategy = "confine" > > * STRIPE - Stripe the VM memory across a specified number of nodes. > [config] > strategy = "stripe" > vnodes = <num> > stripesz = <in pages> > > * SPLIT - Split the VM memory across a specified number of nodes > to construct virtual nodes, which are then exposed to the VM. > For now, we require the number of vnodes and number of vcpus to > be powers of 2 (for symmetric distribution), as opposed to using > multiples. > [config] > strategy = "split" > vnodes = <num> > > * AUTO - Choose a scheme automatically, based on memory distribution > across the nodes. The strategy attempts CONFINE and STRIPE(by > dividing memory in equal parts) in that order. If both fail, then > it reverts to the existing non-numa allocation. > [config] > strategy = "auto" > > * No Configuration - No change from existing behaviour. > > [2] HVM NUMA guests : If the user specifies "split" strategy, we expose the > virtual nodes to the HVM (SRAT/SLIT). > > [3] Disable migration : For now, the allocation information is not preserved > across migration, so we just disable migration. We will address this in the next > patch series. > > [4] PoD (Populate on Demand) : For now, PoD is disabled internally if a NUMA > allocation strategy is specified and applied to a VM. We will address > this in the > next patch series. > > Changes from previous version : > [1] The guest interface structure has been modified per Keir''s suggestions. > Most changes from previous version are due to this. > [2] Cleaned up debug code in setup_guest (spotted by George). > > > -Dulloor > > Signed-off-by: Dulloor <dulloor@gmail.com> > > -- > tools/firmware/hvmloader/acpi/acpi2_0.h | 64 ++++++ > tools/firmware/hvmloader/acpi/build.c | 122 ++++++++++++ > tools/libxc/Makefile | 2 + > tools/libxc/ia64/xc_ia64_hvm_build.c | 1 + > tools/libxc/xc_cpumap.c | 88 +++++++++ > tools/libxc/xc_cpumap.h | 113 +++++++++++ > tools/libxc/xc_dom_numa.c | 901 > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > tools/libxc/xc_dom_numa.h | 73 +++++++ > tools/libxc/xc_hvm_build.c | 574 > ++++++++++++++++++++++++++++++++++++++++------------------ > tools/libxc/xenctrl.h | 19 + > tools/libxc/xenguest.h | 1 + > tools/libxl/libxl.h | 1 + > tools/libxl/libxl_dom.c | 1 + > tools/libxl/xl_cmdimpl.c | 44 ++++ > tools/python/xen/lowlevel/xc/xc.c | 2 +- > xen/include/public/arch-x86/dom_numa.h | 91 +++++++++ > xen/include/public/dom_numa.h | 33 +++ > xen/include/public/hvm/hvm_info_table.h | 10 +- > 18 files changed, 1954 insertions(+), 186 deletions(-) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Dulloor
2010-Aug-02 17:03 UTC
Re: [xen-devel][vNUMA v2][PATCH 0/8] VM memory mgmt for NUMA
On Mon, Aug 2, 2010 at 9:16 AM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Sun, Aug 01, 2010 at 03:00:31PM -0700, Dulloor wrote: >> Sorry for the delay. I have been busy with other things. > > Np. Can you CC these patches in the future to Andre? > His email is Andre Przywara <andre.przywara@amd.com>Sure, thanks ! :) I should be cc''ing Jun Nakajima too.> > In the meantime, I am CC-ing him here.>> >> >> Summary of the patches : >> In this patch series, we implement the following ~ >> >> [1] Memory allocation schemes for VMs on NUMA platforms : The specific >> allocation allocation strategies available as configuration parameters are - >> >> * CONFINE - Confine the VM memory to a single NUMA node. >> [config] >> strategy = "confine" >> >> * STRIPE - Stripe the VM memory across a specified number of nodes. >> [config] >> strategy = "stripe" >> vnodes = <num> >> stripesz = <in pages> >> >> * SPLIT - Split the VM memory across a specified number of nodes >> to construct virtual nodes, which are then exposed to the VM. >> For now, we require the number of vnodes and number of vcpus to >> be powers of 2 (for symmetric distribution), as opposed to using >> multiples. >> [config] >> strategy = "split" >> vnodes = <num> >> >> * AUTO - Choose a scheme automatically, based on memory distribution >> across the nodes. The strategy attempts CONFINE and STRIPE(by >> dividing memory in equal parts) in that order. If both fail, then >> it reverts to the existing non-numa allocation. >> [config] >> strategy = "auto" >> >> * No Configuration - No change from existing behaviour. >> >> [2] HVM NUMA guests : If the user specifies "split" strategy, we expose the >> virtual nodes to the HVM (SRAT/SLIT). >> >> [3] Disable migration : For now, the allocation information is not preserved >> across migration, so we just disable migration. We will address this in the next >> patch series. >> >> [4] PoD (Populate on Demand) : For now, PoD is disabled internally if a NUMA >> allocation strategy is specified and applied to a VM. We will address >> this in the >> next patch series. >> >> Changes from previous version : >> [1] The guest interface structure has been modified per Keir''s suggestions. >> Most changes from previous version are due to this. >> [2] Cleaned up debug code in setup_guest (spotted by George). >> >> >> -Dulloor >> >> Signed-off-by: Dulloor <dulloor@gmail.com> >> >> -- >> tools/firmware/hvmloader/acpi/acpi2_0.h | 64 ++++++ >> tools/firmware/hvmloader/acpi/build.c | 122 ++++++++++++ >> tools/libxc/Makefile | 2 + >> tools/libxc/ia64/xc_ia64_hvm_build.c | 1 + >> tools/libxc/xc_cpumap.c | 88 +++++++++ >> tools/libxc/xc_cpumap.h | 113 +++++++++++ >> tools/libxc/xc_dom_numa.c | 901 >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> tools/libxc/xc_dom_numa.h | 73 +++++++ >> tools/libxc/xc_hvm_build.c | 574 >> ++++++++++++++++++++++++++++++++++++++++------------------ >> tools/libxc/xenctrl.h | 19 + >> tools/libxc/xenguest.h | 1 + >> tools/libxl/libxl.h | 1 + >> tools/libxl/libxl_dom.c | 1 + >> tools/libxl/xl_cmdimpl.c | 44 ++++ >> tools/python/xen/lowlevel/xc/xc.c | 2 +- >> xen/include/public/arch-x86/dom_numa.h | 91 +++++++++ >> xen/include/public/dom_numa.h | 33 +++ >> xen/include/public/hvm/hvm_info_table.h | 10 +- >> 18 files changed, 1954 insertions(+), 186 deletions(-) >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Andre Przywara
2010-Aug-02 21:12 UTC
Re: [xen-devel][vNUMA v2][PATCH 0/8] VM memory mgmt for NUMA
Dulloor wrote:> On Mon, Aug 2, 2010 at 9:16 AM, Konrad Rzeszutek Wilk > <konrad.wilk@oracle.com> wrote: >> On Sun, Aug 01, 2010 at 03:00:31PM -0700, Dulloor wrote: >>> Sorry for the delay. I have been busy with other things. >> Np. Can you CC these patches in the future to Andre? >> His email is Andre Przywara <andre.przywara@amd.com> > Sure, thanks ! :) I should be cc''ing Jun Nakajima too.Thanks for the heads up, although this time I spotted the patches already myself. I already started the review and will send some comments tomorrow. Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 488-3567-12 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel