The following patchset adds NUMA support to the hypervisor. This
includes:
- A full SRAT table parser for 32-bit and 64-bit, based on the ACPI
NUMA parser from linux 2.6.16.29 with data structures to represent
NUMA cpu and memory topology, and NUMA emulation (fake=).
- Changes to the Xen page allocator adding a per-node bucket for each
zone. Xen will continue to prioritize using the requested zone at
the cost of using non-local memory.
- Modified xen memory_ops to use NUMA-aware page allocator functions to
help allocate node-local memory for guests
- export NUMA topology through physinfo hypercall, integreated into
Xend
- fix-up nr_cpus calculation now that nr_nodes is not static
- Add a tool to explore NUMA allocations and Xen heap status.
Nothing has functionally changed since the last posting of the patches;
I''ve just been maintaining the patchset against unstable.
Additionally, I''ve spent the past few weeks collecting some performance
benchmarks comparing with and without the NUMA patches on a system
configured as a 4-socket server(no NUMA characteristics) and as a
multi-node 16-socket NUMA machine.
Machine: x460 4-node, 3Ghz Xeon PIV, 1MB L2, 8MB L3
16 Socket (4 sockets per node) HT, logical 32-way,
64GB RAM, 16G per node
Xen Changeset used: 11536:041be3f6b38e
We ran sdet and dbench, recording aggregate throughput for 4, 8, 12, and
16domUs. This loads the system with 4domUs on each node, one node, two
nodes, etc up to 4 nodes pinning the guest VCPUs to the proper physical
cpus in their respective nodes.
To look for possible regressions on non-NUMA systems, we used a single
node (4 sockets and 16G) for the first 4domU test as a single node does
not have any NUMA characteristics.
In summary, there are no regressions for the single node, non-numa case.
As we scale the number of domains out the NUMA patches show significant
gains.
dbench(tmpfs) 4domU 8domU 12domU 16domU
-------------------------------------------
with 590 1175 1750 2175
without 590 1090 1405 1450
-------------------------------------------
%gain +0.0% +7.8% +24.5% +50.0%
sdet 4domU 8domU 12domU 16domU
---------------------------------------
with 14000 25900 39500 49000
without 13900 24950 34900 38500
---------------------------------------
%gain +0.8% +3.8% +13.2% +27.0%
I''ve also been working with Raj at Unisys to test out the NUMA patches
on their ES7000. Just today Raj confirmed that the previous "dom0
slowdown" that had been reported has been reproduced without the NUMA
patches. I believe there are no outstanding machines that have tested
the NUMA patches that have any issues.
I would like to see the NUMA patches included in xen-unstable once it
is open for wider testing prior to the next release after 3.0.3.
--
Ryan Harper
Software Engineer; Linux Technology Center
IBM Corp., Austin, Tx
(512) 838-9253 T/L: 678-9253
ryanh@us.ibm.com
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
