Andrej Komelj
2014-Sep-14 19:40 UTC
[Xen VGA Passthrough] DomU performance inversely proportional to VM memory size when VGA passthrough is enabled
Hi, I've noticed a peculiar behavior of Xen 4.4.x when enabling VGA passthrough to DomU domain on my desktop computer - performance of the DomU is inversely proportional to the size of the domain's memory. The more memory the domain has, the slower the overall performance of the domain gets. Degradation is considerable and noticeable, not only theoretical or in a range of several percents. The interesting thing though is: If VGA passthrough is disabled, the domain performance remains constant, regardless of the size of memory allocated to it. Needless to say, performance without VGA passthrough is similar to bare-metal system but performance with VGA passthrough starts noticeably degrading when I assign more that 512 MB of memory to DomU. With 8G assigned, it is practically unusable. I've been using a simple gzip/gunzip test to confirm my findings: on bare-metal system, it takes roughly 1 second to compress a big file, on VGA-passthrough-enabled DomU it takes 5 seconds with 4 GB RAM and 15 seconds with 8 GB RAM. I ran additional benchmarking tools (cpu, memory, graphics, overall) and they all confirmed that my system is considerably slower when using passthrough and large amount of memory. Performance drop is noticeable even when using purely text-mode DomU with graphics drivers on the blacklist and disabled framebuffer in DomU linux kernel. Computer HW specs (cpuinfo & dmidecode in attachment) - HP Elite 7500 system: - 8 core Intel i7-3770 CPU @ 3.40GHz - Intel Z75 Express chipset - integrated graphics Intel HD 4000 - 12 GB RAM - VT-d supported and tested (been successfully running VGA passthrough on 4.3.x with Windows 8.1/XP) Dom0s: - Arch Linux (Xen 4.4.0 & Xen 4.4.1 from AUR) - Linux Mint 16 (Ubuntu 14.04 based - Xen 4.4.0) - Ubuntu Linux 14.04 (Xen 4.4.0) DomUs (PV-on-HVM): - Linux Mint 16 (Ubuntu 14.04 based) - kernels 3.13.0, 3.14.1, 3.16.2, 3.17.0 - Arch Linux latest - Debian Jessie (testing) - I've already wiped the logical volume so I don't know what version of kernel was running I've been playing with pretty much every setting in domain configuration file to no avail: - primary/secondary passthrough (gfx_passthru=0/1) - shadow memory size - xen_platform_pci - hap, apic, acpi, ... - pci_permissive, pci_power_mgmt, pci_msitranslate - ... The only thing that has any impact on the performance is: - memory size - the smaller the better :-( - PCI passthrough of graphics adapter - disabled (OK) or enabled (huge performance hit) Does anybody know what else I could try/check? The only similar post on Xen-users I was able to find is: http://xen.1045712.n5.nabble.com/Performance-degradation-with-pci-passthrough-amp-xen-4-4-td5724369.html I've attached various debug/output files if anyone wants to deal with them: - dmidecode output - /proc/cpuinfo - lspci -vv - dom0 xen dmesg log - dom0 kernel log - domU kernel log - domainU configuration file Best regards, Andrej _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users