Gianni Tedesco
2010-Sep-01 09:06 UTC
[Xen-devel] *massive* slowdown when running oprofile in dom0
Hi, I am running oprofile in dom0 (just profiling dom0, not VM''s) and whenever statistics collection is enabled (via opcontrol --start) the performance of VM''s drops through the floor. For example a fresh windows(r) XP(tm) SP3 install takes 2 minutes to boot without oprofile but a short hair under 5 minutes when stats are being collected. It seemed even worse for installing, I had left it going for hours before realising oprofile was the culprit (and still at the yellow progress bar on blue background ''copying files'' stage). In general there is just a noticable ''lagginess'' about everything. I have ruled out the obvious culprits, eg: xenoprof interrupt is only firing 800 or so times a second, oprofiled is not using all the CPU time. So it''s got to be a hypervisor/emulation thing right? Furthermore when I disable collection with opcontrol --stop I get: (XEN) nmi_stop: APIC not good 66560l (XEN) CA-24901: CPU 0 fixed up 1 lost platform-timer cycles (XEN) CPU01: @ 00000ce3b1e78f55, 00000ce3b1e794d1 (XEN) t 000026a6f8ef3731, 00000ce26efa7a65, 00000ce26efa79a4 <<-1 *aaac435f (XEN) c 000026aac1accdc5, 00000ce3b1e75cab, 00000ce3b1e73927 <<-1 *aaabcb5b (XEN) CPU00: @ 00000ce3b1e7ea2b, 00000ce3b1e7f2ab (XEN) t 000026a6f8ef35ba, 00000ce26efa7b28, 00000ce26efa79a4 <<-1 *aaac4113 (XEN) c 000026aac1accc21, 00000ce3b1e759d3, 00000ce3b1e73927 <<-1 *aaabd388 And everything hangs for 5 seconds due to mdelay(5000) !?!?!? Please tell me it''s bad hardware... # cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 15 model : 6 model name : Genuine Intel(R) CPU 3.00GHz stepping : 2 cpu MHz : 2999.892 cache size : 2048 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 6 wp : yes flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc up pni vmx est cid bogomips : 6004.03 clflush size : 64 power management: _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Gianni Tedesco
2010-Sep-01 11:06 UTC
Re: [Xen-devel] *massive* slowdown when running oprofile in dom0
On Wed, 2010-09-01 at 10:06 +0100, Gianni Tedesco wrote:> Please tell me it''s bad hardware... > > # cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 15 > model : 6 > model name : Genuine Intel(R) CPU 3.00GHz > stepping : 2 > cpu MHz : 2999.892 > cache size : 2048 KB > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 6 > wp : yes > flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush > acpi mmx fxsr sse sse2 ss ht nx constant_tsc up pni vmx est cid > bogomips : 6004.03 > clflush size : 64 > power management:Tried on a different machine with following cpuinfo (ie. a core 2, not a p4) - and now it slows down winxpsp3 boot from 30s to more like 15 minutes (yes.. 30x slowdown) with an average of 400 xenoprof interrupts per second and dom0 CPU at 99% idle with very little disk I/O. processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 26 model name : Genuine Intel(R) CPU @ 0000 @ 2.93GHz stepping : 2 cpu MHz : 2933.550 cache size : 8192 KB fdiv_bug : no hlt_bug : no f00f_bug : no coma_bug : no fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht nx constant_tsc up pni vmx est ssse3 sse4_1 sse4_2 x2apic popcnt ida bogomips : 5868.53 clflush size : 64 power management: _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel