Xen devel - Sep 2009 - SMP boot time-out with PCI Passthrough

I am running a Linux 2.6.30.1 hvm guest and the boot cpu is timing out waiting
for a secondary cpu to initialize.  The timeout is coded to 5 seconds.  In this
time the AP disables and re-enables cpu caches by writing to cr0.  In dom0, I
see hvm_set_cr0 take about .8 seconds per GB of memory allocated to the guest
OS.  So if the memory is below about 3 GB (5sec/1.6sec per GB), the
initialization completes.  If greater than that, the boot cpu gives up.

This only happens with a passthrough device defined in the guest config file
with pci=...

I''ve tracked it down so far to the ept_change_entry_emt_with_range
function taking the majority of the time.  Since this function is not executed
if has_arch_pdevs(v->domain) returns zero, it explains why it only happens
with a passthrough device.

Is the .8 seconds per GB an expected execution time or is this a known problem I
missed in my searches?

My configuration is:

Intel Xeon E5506 @ 2.1GHz  L1=32k/32k, L2=256k, L3-4096k Quad core  12GB memory
Dom0=CentOS 5.2/Xen 3.4-unstable


Thanks,  Jerry


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 02/09/2009 14:29, "Fredin, Jerry" <Jerry.Fredin@lsi.com>
wrote:
> I¹ve tracked it down so far to the ept_change_entry_emt_with_range function
> taking the majority of the time.  Since this function is not executed if
> has_arch_pdevs(v->domain) returns zero, it explains why it only happens
with a
> passthrough device.
>  
> Is the .8 seconds per GB an expected execution time or is this a known
problem
> I missed in my searches?
I doubt the function was implemented with performance in mind at all. The
fact it has cost proportional to memory allocated to guest is not great of
course. You could work around this quite satisfactorily I''m sure, by
configuring hap=0 in your domain config file. Shadow page tables work pretty
well, and won''t use this costly function at all.

 -- Keir





_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel