Xen devel - Jun 2011 - INIT-SIPI-SIPI timing and CPU bringup on new Intel CPUs

Hi, 

I''ve come across the problem reported in this thread:
http://lists.xensource.com/archives/html/xen-devel/2011-02/msg00702.html
where on a very new Intel system some of the APs don''t respond when the
BSP tries to wake them.

I''ve found that removing the delays between the IPIs makes this problem
go away.  I''m hoping someone from Intel can help me understand why, and
whether that''s a sensible answer.  Since this is an x2apic system I
guess we don''t have to wait for the IPI to be acked by the sending
APIC,
but the SDMs still recommend delaying between INIT, SIPI and SIPI, and I
can imagine I might be breaking some other systems by doing this.

So the questions I have are:
 - is it safe always to issue INIT-SIPI-SIPI back-to-back on x2apic systems?
 - is there any reason why the delay should make a difference (except 
   maybe that this is an early version of a chip?)

My patch is attached; it removes the delays only in the x2apic case,
so shouldn''t break any older systems.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 16:31 +0100 on 23 Jun (1308846702), Tim Deegan wrote:
> I''ve found that removing the delays between the IPIs makes this
problem
> go away.  I''m hoping someone from Intel can help me understand
why, and
> whether that''s a sensible answer.  Since this is an x2apic system
I
> guess we don''t have to wait for the IPI to be acked by the sending
APIC,
> but the SDMs still recommend delaying between INIT, SIPI and SIPI, and I
> can imagine I might be breaking some other systems by doing this.
> 
> So the questions I have are:
>  - is it safe always to issue INIT-SIPI-SIPI back-to-back on x2apic
systems?
>  - is there any reason why the delay should make a difference (except 
>    maybe that this is an early version of a chip?)
> 
> My patch is attached; it removes the delays only in the x2apic case,
> so shouldn''t break any older systems.
We''ve tested this on the XenServer automated test pools for a couple of
weeks and not seen any ill effects.  In the absence of any negative
feedback, I propose that we apply this patch to xen-unstable.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel