Xen devel - Apr 2006 - [PATCH] add support for XCHG instruction accessing APIC

32-bit SMP Linux uses xchg instruction to access APIC (see
apic_write_atomic()). Attached is patch that adds support for
this instruction.

The patch also fixes missed dependencies (in xen/Rules.mk) on
header files in include/asm-x86/hvm.

-boris


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 4 Apr 2006, at 22:24, Boris Ostrovsky wrote:
> 32-bit SMP Linux uses xchg instruction to access APIC (see
> apic_write_atomic()). Attached is patch that adds support for
> this instruction.
>
> The patch also fixes missed dependencies (in xen/Rules.mk) on
> header files in include/asm-x86/hvm.
I wonder if it is worth adding the locking? xchg is only used to write 
to broken APICs -- the OS does not care about the returned value. The 
APIC is implemented entirely in Xen, so the lock isn''t held across an 
ioreq round-trip to qemu-dm, right?

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5 Apr 2006, at 10:50, Keir Fraser wrote:
>> 32-bit SMP Linux uses xchg instruction to access APIC (see
>> apic_write_atomic()). Attached is patch that adds support for
>> this instruction.
>>
>> The patch also fixes missed dependencies (in xen/Rules.mk) on
>> header files in include/asm-x86/hvm.
>
> I wonder if it is worth adding the locking? xchg is only used to write 
> to broken APICs -- the OS does not care about the returned value. The 
> APIC is implemented entirely in Xen, so the lock isn''t held across
an
> ioreq round-trip to qemu-dm, right?
In fact I''m pretty sure the locking is not needed even if we did care 
about atomicity. You''re only protecting guest accesses from other guest
accesses, and each VCPU has its own local APIC model, so there cannot 
be multiple simultaneous guest accesses to a single local APIC.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
>
> On 5 Apr 2006, at 10:50, Keir Fraser wrote:
>
>>> 32-bit SMP Linux uses xchg instruction to access APIC (see
>>> apic_write_atomic()). Attached is patch that adds support for
>>> this instruction.
>>>
>>> The patch also fixes missed dependencies (in xen/Rules.mk) on
>>> header files in include/asm-x86/hvm.
>>
>>
>> I wonder if it is worth adding the locking? xchg is only used to 
>> write to broken APICs -- the OS does not care about the returned 
>> value. The APIC is implemented entirely in Xen, so the lock
isn''t
>> held across an ioreq round-trip to qemu-dm, right?
>
Yes, that''s right --- we never leave Xen to handle APIC access.
> In fact I''m pretty sure the locking is not needed even if we did
care
> about atomicity. You''re only protecting guest accesses from other 
> guest accesses, and each VCPU has its own local APIC model, so there 
> cannot be multiple simultaneous guest accesses to a single local APIC.

My only argument in favor of using the lock would be for completeness of 
the emulation. You are
absolutely right in that for Linux there seems to be no need to hold the 
lock. My concern is that
other OSs  may treat this differently. And if we don''t have sources, it
may be somewhat difficult
to  figure out that the atomicity (or lack of it) was the cause of a 
problem.

If, however, there is a strong feeling that we don''t need the lock, I
am
happy to drop it.
I guess you are mostly unhappy about adding a new field to hvm_domain, 
not about performance
impact?

-boris

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5 Apr 2006, at 15:01, Boris Ostrovsky wrote:
>> In fact I''m pretty sure the locking is not needed even if we
did care
>> about atomicity. You''re only protecting guest accesses from
other
>> guest accesses, and each VCPU has its own local APIC model, so there 
>> cannot be multiple simultaneous guest accesses to a single local 
>> APIC.
>
>
> My only argument in favor of using the lock would be for completeness 
> of the emulation. You are
> absolutely right in that for Linux there seems to be no need to hold 
> the lock. My concern is that
> other OSs  may treat this differently. And if we don''t have
sources,
> it may be somewhat difficult
> to  figure out that the atomicity (or lack of it) was the cause of a 
> problem.
>
> If, however, there is a strong feeling that we don''t need the
lock, I
> am happy to drop it.
> I guess you are mostly unhappy about adding a new field to hvm_domain, 
> not about performance
> impact?
Yes, also my second argument was that there is *no way* for two VCPUs 
to conflict on a local APIC access, since LAPIC accesses are always to 
the VCPU''s own LAPIC. So there is no potential concurrency that needs 
to be serialised, regardless of the guest OS.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
>>
>> My only argument in favor of using the lock would be for completeness 
>> of the emulation. You are
>> absolutely right in that for Linux there seems to be no need to hold 
>> the lock. My concern is that
>> other OSs  may treat this differently. And if we don''t have
sources,
>> it may be somewhat difficult
>> to  figure out that the atomicity (or lack of it) was the cause of a 
>> problem.
>>
>> If, however, there is a strong feeling that we don''t need the
lock, I
>> am happy to drop it.
>> I guess you are mostly unhappy about adding a new field to 
>> hvm_domain, not about performance
>> impact?
>
>
> Yes, also my second argument was that there is *no way* for two VCPUs 
> to conflict on a local APIC access, since LAPIC accesses are always to 
> the VCPU''s own LAPIC. So there is no potential concurrency that
needs
> to be serialised, regardless of the guest OS.

OK, that''s fair. Here is updated patch with lock removed.

I don''t think I then understand why Linux is using atomic accesses to
local APICs. It''s interesting though that 64-bit code doesn''t
do it ---
they use vanilla apic_write().

-boris


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 5 Apr 2006, at 15:26, Boris Ostrovsky wrote:
> OK, that''s fair. Here is updated patch with lock removed.
Thanks.
> I don''t think I then understand why Linux is using atomic accesses
to
> local APICs. It''s interesting though that 64-bit code
doesn''t do it ---
> they use vanilla apic_write().
The atomic accesses are to work around APIC bugs in the original 
Pentium processors, where an APIC write in some cases needs to be 
preceded by a read for the write to work correctly.

The XCHG trick isn''t needed on x86_64 because only modern processors 
support 64-bit mode and none of them have a buggy LAPIC.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel