I can confirm this patch works. HPET is now chosen over LAPIC as the
eventtimer source, and the system works smoothly without disabling C2 or
mwait.
On Fri, Nov 25, 2016 at 4:12 AM, Jason Harmening <jason.harmening at
gmail.com>
wrote:
> On Fri, Nov 25, 2016 at 1:25 AM, Konstantin Belousov <kostikbel at
gmail.com>
> wrote:
>
>> On Wed, Nov 02, 2016 at 06:28:08PM +0200, Konstantin Belousov wrote:
>> > On Wed, Nov 02, 2016 at 09:18:15AM -0700, Jason Harmening wrote:
>> > > I think you are probably right. Hacking out the
Intel-specific
>> > > additions to C-state parsing in acpi_cpu_cx_cst() from
r282678 (thus
>> > > going back to sti;hlt instead of monitor+mwait at C1) fixed
the
>> problem
>> > > for me. But r282678 also had the effect of enabling C2 and
C3 on my
>> > > system, because ACPI only presents MWAIT entries for those
states and
>> > > not p_lvlx.
>> > You can do the same with "debug.acpi.disabled=mwait"
loader tunable
>> > without hacking the code. And set sysctl hw.acpi.cpu.cx_lowest to
C1 to
>> > enforce use of hlt instruction even when mwait states were
requested.
>>
>> I believe I now understood the problem. First, I got the definitive
>> confirmation that LAPIC timer on Nehalems is stopped in any C mode
>> higher than C1/C1E, i.e. even if C2 is enabled LAPIC eventtimer cannot
>> be used. This is consistent with the ARAT CPUID bit CPUID[0x6].eax[2]
>> reported zero.
>>
>> On SandyBridge and IvyBridge CPUs, it seems that ARAT might be both 0
>> and 1 according to the same source, but all CPUs I saw have ARAT = 1.
>> And for Haswell and later generations, ARAT is claimed to be always
>> implemented.
>>
>> The actual issue is somewhat silly bug, I must admit: if ncpus >= 8,
and
>> non-FSB interrupt routing from HPET, default HPET eventtimer quality
450
>> is reduced by 100, i.e. it is 350. OTOH, LAPIC default quality is 600
>> and it is reduced by 200 if ARAT is not reported. We end up with HPET
>> quality 350 < LAPIC quality 400, despite ARAT is not set.
>>
>> The patch below sets LAPIC eventtimer quality to 100 if not ARAT. Also
>> I realized that there is no reason to disable deadline mode regardless
>> of ARAT.
>>
>> diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
>> index d9a3453..1b1547d 100644
>> --- a/sys/x86/x86/local_apic.c
>> +++ b/sys/x86/x86/local_apic.c
>> @@ -478,8 +478,9 @@ native_lapic_init(vm_paddr_t addr)
>> lapic_et.et_quality = 600;
>> if (!arat) {
>> lapic_et.et_flags |= ET_FLAGS_C3STOP;
>> - lapic_et.et_quality -= 200;
>> - } else if ((cpu_feature & CPUID_TSC) != 0
&&
>> + lapic_et.et_quality = 100;
>> + }
>> + if ((cpu_feature & CPUID_TSC) != 0 &&
>> (cpu_feature2 & CPUID2_TSCDLT) != 0 &&
>> tsc_is_invariant && tsc_freq != 0) {
>> lapic_timer_tsc_deadline = 1;
>>
>> Ah, that makes sense. Thanks!
>
> I'll try the patch as soon as I get back from vacation. I've been
able to
> verify that setting cx_lowest and disabling mwait fixes the problem without
> hacking the code. But I've been too busy at $(WORK) to check anything
> else, namely whether forcing HPET would also fix the problem.
>
>