On Fri, Nov 25, 2016 at 1:25 AM, Konstantin Belousov <kostikbel at
gmail.com>
wrote:
> On Wed, Nov 02, 2016 at 06:28:08PM +0200, Konstantin Belousov wrote:
> > On Wed, Nov 02, 2016 at 09:18:15AM -0700, Jason Harmening wrote:
> > > I think you are probably right. Hacking out the Intel-specific
> > > additions to C-state parsing in acpi_cpu_cx_cst() from r282678
(thus
> > > going back to sti;hlt instead of monitor+mwait at C1) fixed the
problem
> > > for me. But r282678 also had the effect of enabling C2 and C3 on
my
> > > system, because ACPI only presents MWAIT entries for those states
and
> > > not p_lvlx.
> > You can do the same with "debug.acpi.disabled=mwait" loader
tunable
> > without hacking the code. And set sysctl hw.acpi.cpu.cx_lowest to C1
to
> > enforce use of hlt instruction even when mwait states were requested.
>
> I believe I now understood the problem. First, I got the definitive
> confirmation that LAPIC timer on Nehalems is stopped in any C mode
> higher than C1/C1E, i.e. even if C2 is enabled LAPIC eventtimer cannot
> be used. This is consistent with the ARAT CPUID bit CPUID[0x6].eax[2]
> reported zero.
>
> On SandyBridge and IvyBridge CPUs, it seems that ARAT might be both 0
> and 1 according to the same source, but all CPUs I saw have ARAT = 1.
> And for Haswell and later generations, ARAT is claimed to be always
> implemented.
>
> The actual issue is somewhat silly bug, I must admit: if ncpus >= 8, and
> non-FSB interrupt routing from HPET, default HPET eventtimer quality 450
> is reduced by 100, i.e. it is 350. OTOH, LAPIC default quality is 600
> and it is reduced by 200 if ARAT is not reported. We end up with HPET
> quality 350 < LAPIC quality 400, despite ARAT is not set.
>
> The patch below sets LAPIC eventtimer quality to 100 if not ARAT. Also
> I realized that there is no reason to disable deadline mode regardless
> of ARAT.
>
> diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c
> index d9a3453..1b1547d 100644
> --- a/sys/x86/x86/local_apic.c
> +++ b/sys/x86/x86/local_apic.c
> @@ -478,8 +478,9 @@ native_lapic_init(vm_paddr_t addr)
> lapic_et.et_quality = 600;
> if (!arat) {
> lapic_et.et_flags |= ET_FLAGS_C3STOP;
> - lapic_et.et_quality -= 200;
> - } else if ((cpu_feature & CPUID_TSC) != 0 &&
> + lapic_et.et_quality = 100;
> + }
> + if ((cpu_feature & CPUID_TSC) != 0 &&
> (cpu_feature2 & CPUID2_TSCDLT) != 0 &&
> tsc_is_invariant && tsc_freq != 0) {
> lapic_timer_tsc_deadline = 1;
>
> Ah, that makes sense. Thanks!
I'll try the patch as soon as I get back from vacation. I've been able
to
verify that setting cx_lowest and disabling mwait fixes the problem without
hacking the code. But I've been too busy at $(WORK) to check anything
else, namely whether forcing HPET would also fix the problem.