On Thu, Sep 01, 2016 at 02:45:00PM +0300, Konstantin Belousov wrote:
> On Thu, Sep 01, 2016 at 02:27:24PM +0300, Slawa Olhovchenkov wrote:
> > On Sat, Dec 12, 2015 at 03:35:13PM +0200, Konstantin Belousov wrote:
> >
> > > On Sat, Dec 12, 2015 at 04:06:15PM +0300, Slawa Olhovchenkov
wrote:
> > > > Does STABLE support X2APIC?
> > > > I see X2APIC related commits in CURRENT, what is status for
STABLE?
> > > > I am try to enable X2APIC support on X10DRi and see kernel
trap on
> > > > boot.
> > >
> > > x2APIC is only supported in HEAD. The code to parse processor
x2APIC
> > > structures in MADT was committed, but AFAIK never tested on real
hardware.
> > >
> > > There are no plans to merge this to stable/10.
> >
> > I am try now on real hardware and got kernel fault at boot when set in
> > BIOS X2APIC_OPT_OUT Flag to [Enable].
> > https://s9.postimg.org/6mt8bu21b/fault2.jpg
> How do you know that the issue is due to that flag ?
Just by turn on and off this flag and try boot at both cases.
> I doubt that BIOS would provide direct knob to manage that flag, since
requesting that
> out-out only makes sense if BIOS is unable to handle x2APIC mode, e.g.
> in SMI handlers. How precisely the BIOS setting called which you frobbed ?
As I point before: 'X2APIC_OPT_OUT Flag'.
Hidden before 'X2APIC' set to '[Enable]'
https://s15.postimg.org/3k5l7z397/bios_x2.jpg
> Show complete verbose dmesg from the first line to backtrace.
Booting...
\01;00Table 'FACP' at 0x79b189a8
Table 'APIC' at 0x79b18ab8
APIC: Found table at 0x79b18ab8
APIC: Using the MADT enumerator.
MADT: Found CPU APIC ID 0 ACPI ID 0: enabled
SMP: Added CPU 0 (AP)
MADT: Found CPU APIC ID 2 ACPI ID 2: enabled
SMP: Added CPU 2 (AP)
MADT: Found CPU APIC ID 4 ACPI ID 4: enabled
SMP: Added CPU 4 (AP)
MADT: Found CPU APIC ID 6 ACPI ID 6: enabled
SMP: Added CPU 6 (AP)
MADT: Found CPU APIC ID 8 ACPI ID 8: enabled
SMP: Added CPU 8 (AP)
MADT: Found CPU APIC ID 10 ACPI ID 10: enabled
SMP: Added CPU 10 (AP)
MADT: Found CPU APIC ID 16 ACPI ID 16: enabled
SMP: Added CPU 16 (AP)
MADT: Found CPU APIC ID 18 ACPI ID 18: enabled
SMP: Added CPU 18 (AP)
MADT: Found CPU APIC ID 20 ACPI ID 20: enabled
SMP: Added CPU 20 (AP)
MADT: Found CPU APIC ID 22 ACPI ID 22: enabled
SMP: Added CPU 22 (AP)
MADT: Found CPU APIC ID 24 ACPI ID 24: enabled
SMP: Added CPU 24 (AP)
MADT: Found CPU APIC ID 26 ACPI ID 26: enabled
SMP: Added CPU 26 (AP)
MADT: Found CPU APIC ID 32 ACPI ID 32: enabled
SMP: Added CPU 32 (AP)
MADT: Found CPU APIC ID 34 ACPI ID 34: enabled
SMP: Added CPU 34 (AP)
MADT: Found CPU APIC ID 36 ACPI ID 36: enabled
SMP: Added CPU 36 (AP)
MADT: Found CPU APIC ID 38 ACPI ID 38: enabled
SMP: Added CPU 38 (AP)
MADT: Found CPU APIC ID 40 ACPI ID 40: enabled
SMP: Added CPU 40 (AP)
MADT: Found CPU APIC ID 42 ACPI ID 42: enabled
SMP: Added CPU 42 (AP)
MADT: Found CPU APIC ID 48 ACPI ID 48: enabled
SMP: Added CPU 48 (AP)
MADT: Found CPU APIC ID 50 ACPI ID 50: enabled
SMP: Added CPU 50 (AP)
MADT: Found CPU APIC ID 52 ACPI ID 52: enabled
SMP: Added CPU 52 (AP)
MADT: Found CPU APIC ID 54 ACPI ID 54: enabled
SMP: Added CPU 54 (AP)
MADT: Found CPU APIC ID 56 ACPI ID 56: enabled
SMP: Added CPU 56 (AP)
MADT: Found CPU APIC ID 58 ACPI ID 58: enabled
SMP: Added CPU 58 (AP)
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-RELEASE-p305117 #0: Wed Aug 31 17:58:35 UTC 2016
slw at storage01.int.integros.com:/usr/obj/usr/src/sys/VSTREAM amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM
3.8.0)
Table 'FACP' at 0x79b189a8
Table 'APIC' at 0x79b18ab8
Table 'FPDT' at 0x79b18c70
Table 'FIDT' at 0x79b18cb8
Table 'SPMI' at 0x79b18d58
Table 'MCFG' at 0x79b18da0
Table 'UEFI' at 0x79b18de0
Table 'MCEJ' at 0x79b18e28
Table 'HPET' at 0x79b18f58
Table 'WDDT' at 0x79b18f90
Table 'SSDT' at 0x79b18fd0
Table 'SSDT' at 0x79b30020
Table 'SSDT' at 0x79b32670
Table 'PRAD' at 0x79b326d8
Table 'DMAR' at 0x79b32798
Table 'HEST' at 0x79b328e8
Table 'BERT' at 0x79b32990
Table 'ERST' at 0x79b329c0
Table 'EINJ' at 0x79b32bf0
ACPI: No SRAT table found
PPIM 0: PA=0xb8000, VA=0xffffffff81610000, size=0x8000, mode=0
VT(vga): text 80x25
Preloaded elf kernel "/boot/kernel.VSTREAM/kernel" at
0xffffffff814a8000.
Preloaded /boot/entropy "/boot/entropy" at 0xffffffff814a8ac0.
Preloaded elf obj module "/boot/kernel.VSTREAM/zfs.ko" at
0xffffffff814a8b10.
Preloaded elf obj module "/boot/kernel.VSTREAM/krpc.ko" at
0xffffffff814a9300.
Preloaded elf obj module "/boot/kernel.VSTREAM/opensolaris.ko" at
0xffffffff814a9970.
Preloaded elf obj module "/boot/kernel.VSTREAM/if_igb.ko" at
0xffffffff814a9fa8.
Preloaded elf obj module "/boot/kernel.VSTREAM/if_lagg.ko" at
0xffffffff814aa618.
Preloaded elf obj module "/boot/kernel.VSTREAM/ukbd.ko" at
0xffffffff814aacc8.
Preloaded elf obj module "/boot/kernel.VSTREAM/usb.ko" at
0xffffffff814ab378.
Preloaded elf obj module "/boot/kernel.VSTREAM/umass.ko" at
0xffffffff814abaa8.
Preloaded elf obj module "/boot/kernel.VSTREAM/accf_http.ko" at
0xffffffff814ac098.
Preloaded elf obj module "/boot/kernel.VSTREAM/cc_htcp.ko" at
0xffffffff814ac610.
Preloaded elf obj module "/boot/kernel.VSTREAM/uhci.ko" at
0xffffffff814acc40.
Preloaded elf obj module "/boot/kernel.VSTREAM/ohci.ko" at
0xffffffff814ad1f0.
Preloaded elf obj module "/boot/kernel.VSTREAM/ehci.ko" at
0xffffffff814ad7a0.
Preloaded elf obj module "/boot/kernel.VSTREAM/xhci.ko" at
0xffffffff814add50.
Calibrating TSC clock ... TSC clock: 2200043851 Hz
CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2200.04-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x406f1 Family=0x6 Model=0x4f
Stepping=1
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7ffefbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended
Features=0x21cbfbb<FSGSBASE,TSCADJ,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,PQM,NFPUSG,PQE,RDSEED,ADX,SMAP,PROCTRACE>
XSAVE Features=0x1<XSAVEOPT>
VT-x: Basic Features=0xda0400<SMM,INS/OUTS,TRUE>
Pin-Based Controls=0xff<ExtINT,NMI,VNMI,PreTmr,PostIntr>
Primary Processor
Controls=0xfff9fffe<INTWIN,TSCOff,HLT,INVLPG,MWAIT,RDPMC,RDTSC,CR3-LD,CR3-ST,CR8-LD,CR8-ST,TPR,NMIWIN,MOV-DR,IO,IOmap,MTF,MSRmap,MONITOR,PAUSE>
Secondary Processor
Controls=0x77fff<APIC,EPT,DT,RDTSCP,x2APIC,VPID,WBINVD,UG,APIC-reg,VID,PAUSE-loop,RDRAND,INVPCID,VMFUNC,VMCS,XSAVES>
Exit Controls=0xda0400<PAT-LD,EFER-SV,PTMR-SV>
Entry Controls=0xda0400
EPT Features=0x6334141<XO,PW4,UC,WB,2M,1G,INVEPT,AD,single,all>
VPID Features=0xf01<INVVPID,individual,single,all,single-globals>
TSC: P-state invariant, performance statistics
Data TLB: 2 MByte or 4 MByte pages, 4-way set associative, 32 entries and a
separate array with 1 GByte pages, 4-way set associative, 4 entries
Data TLB: 4 KB pages, 4-way set associative, 64 entries
Instruction TLB: 2M/4M pages, fully associative, 8 entries
Instruction TLB: 4KByte pages, 8-way set associative, 128 entries
64-Byte prefetching
Shared 2nd-Level TLB: 4 KByte /2 MByte pages, 6-way associative, 1536 entries.
Also 1GBbyte pages, 4-way, 16 entries
L2 cache: 256 kbytes, 8-way associative, 64 bytes/line
real memory = 137438953472 (131072 MB)
Physical memory chunk(s):
0x0000000000010000 - 0x0000000000095fff, 548864 bytes (134 pages)
0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages)
0x00000000014e9000 - 0x000000007918bfff, 2009739264 bytes (490659 pages)
0x0000000100000000 - 0x0000001fabf78fff, 131734147072 bytes (32161657 pages)
avail memory = 133408907264 (127228 MB)
Table 'FACP' at 0x79b189a8
Table 'APIC' at 0x79b18ab8
Table 'FPDT' at 0x79b18c70
Table 'FIDT' at 0x79b18cb8
Table 'SPMI' at 0x79b18d58
Table 'MCFG' at 0x79b18da0
Table 'UEFI' at 0x79b18de0
Table 'MCEJ' at 0x79b18e28
Table 'HPET' at 0x79b18f58
Table 'WDDT' at 0x79b18f90
Table 'SSDT' at 0x79b18fd0
Table 'SSDT' at 0x79b30020
Table 'SSDT' at 0x79b32670
Table 'PRAD' at 0x79b326d8
Table 'DMAR' at 0x79b32798
DMAR: Found table at 0x79b32798
x2APIC available but disabled by DMAR table
Event timer "LAPIC" quality 600
LAPIC: ipi_wait() us multiplier 1 (r 116268019 tsc 2200043851)
ACPI APIC Table: <ALASKA A M I >
Package ID shift: 5
L3 cache ID shift: 5
L2 cache ID shift: 1
L1 cache ID shift: 1
Core ID shift: 1
kernel trap 12 with interrupts disabled
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = ff
fault virtual address = 0x0
fault code = supervisor read data, page not present
instruction pointer = 0x20:0xffffffff80537e74
stack pointer = 0x28:0xffffffff814b4a60
frame pointer = 0x28:0xffffffff814b4a70
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags = resume, IOPL = 0
current process = 0 ()
trap number = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff805272e7 at kdb_backtrace+0x67
#1 0xffffffff804dd662 at vpanic+0x182
#2 0xffffffff804dd4d3 at panic+0x43
#3 0xffffffff807a3791 at trap_fatal+0x351
#4 0xffffffff807a3983 at trap_pfault+0x1e3
#5 0xffffffff807a2f0c at trap+0x26c
#6 0xffffffff80787ca1 at calltrap+0x8
#7 0xffffffff8083b52a at topo_probe+0x61a
#8 0xffffffff8078fe81 at cpu_mp_start+0x1b1
#9 0xffffffff805382ca at mp_start+0x3a
#10 0xffffffff80465cd8 at mi_startup+0x118
#11 0xffffffff8028dfac at btext+0x2c
Uptime: 1s
> >
> > 0xffffffff80537e74 is /usr/src/sys/kern/subr_smp.c:959
> And this is a topology probe code, which should not be affected by the
> x2APIC mode at all, except possibly due to longer format of the APIC IDs.
>
> I added Andrey to Cc:, he might give some more guidance.
> >
> > Boot success if X2APIC_OPT_OUT Flag set to [Disable]
> > Supermicro X10DRi, dual E5-2650v4