The EARLY_AP_STARTUP kernel option causes kernel panics on SUN X4200 AMD machines. COmmenting it out of GENERIC and new kernel allows boot. safe mode boot works because it disables smp. an svn bisect was performed for 11-stable and the addition of EARLY_AP_STARTUP caused the problem, version r318763 broken. --------------------------------------------------------------------------- /boot/kernel.old/kernel text=0x14972f8 data=0x1384c0+0x4c15e8 syms=[0x8+0x15e8b0+0x8+0x178422] /boot/entropy size=0x1000 Booting... Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-RELEASE #0 r321309: Fri Jul 21 02:08:28 UTC 2017 root at releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0) VT(vga): resolution 640x480 CPU: Dual-Core AMD Opteron(tm) Processor 2218 (2593.16-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x40f12 Family=0xf Model=0x41 Stepping=2 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x2001<SSE3,CX16> AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!> AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8> SVM: NAsids=64 real memory = 4563402752 (4352 MB) avail memory = 4104478720 (3914 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <SUN X4200 M2> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) random: unblocking device. ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe0Block: 64/32 (20170303/tbfadt-748) ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe1Block: 128/64 (20170303/tbfadt-748) ioapic1: Changing APIC ID to 16 ioapic2: Changing APIC ID to 17 ioapic3 <Version 1.1> irqs 24-47 on motherboard ioapic0 <Version 1.1> irqs 0-23 on motherboard ioapic1 <Version 1.1> irqs 48-54 on motherboard ioapic2 <Version 1.1> irqs 56-62 on motherboard SMP: AP CPU SMP: AP CPU k F aF aFat F aF kkernel trap 12 with interrupts disab?????????????????????????????kernel trap 12 with interrupts disabled Fatal trap 60276736: UN????kernel trap 12 with interrupts disabled iatal trap -2130508367: UNKNOWN whilpanierc: staeck ovlerflowt detrected; backt1race mawy bei corrup ted cpuid t= 1 KrDB: stack bupackttrace: #0 0xffffffffs80aada97 at lkdb_backtrace+0x67 xffffffff80a6bb76 aFat vpanic+0x1l86 #r2 0xfpffffff1f80a6b9e3 at panipc+0x43g #3e 0xffffaffff80a9b072u at __tstackw_chkh_lfail+0x12 i#4 0xffffff kff80eab3f2b eat vprintf+0 x10b 000 atcp dmapbase+0x397c000 Upt =ime: 12s Au;toma tic rebpoot inc 15 secondsi - press a key on 0the con sole fto abort -u-> Press a key on the iconsroule to reboot , -a-> or drswitech off the =system xnow. ke??????????????????kernel trap 12 with interrupts disabled Fatal trap 12: page fault while in kernel mode c?? c ------------------------------------------------------------------------------------------------ The corruption above is how it was on the screen. the memory detected is also wrong, or it is how much a single cpu sees. there are 12 gigs in this machine. Here is a boot from 11-stable with EARLY_AP_STARTUP commented out Copyright (c) 1992-2017 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.1-STABLE #11 r322170M: Tue Aug 8 13:13:21 EDT 2017 root at xxx:/usr/obj/usr/src/sys/GENERIC amd64 FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0) VT(vga): resolution 640x480 CPU: Dual-Core AMD Opteron(tm) Processor 2216 (2393.69-MHz K8-class CPU) Origin="AuthenticAMD" Id=0x40f13 Family=0xf Model=0x41 Stepping=3 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x2001<SSE3,CX16> AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!> AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8> SVM: NAsids=64 real memory = 12884901888 (12288 MB) avail memory = 12435623936 (11859 MB) Event timer "LAPIC" quality 100 ACPI APIC Table: <SUN X4200 M2> FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) random: unblocking device. ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe0Block: 64/32 (20170303/tbfadt-748) ACPI BIOS Warning (bug): 32/64X length mismatch in FADT/Gpe1Block: 128/64 (20170303/tbfadt-748) ioapic1: Changing APIC ID to 16 ioapic2: Changing APIC ID to 17 ioapic3 <Version 1.1> irqs 24-47 on motherboard ioapic0 <Version 1.1> irqs 0-23 on motherboard ioapic1 <Version 1.1> irqs 48-54 on motherboard ioapic2 <Version 1.1> irqs 56-62 on motherboard random: entropy device external interface kbd1 at kbdmux0 netmap: loaded module module_register_init: MOD_LOAD (vesa, 0xffffffff80f67f60, 0) error 19 nexus0 vtvga0: <VT VGA driver> on motherboard cryptosoft0: <software crypto> on motherboard acpi0: <SUN X4200 M2> on motherboard acpi0: Power Button (fixed) cpu0: <ACPI CPU> on acpi0 cpu1: <ACPI CPU> on acpi0 cpu2: <ACPI CPU> on acpi0 cpu3: <ACPI CPU> on acpi0 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed00fff on acpi0 Timecounter "HPET" frequency 25000000 Hz quality 950 hpet1: <High Precision Event Timer> iomem 0xfed10000-0xfed10fff on acpi0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x2008-0x200b on acpi0 pcib0: <ACPI Host-PCI bridge> on acpi0 pci0: <ACPI PCI bus> on pcib0 pci0: <memory> at device 0.0 (no driver attached) isab0: <PCI-ISA bridge> at device 1.0 on pci0 isa0: <ISA bus> on isab0 ohci0: <nVidia nForce CK804 USB Controller> mem 0xfe3ff000-0xfe3fffff irq 20 at device 2.0 on pci0 usbus0 on ohci0 ehci0: <NVIDIA nForce CK804 USB 2.0 controller> mem 0xfe3fec00-0xfe3fecff irq 21 at device 2.1 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci0 atapci0: <nVidia nForce CK804 UDMA133 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x9100-0x910f at device 6.0 on pci0 ata0: <ATA channel> at channel 0 on atapci0 ata1: <ATA channel> at channel 1 on atapci0 pcib1: <ACPI PCI-PCI bridge> at device 9.0 on pci0 pci1: <ACPI PCI bus> on pcib1 vgapci0: <VGA-compatible display> port 0xc800-0xc8ff mem 0xfd000000-0xfdffffff,0xfe2ff000-0xfe2fffff irq 16 at device 3.0 on pci1 vgapci0: Boot video device nfe0: <NVIDIA nForce4 CK804 MCP9 Networking Adapter> port 0xdc00-0xdc07 mem 0xfe3fd000-0xfe3fdfff irq 22 at device 10.0 on pci0 miibus0: <MII bus> on nfe0 e1000phy0: <Marvell 88E1111 Gigabit PHY> PHY 1 on miibus0 e1000phy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseSX, 1000baseSX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow nfe0: Using defaults for TSO: 65518/35/2048 nfe0: Ethernet address: pcib2: <ACPI PCI-PCI bridge> at device 11.0 on pci0 pci2: <ACPI PCI bus> on pcib2 pcib3: <ACPI PCI-PCI bridge> at device 12.0 on pci0 pci3: <ACPI PCI bus> on pcib3 pcib4: <ACPI PCI-PCI bridge> at device 13.0 on pci0 pci4: <ACPI PCI bus> on pcib4 pcib5: <ACPI PCI-PCI bridge> at device 14.0 on pci0 pci5: <ACPI PCI bus> on pcib5 pcib6: <ACPI Host-PCI bridge> on acpi0 pci6: <ACPI PCI bus> on pcib6 pci6: <memory> at device 0.0 (no driver attached) pci6: <memory> at device 1.0 (no driver attached) nfe1: <NVIDIA nForce4 CK804 MCP9 Networking Adapter> port 0xfc00-0xfc07 mem 0xfeafe000-0xfeafefff irq 44 at device 10.0 on pci6 miibus1: <MII bus> on nfe1 e1000phy1: <Marvell 88E1111 Gigabit PHY> PHY 1 on miibus1 e1000phy1: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseSX, 1000baseSX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow nfe1: Using defaults for TSO: 65518/35/2048 nfe1: Ethernet address: pcib7: <ACPI PCI-PCI bridge> at device 11.0 on pci6 pci7: <ACPI PCI bus> on pcib7 pcib8: <ACPI PCI-PCI bridge> at device 12.0 on pci6 pci8: <ACPI PCI bus> on pcib8 pcib9: <ACPI PCI-PCI bridge> at device 13.0 on pci6 pci9: <ACPI PCI bus> on pcib9 pcib10: <ACPI PCI-PCI bridge> at device 14.0 on pci6 pci10: <ACPI PCI bus> on pcib10 pcib11: <ACPI PCI-PCI bridge> at device 16.0 on pci6 pci11: <ACPI PCI bus> on pcib11 pcib12: <ACPI PCI-PCI bridge> at device 17.0 on pci6 pci12: <ACPI PCI bus> on pcib12 em0: <Intel(R) PRO/1000 Legacy Network Connection 1.1.0> port 0xec00-0xec3f mem 0xfe9e0000-0xfe9fffff irq 56 at device 1.0 on pci12 em0: Ethernet address: em0: netmap queues/slots: TX 1/4096, RX 1/4096 em1: <Intel(R) PRO/1000 Legacy Network Connection 1.1.0> port 0xe800-0xe83f mem 0xfe9c0000-0xfe9dffff irq 57 at device 1.1 on pci12 em1: Ethernet address: em1: netmap queues/slots: TX 1/4096, RX 1/4096 mpt0: <LSILogic SAS/SATA Adapter> port 0xe400-0xe4ff mem 0xfe9bc000-0xfe9bffff,0xfe9a0000-0xfe9affff irq 58 at device 2.0 on pci12 mpt0: MPI Version=1.5.16.0 mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 ) mpt0: 1 Active Volume (2 Max) mpt0: 2 Hidden Drive Members (14 Max) acpi_button0: <Power Button> on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: console (9600,n,8,1) orm0: <ISA Option ROMs> at iomem 0xc0000-0xc9fff,0xca000-0xcb7ff,0xcb800-0xcc7ff,0xcc800-0xcd7ff,0xd3800-0xd47ff on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: cannot reserve I/O port range powernow0: <PowerNow! K8> on cpu0 powernow1: <PowerNow! K8> on cpu1 powernow2: <PowerNow! K8> on cpu2 powernow3: <PowerNow! K8> on cpu3 Timecounters tick every 1.000 msec nvme cam probe device init usbus0: 12Mbps Full Speed USB v1.0 usbus1: 480Mbps High Speed USB v2.0 ugen0.1: <nVidia OHCI root HUB> at usbus0 uhub0: <nVidia OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen1.1: <nVidia EHCI root HUB> at usbus1 uhub1: <nVidia EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1 mpt0: mpt_read_cfg_page: Config Info Status 22 mpt0:vol0(mpt0:0:0): mpt_refresh_raid_vol: Failed to read RAID Vol Page(0) mpt0:vol0(mpt0:0:0): Settings ( ) mpt0:vol0(mpt0:0:0): 0 Members: mpt0:vol0(mpt0:0:0): uhub0: 7 ports with 7 removable, self powered RAID-0 - Optimal