freebsd stable - Feb 2021 - Microcode update prevents boot

On Sun, Feb 14, 2021 at 02:01:14PM +0100, Leon Dietrich
wrote:
> Hi there,
> 
> I already worked around the issue myself. I'm just writing this here in
> case someone else may have the same issue and is seeking an answer.
> 
> 
> I recently upgraded the intel cpu microcode update package. Since then
> the boot process hang at the stage where the other cpu cores where
> enabled (shortly after enabling acpi). In order to resolve the issue one
> has to boot in safe mode (not single user mode!) and comment (or remove)
> the lines enabling the cpu microcode update on boot in
> /boot/loader.conf. One can and should reboot then.
> 
> After making these changes the system boots again and all cores are
> started and SMT works as well. One should note that one's not running
> the newer microcode (including some security-) fixes. Having
> microcode_update_enable="YES" in /etc/rc.conf doesn't prevent
booting
> and does not cause noticeable instability.
> 
> For reference: Im running FreeBSD 12.1 on a supermicro embedded board
> with intel xeon E3-1585L v5 cpus.
> 
> 
> I hope someone will find this info useful.
I see that r347931 was not merged to stable/12 branch, but the lockless
delayed invalidation changes were indeed present in 12.1.  Could you see
if the hang persists when boot-time ucode loading is enabled and
vm.pmap.di_locked=1 is configured?  Note that you could apply both
configurations at the loader prompt, i.e., without having to edit
loader.conf and boot in safe mode to revert the change.

On Sun, Feb 14, 2021 at 07:16:29PM -0500, Mark Johnston
wrote:
> On Sun, Feb 14, 2021 at 02:01:14PM +0100, Leon Dietrich wrote:
> > Hi there,
> > 
> > I already worked around the issue myself. I'm just writing this
here in
> > case someone else may have the same issue and is seeking an answer.
> > 
> > 
> > I recently upgraded the intel cpu microcode update package. Since then
> > the boot process hang at the stage where the other cpu cores where
> > enabled (shortly after enabling acpi). In order to resolve the issue
one
> > has to boot in safe mode (not single user mode!) and comment (or
remove)
> > the lines enabling the cpu microcode update on boot in
> > /boot/loader.conf. One can and should reboot then.
> > 
> > After making these changes the system boots again and all cores are
> > started and SMT works as well. One should note that one's not
running
> > the newer microcode (including some security-) fixes. Having
> > microcode_update_enable="YES" in /etc/rc.conf doesn't
prevent booting
> > and does not cause noticeable instability.
> > 
> > For reference: Im running FreeBSD 12.1 on a supermicro embedded board
> > with intel xeon E3-1585L v5 cpus.
> > 
> > 
> > I hope someone will find this info useful.
> 
> I see that r347931 was not merged to stable/12 branch, but the lockless
> delayed invalidation changes were indeed present in 12.1.  Could you see
> if the hang persists when boot-time ucode loading is enabled and
> vm.pmap.di_locked=1 is configured?  Note that you could apply both
> configurations at the loader prompt, i.e., without having to edit
> loader.conf and boot in safe mode to revert the change.
Please check that this patch helps:

commit c0faf2999bfaad2fdcead26d59d60c9b9e01988a
Author: Konstantin Belousov <kib at FreeBSD.org>
Date:   Fri May 17 17:11:01 2019 +0000

    Free microcode memory later.
    
    (cherry picked from commit 8f7f38457f940798c149ae40b73e0d20672812de)

diff --git a/sys/x86/x86/ucode.c b/sys/x86/x86/ucode.c
index 93f82e37eb66..d8beeed68215 100644
--- a/sys/x86/x86/ucode.c
+++ b/sys/x86/x86/ucode.c
@@ -260,7 +260,7 @@ ucode_release(void *arg __unused)
 		goto restart;
 	}
 }
-SYSINIT(ucode_release, SI_SUB_KMEM + 1, SI_ORDER_ANY, ucode_release, NULL);
+SYSINIT(ucode_release, SI_SUB_SMP + 1, SI_ORDER_ANY, ucode_release, NULL);
 
 void
 ucode_load_ap(int cpu)