CentOS - Aug 2020 - 8.2.2004 Latest yum update renders machine unbootable

Hi Johnny,
thank you very much for clarification.

You said that in the centos infrastructure only one server got the problem.
What are the conditions that permit the breakage? There is a particular
configuration (hw/sw) case that match always the problem or it is random?

Thank you

Il Sab 1 Ago 2020, 05:20 Johnny Hughes <johnny at centos.org> ha scritto:
> On 7/31/20 11:24 AM, Alessandro Baggi wrote:
> >
> > Il 31/07/20 13:08, ja ha scritto:
> >> On Fri, 2020-07-31 at 22:35 +1200, Alan McRae via CentOS wrote:
> >>> I am running an Intel x64 machine using UEFI to boot an SSD.
> >>>
> >>> Installing the latest yum update which includes grub2 and
kernel
> >>> 4.18.0-193.14.2.el8_2.x86_64 renders the machine unbootable,
blank
> >>> screen where grub should be, no error messages, just hangs.
> >>>
> >>> After some hours I managed to modify another bootable
partition
> >>> (containing older software) and boot it from there.
> >>>
> >>> After that, I  found out it is a known problem.
> >>>
> >>> The main point of this message is to make people aware of the
problem
> >>> and suggest admins don't run 'yum update' until
they understand the
> >>> problem and have a fix at hand.
> >>>
> >>> See 'UEFI boot blank screen post update' for a
solution and directions
> >>> to the redhat article.
> >>>
> >>> Regards
> >>>
> >>> Alan
> >>>
> >> I have been punished by this bug - it is/was very nasty.
> >>
> > Me too. Luckily it happened on a test machine.
> >
> > Sorry but seems that those packages were not tested before pushing
them
> > in the update repo. Would be great to know what happened to the
> > mainstream chains and how a package like grub reached the update repo
> > when it has serious problem (genuine curiosity but not to blame them).
>
> Of course it was tested before it was pushed. Obviously this is not a
> problem with every install.  Surely you don't think we push items
> without doing any testing.  Certainly not items as important as this
> update.
>
>  The CentOS infrastructure has hundreds of servers, most of them were
> not impacted (as an example). In fact, we seem to have had this happen
> on only one machine in those hundreds so far.  It is a problem, obviously.
>
> The issue seems to be with the shim package (not the grub or kernel
> packages) and we are currently working with Red Hat on a fix.  This
> issue happened in many Linux OSes and even Windows, not just RHEL and
> CentOS.
>
> We will push a fix as soon as one is available.
>
> I would hold off on installing this until we release the new fixes.
>
> >
> > In my case, the restore procedure reported by RH in case of reboot
does
> > not work and reports that the packages are already at the lowest
version
> > and that the downgrade is not possibile. I don't know why.
>
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos
>

At 02:54 AM 8/1/2020, Alessandro Baggi wrote:
>Hi Johnny,
>thank you very much for clarification.
>
>You said that in the centos infrastructure only one server got the problem.
>What are the conditions that permit the breakage? There is a particular
>configuration (hw/sw) case that match always the problem or it is random?
>
>Thank you
I have two servers running Centos 7 on apple hardware (one mac-mini 
and one mac server).  They both failed to reboot a few days ago.  So 
perhaps whatever anti-boot bug hit Centos 8, also hit Centos 7.  I 
can't tell what version got updated since the system simply fails to 
boot.  I don't even get a grub screen.  I'll have to rebuild the 
systems from scratch.

David

On 8/1/20 5:54 AM, Alessandro Baggi wrote:
> You said that in the centos infrastructure only one server got the problem.
> What are the conditions that permit the breakage? There is a particular
> configuration (hw/sw) case that match always the problem or it is random?
I experienced the issue on a Dell Precision M6700 (UEFI, Core i7-3740QM) 
on the first attempt at update; I recovered by rolling back and 
reinstalling grub2-*; I then updated on the command line with dnf update 
and now that same hardware that crashed out the first time is working 
fine with the same package versions.? It seems to be intermittent, and I 
can't reproduce it.

On 8/1/20 6:56 AM, david wrote:
> At 02:54 AM 8/1/2020, Alessandro Baggi wrote:
>> Hi Johnny,
>> thank you very much for clarification.
>>
>> You said that in the centos infrastructure only one server got the 
>> problem.
>> What are the conditions that permit the breakage? There is a particular
>> configuration (hw/sw) case that match always the problem or it is 
>> random?
>>
>> Thank you
>
> I have two servers running Centos 7 on apple hardware (one mac-mini 
> and one mac server).? They both failed to reboot a few days ago.? So 
> perhaps whatever anti-boot bug hit Centos 8, also hit Centos 7.? I 
> can't tell what version got updated since the system simply fails to 
> boot.? I don't even get a grub screen. I'll have to rebuild the 
> systems from scratch.
>
>
You should be able to boot off of installation media into rescue mode, 
and downgrade the grub2* and/or shim* RPMs.

-Greg