CentOS - Aug 2016 - GRUB 2 dumps to grub prompt when installed on >4TB disk

----- Original Message -----
| On Thu, Aug 18, 2016 at 11:57 AM, James A. Peltier <jpeltier at sfu.ca>
wrote:
| > Hi All,
| >
| > I have a Dell R710 that has 6x1TB in a RAID-5 configuration.
| 
| 
| This is hardware RAID 5? Because it's pretty screwy how this ends up
| working when using software RAID and might take additional
| troubleshooting.

Yes, it's a Dell R710XD

| >  When installing CentOS 7 using the full disk capacity and booting in
UEFI
| >  mode the machine dumps me into a GRUB rescue mode prompt.
| >   error: disk `,gpt2' not found
| >   Entering rescue mode...
| >   grub rescue>
| 
| 
| This is confusing to me because there should be no such thing as grub
| rescue on UEFI. On BIOS systems, there is boot.img (formerly stage 1)
| and core.img in the MBR gap or on BIOS Boot if GPT disk (formerly
| stage 1.5 and stage 2). The core.img is where grub rescue comes from
| when it can't find grub modules, in particular normal.mod.
| 
| But on UEFI, core.img, normal.mod, and a pile of other modules are all
| baked into the grubx64.efi file founds on the EFI system partition.
| 
| I suspect two things that can cause normal.mod to not be found:
| a. The system is not in fact booting in UEFI mode and there's been
| some mistake in the installation of grub.
| b. The system is in UEFI mode, but either the installer, or
| post-install, grub2-install was run which obliterates the grub2-efi
| package installed grubx64.efi, i.e. it's not really proper to run
| grub2-install on UEFI systems.

I suspect this is the case.  when attempting to run grub-install the system
claims that the grub2-efi-modules packages aren't installed, so this may be
an installer bug.

| Boot off install media with boot parameter inst.rescue and choose all
| the default options; this ought to assemble the file system per fstab,
| and you can
| 
| chroot /mnt/sysimage
| yum reinstall grub2-efi
| efibootmgr -v
| grep efibootmgr /var/log/anaconda/program.log   ## I think that's
| right it might be anaconda.program.log though
| 
| 
| It's really just reinstalling grub2-efi that should fix the problem,
| the following two options are just information gathering in case the
| reboot still doesn't work.

We'll try this and get back to you soon.

-- 
James A. Peltier
IT Services - Research Computing Group
Simon Fraser University - Burnaby Campus
Phone   : 604-365-6432
Fax     : 778-782-3045
E-Mail  : jpeltier at sfu.ca
Website : http://www.sfu.ca/itservices
Twitter : @sfu_rcg
Powering Engagement Through Technology

On Fri, Aug 19, 2016 at 4:59 PM, James A. Peltier <jpeltier at sfu.ca>
wrote:
>
>
> ----- Original Message -----
> | On Thu, Aug 18, 2016 at 11:57 AM, James A. Peltier <jpeltier at
sfu.ca> wrote:
> | > Hi All,
> | >
> | > I have a Dell R710 that has 6x1TB in a RAID-5 configuration.
> |
> |
> | This is hardware RAID 5? Because it's pretty screwy how this ends up
> | working when using software RAID and might take additional
> | troubleshooting.
>
> Yes, it's a Dell R710XD
>
> | >  When installing CentOS 7 using the full disk capacity and booting
in UEFI
> | >  mode the machine dumps me into a GRUB rescue mode prompt.
> | >   error: disk `,gpt2' not found
> | >   Entering rescue mode...
> | >   grub rescue>
> |
> |
> | This is confusing to me because there should be no such thing as grub
> | rescue on UEFI. On BIOS systems, there is boot.img (formerly stage 1)
> | and core.img in the MBR gap or on BIOS Boot if GPT disk (formerly
> | stage 1.5 and stage 2). The core.img is where grub rescue comes from
> | when it can't find grub modules, in particular normal.mod.
> |
> | But on UEFI, core.img, normal.mod, and a pile of other modules are all
> | baked into the grubx64.efi file founds on the EFI system partition.
> |
> | I suspect two things that can cause normal.mod to not be found:
> | a. The system is not in fact booting in UEFI mode and there's been
> | some mistake in the installation of grub.
> | b. The system is in UEFI mode, but either the installer, or
> | post-install, grub2-install was run which obliterates the grub2-efi
> | package installed grubx64.efi, i.e. it's not really proper to run
> | grub2-install on UEFI systems.
>
> I suspect this is the case.  when attempting to run grub-install the system
claims that the grub2-efi-modules packages aren't installed, so this may be
an installer bug.
What is attempting to run grub-install? Or even grub2-install? If the
installer is doing this, it's an installer bug. If the user is doing
it, it's user error.

Also, you will need to check the NVRAM for stale values because
grub2-install also populates NVRAM with what will become the wrong
entry. You'll need to use 'efibootmgr -v' to get a listing to find
the
bogus entry, which will be pointing to a path that includes
grubx64.efi, note the boot number and then do 'efibootmgr -b <bootnum>
-B'   Where bootnum is the four digit value for the bogus entry.

What should happen if there are no valid entries is shim.efi will work
with fallback.efi to create a proper NVRAM entry. The proper entry can
be found with the earlier grep efibootmgr command, and you can just
use that, while adding an additional \ for each \, so that it's \\.
NVRAM should point to shim.efi and it's shim.efi that loads the
prebaked grubx64.efi.


-- 
Chris Murphy

----- Original Message -----
| On Fri, Aug 19, 2016 at 4:59 PM, James A. Peltier <jpeltier at sfu.ca>
wrote:
| >
| >
| > ----- Original Message -----
| > | On Thu, Aug 18, 2016 at 11:57 AM, James A. Peltier <jpeltier at
sfu.ca>
| > | wrote:
| > | > Hi All,
| > | >
| > | > I have a Dell R710 that has 6x1TB in a RAID-5 configuration.
| > |
| > |
| > | This is hardware RAID 5? Because it's pretty screwy how this ends
up
| > | working when using software RAID and might take additional
| > | troubleshooting.
| >
| > Yes, it's a Dell R710XD
| >
| > | >  When installing CentOS 7 using the full disk capacity and booting
in
| > | >  UEFI
| > | >  mode the machine dumps me into a GRUB rescue mode prompt.
| > | >   error: disk `,gpt2' not found
| > | >   Entering rescue mode...
| > | >   grub rescue>
| > |
| > |
| > | This is confusing to me because there should be no such thing as grub
| > | rescue on UEFI. On BIOS systems, there is boot.img (formerly stage 1)
| > | and core.img in the MBR gap or on BIOS Boot if GPT disk (formerly
| > | stage 1.5 and stage 2). The core.img is where grub rescue comes from
| > | when it can't find grub modules, in particular normal.mod.
| > |
| > | But on UEFI, core.img, normal.mod, and a pile of other modules are all
| > | baked into the grubx64.efi file founds on the EFI system partition.
| > |
| > | I suspect two things that can cause normal.mod to not be found:
| > | a. The system is not in fact booting in UEFI mode and there's been
| > | some mistake in the installation of grub.
| > | b. The system is in UEFI mode, but either the installer, or
| > | post-install, grub2-install was run which obliterates the grub2-efi
| > | package installed grubx64.efi, i.e. it's not really proper to run
| > | grub2-install on UEFI systems.
| >
| > I suspect this is the case.  when attempting to run grub-install the
system
| > claims that the grub2-efi-modules packages aren't installed, so this
may
| > be an installer bug.
| 
| What is attempting to run grub-install? Or even grub2-install? If the
| installer is doing this, it's an installer bug. If the user is doing
| it, it's user error.
| 
| Also, you will need to check the NVRAM for stale values because
| grub2-install also populates NVRAM with what will become the wrong
| entry. You'll need to use 'efibootmgr -v' to get a listing to find
the
| bogus entry, which will be pointing to a path that includes
| grubx64.efi, note the boot number and then do 'efibootmgr -b
<bootnum>
| -B'   Where bootnum is the four digit value for the bogus entry.
| 
| What should happen if there are no valid entries is shim.efi will work
| with fallback.efi to create a proper NVRAM entry. The proper entry can
| be found with the earlier grep efibootmgr command, and you can just
| use that, while adding an additional \ for each \, so that it's \\.
| NVRAM should point to shim.efi and it's shim.efi that loads the
| prebaked grubx64.efi.

When running grub2-install from within recovery mode I can assure you it is not
a user error because simply installing the grub2-efi-modules package allows for
grub2-install to work.  We will try the efibootmgr -v out too.


-- 
James A. Peltier
IT Services - Research Computing Group
Simon Fraser University - Burnaby Campus
Phone   : 604-365-6432
Fax     : 778-782-3045
E-Mail  : jpeltier at sfu.ca
Website : http://www.sfu.ca/itservices
Twitter : @sfu_rcg
Powering Engagement Through Technology