CentOS - Sep 2020 - Server entering Emergency Shell, but continues fine after pressing Enter

On 9/11/20 5:29 PM, Quinn Comendant wrote:
> Those have always reported success (even before I removed the OnFailure
option):

In that case, I'd revert the change you made, unlock the root account so 
that you can use the emergency shell, let the system boot to an 
emergency shell, and collect the output of "systemctl status 
initrd-parse-etc.service" and "journalctl -b 0".

(You can still do that in the VM, right?)

On 12 Sep 2020 11:40:48, Gordon Messmer wrote:
> In that case, I'd revert the change you made, unlock the root account 
> so that you can use the emergency shell, let the system boot to an 
> emergency shell, and collect the output of "systemctl status 
> initrd-parse-etc.service" and "journalctl -b 0".
Ok, I was able to log in as root in the emergency shell.

I don't see any errors from `systemctl status initrd-parse-etc.service` or
 `journalctl -b 0` (I've pasted the full output here:
https://write.as/at21opjv3o9fin1t.txt)

However, if I run `systemctl list-units --failed` it says
initrd-switch-root.service is failed. Status of that service:

[root at myhost ~] systemctl status initrd-switch-root.service
? initrd-switch-root.service - Switch Root
   Loaded: loaded (/usr/lib/systemd/system/initrd-switch-root.service; static;
vendor preset: disabled)
   Active: failed (Result: signal) since Sat 2020-09-12 19:41:13 UTC; 17min ago
  Process: 204 ExecStart=/usr/bin/systemctl --no-block --force switch-root
/sysroot (code=killed, signal=TERM)
 Main PID: 204 (code=killed, signal=TERM)
Sep 12 19:41:13 durian systemd[1]: Starting Switch Root...

But the logs don't show any errors:

[root at myhost ~] journalctl -u  initrd-switch-root.service
-- Logs begin at Sat 2020-09-12 19:41:07 UTC, end at Sat 2020-09-12 19:41:19 UTC
Sep 12 19:41:13 durian systemd[1]: Starting Switch Root...

That's it ? just "Starting Switch Root" and nothing more? ?

Thanks,
Quinn

On 9/12/20 1:04 PM, Quinn Comendant wrote:
> I don't see any errors from `systemctl status initrd-parse-etc.service`
or
>   `journalctl -b 0` (I've pasted the full output
here:https://write.as/at21opjv3o9fin1t.txt)

I see errors in the journalctl output.? Look into these:

Sep 12 19:41:12 myhost systemd-vconsole-setup[84]: /usr/bin/setfont failed with
error code 71.
Sep 12 19:41:12 myhost systemd-vconsole-setup[84]: ...................
Sep 12 19:41:12 myhost systemd-vconsole-setup[84]: setfont: putfont: 512,8x16: 
failed: -1
Sep 12 19:41:12 myhost systemd-vconsole-setup[84]: putfont: PIO_FONT: Invalid
argument
Sep 12 19:41:12 myhost systemd-vconsole-setup[117]: /usr/bin/setfont failed with
error code 71.

Sep 12 19:41:13 myhost systemd[1]: [/etc/systemd/system.conf:15] Unknown lvalue
'TimeoutSec' in section 'Manager'

The former errors might be normal on a system with no VGA, and you 
mentioned seeing this over serial.? Do you see those errors on systems 
that boot normally?? Do any of those system share a hardware 
configuration with the system that doesn't?

The latter error seems more likely to be the cause of the emergency 
shell, but that's a guess.

On 9/12/20 1:04 PM, Quinn Comendant wrote:
> [root at myhost ~] systemctl status initrd-switch-root.service
> ? initrd-switch-root.service - Switch Root
>     Loaded: loaded (/usr/lib/systemd/system/initrd-switch-root.service;
static; vendor preset: disabled)
>     Active: failed (Result: signal) since Sat 2020-09-12 19:41:13 UTC;
17min ago
>    Process: 204 ExecStart=/usr/bin/systemctl --no-block --force switch-root
/sysroot (code=killed, signal=TERM)

Ah, this seems to be a known issue with the update to 7.8:
https://access.redhat.com/solutions/4973191

The initrd contains an old systemd binary, and should be rebuilt.? The 
replacement initrd will have an updated binary:

|# cp /boot/initramfs-*.x86_64.img /root ||# dracut --force --regenerate-all|
||