I have a KVM vm running CentOS-6.8 on a host also running CentOS-6.8.
This instance is used for occasional development projects which
require segregation. Thus it is seldom accessed.
At some point in the recent past this guest developed an issue with
starting. Specifically these messages were found in the system log
files:
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_cm_xnet245.harte-lyne.ca_00' [31.25 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet241.harte-lyne.ca_00' [32.00 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet241.harte-lyne.ca_01' [32.00 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet241.harte-lyne.ca_02' [32.00 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet242.harte-lyne.ca_00' [31.25 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet243.harte-lyne.ca_00' [31.25 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet244.harte-lyne.ca_00' [31.25 GiB]
inherit
/var/log/messages-20170312:Mar 10 16:31:06 vhost04 kernel: dracut:
inactive '/dev/vg_vhost04/lv_vm_xnet245.harte-lyne.ca_00' [31.25 GiB]
inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_cm_xnet245.harte-lyne.ca_00' [31.25 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet241.harte-lyne.ca_00' [32.00 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet241.harte-lyne.ca_01' [32.00 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet241.harte-lyne.ca_02' [32.00 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet242.harte-lyne.ca_00' [31.25 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet243.harte-lyne.ca_00' [31.25 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet244.harte-lyne.ca_00' [31.25 GiB] inherit
/var/log/messages:Mar 20 08:52:10 vhost04 kernel: dracut: inactive
'/dev/vg_vhost04/lv_vm_xnet245.harte-lyne.ca_00' [31.25 GiB] inherit
It appears that this issue was first encountered on March 10 as our
log files go back much further than that and have no earlier record.
The symptom presented on the guest console during boot is:
Error 13: Invalid or unsupported executable format
Press any key to continue...
Pressing enter brings up the following text display
GNU GRUB version 0.97 (615K lower / 3668980K upper memory)
CentOS (2.6.32-641.15.1.el6.x86_64)
CentOS (2.6.32-641.13.1.el6.x86_64)
CentOS (2.6.32-641.11.1.el6.x86_64)
CentOS (2.6.32-641.2.1.el6.x86_64)
Use the ↑ and ↓ keys to select which entry is
highlighted
Press enter to boot the selected OS, 'e' to edit . . .
The first choice repeats the Error. The second choice boots cleanly.
Looking at yum history I see this:
yum history
Loaded plugins: etckeeper, fastestmirror, priorities,
refresh-packagekit, security
ID | Login user | Date and time | Action(s) | Altered
---------------------------------------------------------------------
. . .
368 | root <root> | 2017-03-10 16:46 | Update | 4
367 | root <root> | 2017-03-10 16:42 | E, U | 35 EE
. . .
Looking at transaction 367 more closely we see that the kernel was
updated to 2.6.32-642.15.1.el6.x86_64 on March 10 but that a number or
errors, whose nature I do not comprehend, were also reported.
# yum history info 367
Loaded plugins: etckeeper, fastestmirror, priorities,
refresh-packagekit, security
Transaction ID : 367
Begin time : Fri Mar 10 16:42:32 2017
Begin rpmdb : 1489:fd0eb9a01b1667f826b8fead9bc0a05e5bc43efd
End time : 16:43:59 2017 (87 seconds)
End rpmdb : 1461:cac690d6280fa97910ccb59d0d1f6d43990dfd0a
User : root <root>
Return-Code : Success
Transaction performed with:
Installed rpm-4.8.0-55.el6.x86_64 @base
Installed yum-3.2.29-75.el6.centos.noarch @updates
Installed yum-metadata-parser-1.1.2-16.el6.x86_64
@anaconda-CentOS-201207061011.x86_64/6.3
Installed yum-plugin-fastestmirror-1.1.30-37.el6.noarch @base
Installed yum-utils-1.1.30-37.el6.noarch @base
Packages Altered:
Updated firefox-45.7.0-1.el6.centos.x86_64 @updates
Update 45.7.0-2.el6.centos.x86_64 @updates
Updated gnome-settings-daemon-2.28.2-35.el6.x86_64 @base
Update 2.28.2-35.el6_8.2.x86_64 @updates
Erase initscripts-9.03.53-1.el6.centos.1.x86_64 @updates
Erase kernel-2.6.32-642.4.2.el6.x86_64 @updates
Erase kernel-firmware-2.6.32-642.13.1.el6.noarch @updates
Updated kernel-headers-2.6.32-642.13.1.el6.x86_64 @updates
Update 2.6.32-642.15.1.el6.x86_64 @updates
Updated kexec-tools-2.0.0-300.el6_8.1.x86_64 @updates
Update 2.0.0-300.el6_8.2.x86_64 @updates
Erase libbasicobjects-0.1.1-11.el6.x86_64 @base
Erase libblkid-2.17.2-12.24.el6_8.1.x86_64 @updates
Erase libcollection-0.6.2-11.el6.x86_64 @base
Erase libdhash-0.4.3-11.el6.x86_64 @base
Erase libini_config-1.1.0-11.el6.x86_64 @base
Erase libipa_hbac-1.13.3-22.el6_8.4.x86_64 @updates
Erase libpath_utils-0.2.1-11.el6.x86_64 @base
Erase libref_array-0.1.4-11.el6.x86_64 @base
Erase libsss_idmap-1.13.3-22.el6_8.4.x86_64 @updates
Erase libuuid-2.17.2-12.24.el6_8.1.x86_64 @updates
Updated openssl-1.0.1e-48.el6_8.3.i686 @updates
Erase openssl-1.0.1e-48.el6_8.3.x86_64 @updates
Update openssl-1.0.1e-48.el6_8.4.i686 @updates
Updated openssl-devel-1.0.1e-48.el6_8.3.x86_64 @updates
Update 1.0.1e-48.el6_8.4.x86_64 @updates
Updated python-libipa_hbac-1.13.3-22.el6_8.4.x86_64 @updates
Update 1.13.3-22.el6_8.6.x86_64 @updates
Erase python-sssdconfig-1.13.3-22.el6_8.4.noarch @updates
Erase selinux-policy-3.7.19-292.el6_8.2.noarch @updates
Erase selinux-policy-targeted-3.7.19-292.el6_8.2.noarch @updates
Erase sssd-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-ad-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-client-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-common-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-common-pac-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-ipa-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-krb5-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-krb5-common-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-ldap-1.13.3-22.el6_8.4.x86_64 @updates
Erase sssd-proxy-1.13.3-22.el6_8.4.x86_64 @updates
Erase util-linux-ng-2.17.2-12.24.el6_8.1.x86_64 @updates
Scriptlet output:
1 warning: erase unlink of
/lib/modules/2.6.32-642.4.2.el6.x86_64/weak-updates failed: No such
file or directory
2 warning: erase unlink of
/lib/modules/2.6.32-642.4.2.el6.x86_64/modules.order failed: No
such file or directory
3 warning: erase unlink of
/lib/modules/2.6.32-642.4.2.el6.x86_64/modules.networking failed:
No such file or directory
4 warning: erase unlink of
/lib/modules/2.6.32-642.4.2.el6.x86_64/modules.modesetting failed:
No such file or directory
5 warning: erase unlink of
/lib/modules/2.6.32-642.4.2.el6.x86_64/modules.drm failed: No such
file or directory
6 warning: erase unlink of
/lib/modules/2.6.32-642.4.2.el6.x86_64/modules.block failed: No
such file or directory
Reviewing grub.conf I note that the initrd entry is missing from the
newest kernel entry. Which of course explains the boot error message.
However, as kernel updates are entirely automated when performed by
yum I am at a lost to explain its absence.
default=saved
fallback= 0 1 2
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-642.15.1.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-642.15.1.el6.x86_64 ro
root=/dev/mapper/vg_xnet241-lv_root rd_NO_LUKS
LANG=en_CA at yyyy-mmm-dd.UTF-8 rd_LVM_LV=vg_xnet241/lv_swap
rd_LVM_LV=vg_xnet241/lv_root rd_NO_MD
SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
rd_NO_DM rhgb quiet crashkernel=auto radeon.modeset=0
title CentOS (2.6.32-642.13.1.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-642.13.1.el6.x86_64 ro
root=/dev/mapper/vg_xnet241-lv_root rd_NO_LUKS
LANG=en_CA at yyyy-mmm-dd.UTF-8 rd_LVM_LV=vg_xnet241/lv_swap
rd_LVM_LV=vg_xnet241/lv_root rd_NO_MD
SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us
rd_NO_DM rhgb quiet crashkernel=auto radeon.modeset=0
initrd /initramfs-2.6.32-642.13.1.el6.x86_64.img
There is no initramfs-2.6.32-642.15.1.el6.x86_64.img file present in
/boot so something is seriously wrong somewhere.
As shown above I have since reconfigured grub.conf to fallback to the
last working kernel. Nonetheless, I would like guidance on how to
proceed with fixing/removing the broken one.
Suggestions?
--
*** e-Mail is NOT a SECURE channel ***
Do NOT transmit sensitive data via e-Mail
Do NOT open attachments nor follow links sent by e-Mail
James B. Byrne mailto:ByrneJB at Harte-Lyne.ca
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3