CentOS virt - Sep 2009 - Domain0 with Kernel 2.6.18-92.1.22.el5xen fails to boot Guests with Kernel 2.6.18-164.el5xen

Hi all,

I am in a bit of a fix here, I have several Guests paravirtualized
running on a Dom0 that is currently running the 2.6.18-92.1.22.el5xen
Kernel. All my guests (administered by different people) have
subsequently been updated to later versions of the kernel (namely
2.6.18-164.el5xen or 2.6.18-128.4.1.el5xen).

Ones that are still running are using the Kernel 2.6.18-128.4.1.el5xen
while Dom0 is still booted up into 2.6.18-92.1.22.el5xen

I restarted a Guest that's now trying to boot with the
2.6.18-164.el5xen Kernel and it spits out messages like
XENBUS: Waiting for devices to initialise: 295s...290s...

and eventually reboots.

If I try and interrupt the boot and choose a previous version of the
Kernel on the Guest, it says the following and fails to boot. The
guests use a single disk image that has the SWAP and Root partitions.

Is this happening because the version of the kernel on the Dom0 is way
too old for guests to boot? I am just a bit hesitant in rebooting my
server for the new kernel to take effect unless I am sure :)

device-mapper: dm-raid45: initialized v0.2429
Scanning and configuring dmraid supported devices
Scanning logical volumes
  Reading all physical volumes.  This may take a while...
Activating logical volumes
  Volume group "VolGroup00" not found
Creating root device.
Mounting root filesystem.
mount: could not find filesystem '/dev/root'
Setting up other filesystems.
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!


-- 
"The secret impresses no-one, the trick you use it for is everything"
- Alfred Borden (The Prestiege)

On Sun, Sep 27, 2009 at 12:46:46PM +1000, Devraj Mukherjee
wrote:
> Hi all,
> 
> I am in a bit of a fix here, I have several Guests paravirtualized
> running on a Dom0 that is currently running the 2.6.18-92.1.22.el5xen
> Kernel. All my guests (administered by different people) have
> subsequently been updated to later versions of the kernel (namely
> 2.6.18-164.el5xen or 2.6.18-128.4.1.el5xen).
> 
> Ones that are still running are using the Kernel 2.6.18-128.4.1.el5xen
> while Dom0 is still booted up into 2.6.18-92.1.22.el5xen
> 
> I restarted a Guest that's now trying to boot with the
> 2.6.18-164.el5xen Kernel and it spits out messages like
> XENBUS: Waiting for devices to initialise: 295s...290s...
> 
> and eventually reboots.
> 
> If I try and interrupt the boot and choose a previous version of the
> Kernel on the Guest, it says the following and fails to boot. The
> guests use a single disk image that has the SWAP and Root partitions.
> 
> Is this happening because the version of the kernel on the Dom0 is way
> too old for guests to boot? I am just a bit hesitant in rebooting my
> server for the new kernel to take effect unless I am sure :)
>
No, that shouldn't be the reason. 

The layer between dom0 and guests is Xen hypervisor, not the kernel.
Although Redhat ships the hypervisor in the same rpm as the kernel :)

The output looks like the initrd image is broken; it doesn't set up the root
device.. does it load the xenblk driver? Does it detect the xvd* disks?

Please paste the full domU kernel boot log..

-- Pasi

----- "Devraj Mukherjee" <devraj at gmail.com> wrote:
> Hi all,
> 
> I am in a bit of a fix here, I have several Guests paravirtualized
> running on a Dom0 that is currently running the 2.6.18-92.1.22.el5xen
> Kernel. All my guests (administered by different people) have
> subsequently been updated to later versions of the kernel (namely
> 2.6.18-164.el5xen or 2.6.18-128.4.1.el5xen).
[...]
> I restarted a Guest that's now trying to boot with the
> 2.6.18-164.el5xen Kernel and it spits out messages like
> XENBUS: Waiting for devices to initialise: 295s...290s...
> 
> and eventually reboots.
IIRC, there were changes to the block back-/frontend drivers a while back, but
that probably wouldn't do this. I would first look to make sure that your
dom0 Xen daemons are in good condition. Occasionally, they will die or stop
working. Watching the logs when you start your guest may give you some clues as
to what dom0 may be hanging on. Restarting xend and/or xenconsoled (typically
when your guest can't get a console or when ``xm console''
mysteriously doesn't work) may help. Anyway, you should update your dom0
kernel. That's pretty old. :)

-- 
Christopher G. Stach II

Hi,

where does you root-device reside on?
Do you use sda or hda?

Can you post your DomU-config (without comments) here?

What messages do you get before LVM complains?
It might be that you don't have the rigth drivers in your DomU-initrd
(any more)...

Kind regards,

Nils 
> -----Original Message-----
> From: centos-virt-bounces at centos.org 
> [mailto:centos-virt-bounces at centos.org] On Behalf Of Devraj Mukherjee
> Sent: Sunday, September 27, 2009 4:47 AM
> To: Discussion about the virtualization on CentOS
> Subject: [CentOS-virt] Domain0 with Kernel 
> 2.6.18-92.1.22.el5xen fails toboot Guests with Kernel 
> 2.6.18-164.el5xen
> 
> Hi all,
> 
> I am in a bit of a fix here, I have several Guests 
> paravirtualized running on a Dom0 that is currently running 
> the 2.6.18-92.1.22.el5xen Kernel. All my guests (administered 
> by different people) have subsequently been updated to later 
> versions of the kernel (namely 2.6.18-164.el5xen or 
> 2.6.18-128.4.1.el5xen).
> 
> Ones that are still running are using the Kernel 
> 2.6.18-128.4.1.el5xen while Dom0 is still booted up into 
> 2.6.18-92.1.22.el5xen
> 
> I restarted a Guest that's now trying to boot with the 
> 2.6.18-164.el5xen Kernel and it spits out messages like
> XENBUS: Waiting for devices to initialise: 295s...290s...
> 
> and eventually reboots.
> 
> If I try and interrupt the boot and choose a previous version 
> of the Kernel on the Guest, it says the following and fails 
> to boot. The guests use a single disk image that has the SWAP 
> and Root partitions.
> 
> Is this happening because the version of the kernel on the 
> Dom0 is way too old for guests to boot? I am just a bit 
> hesitant in rebooting my server for the new kernel to take 
> effect unless I am sure :)
> 
> device-mapper: dm-raid45: initialized v0.2429 Scanning and 
> configuring dmraid supported devices Scanning logical volumes
>   Reading all physical volumes.  This may take a while...
> Activating logical volumes
>   Volume group "VolGroup00" not found
> Creating root device.
> Mounting root filesystem.
> mount: could not find filesystem '/dev/root'
> Setting up other filesystems.
> Setting up new root fs
> setuproot: moving /dev failed: No such file or directory no 
> fstab.sys, mounting internal defaults
> setuproot: error mounting /proc: No such file or directory
> setuproot: error mounting /sys: No such file or directory 
> Switching to new root and running init.
> unmounting old /dev
> unmounting old /proc
> unmounting old /sys
> switchroot: mount failed: No such file or directory Kernel 
> panic - not syncing: Attempted to kill init!
> 
> 
> --
> "The secret impresses no-one, the trick you use it for is
everything"
> - Alfred Borden (The Prestiege)
> _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> http://lists.centos.org/mailman/listinfo/centos-virt
>