Fedora xen - May 2007 - fc6xen kernel panic after yum update

Hello,

I used yum update on a newly installed dual cpu i386 FC6 system a
couple days ago. My system was updated from Fedora Core
2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went fine.
After reboot, I got the following kernel panic:

Scanning logical volumes
  Reading all physical volumes. This may take a while...
  No volume groups found
Activating logical volumnes
  Volume group "vg0" not found
Creating root device.
Mounting root filesystem.
mount: could not fine filesystem ''/dev/root''
Setting up other filesystems
Setting up new root fs
setuproot: moving /dev failed: No such file or directory
no fstab.sys, mounting internal defaults
setuproot: error mounting /proc failed: No such file or directory
setuproot: error mounting /sys failed: No such file or directory
Switching to new root and running init.
unmounting old /dev
unmounting old /proc
unmounting old /sys
switchroot: mount failed: No such file or directory
Kernel panic - not syncing: Attempted to kill init!

If I''d boot into the original kernel, it booted just fine.

So, today I took a spare system and installed FC6 fresh from
installation disks. The very first thing I did after the installation
was completed was do a yum update. After reboot, the exact same
problem. Both systems don''t work with the latest XEN kernel.

For reference, the only packages installed from the graphical
installer were these:

Applications/Editors:
	vim-enhanced

Applications/Text-based Internet:
	lynx

Base System/Base:
	finger
	gnupg
	man-pages
	mdadm
	mlocate
	nfs-utils
	openssh-clients
	oopenssh-server
	rsync
	sudo
	telnet
	wget
	which
	yum
Base System/Virtualization:
	virt-manager	

Since the errors indicate LVM problems, here''s info about my disk
layout. I''m configured using two 72GB scsi disks using raid1, then lvm
on top. Disk partitioning:

/dev/sda (72GB)
	/dev/sda1 (102MB)
	/dev/sda2 (69.9GB)
/dev/sdb (72GB)
	/dev/sdb1 (102MB)
	/dev/sdb2 (69.9GB)

/dev/sda1 & /dev/sdb1 are mirrored to /dev/md0
	used by /boot partition (102MB)

/dev/sda2 & /dev/sdb2 are mirrored to physical volume /dev/md1
	used in VolGroup00

/dev/md1 is LVM VolGroup00 and contains these logical volumes
	/dev/VolGroup00/lvdom0 (root)		3GB
	/dev/VolGroup00/lvswap0 (swap)		4GB

The rest of the space was left empty for xen guest partitions.

I''ve edited grub.conf to use /dev/mapper vs /dev/VolGroup00, but
nothing has booted with the new kernel.

Is there a problem with the new kernel? Where should I go for a
solution? Should I just run the old kernel for now? If I''m not in the
best forum for this, where should I go for help?

Any help would be appreciated!

Thanks!

Has anyone else had this problem?  I can''t be the only one who had
updated to this version.  If you have updated and things are working
fine, I''d like to know.  That way at least I''ll know it is
something
on my end.

Thanks,
Tauren


On 5/14/07, Tauren Mills <tauren@groovee.com>
wrote:
> Hello,
>
> I used yum update on a newly installed dual cpu i386 FC6 system a
> couple days ago. My system was updated from Fedora Core
> 2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went fine.
> After reboot, I got the following kernel panic:
>
> Scanning logical volumes
>   Reading all physical volumes. This may take a while...
>   No volume groups found
> Activating logical volumnes
>   Volume group "vg0" not found
> Creating root device.
> Mounting root filesystem.
> mount: could not fine filesystem ''/dev/root''
> Setting up other filesystems
> Setting up new root fs
> setuproot: moving /dev failed: No such file or directory
> no fstab.sys, mounting internal defaults
> setuproot: error mounting /proc failed: No such file or directory
> setuproot: error mounting /sys failed: No such file or directory
> Switching to new root and running init.
> unmounting old /dev
> unmounting old /proc
> unmounting old /sys
> switchroot: mount failed: No such file or directory
> Kernel panic - not syncing: Attempted to kill init!
>
> If I''d boot into the original kernel, it booted just fine.
>
> So, today I took a spare system and installed FC6 fresh from
> installation disks. The very first thing I did after the installation
> was completed was do a yum update. After reboot, the exact same
> problem. Both systems don''t work with the latest XEN kernel.
>
> For reference, the only packages installed from the graphical
> installer were these:
>
> Applications/Editors:
>         vim-enhanced
>
> Applications/Text-based Internet:
>         lynx
>
> Base System/Base:
>         finger
>         gnupg
>         man-pages
>         mdadm
>         mlocate
>         nfs-utils
>         openssh-clients
>         oopenssh-server
>         rsync
>         sudo
>         telnet
>         wget
>         which
>         yum
> Base System/Virtualization:
>         virt-manager
>
> Since the errors indicate LVM problems, here''s info about my disk
> layout. I''m configured using two 72GB scsi disks using raid1, then
lvm
> on top. Disk partitioning:
>
> /dev/sda (72GB)
>         /dev/sda1 (102MB)
>         /dev/sda2 (69.9GB)
> /dev/sdb (72GB)
>         /dev/sdb1 (102MB)
>         /dev/sdb2 (69.9GB)
>
> /dev/sda1 & /dev/sdb1 are mirrored to /dev/md0
>         used by /boot partition (102MB)
>
> /dev/sda2 & /dev/sdb2 are mirrored to physical volume /dev/md1
>         used in VolGroup00
>
> /dev/md1 is LVM VolGroup00 and contains these logical volumes
>         /dev/VolGroup00/lvdom0 (root)           3GB
>         /dev/VolGroup00/lvswap0 (swap)          4GB
>
> The rest of the space was left empty for xen guest partitions.
>
> I''ve edited grub.conf to use /dev/mapper vs /dev/VolGroup00, but
> nothing has booted with the new kernel.
>
> Is there a problem with the new kernel? Where should I go for a
> solution? Should I just run the old kernel for now? If I''m not in
the
> best forum for this, where should I go for help?
>
> Any help would be appreciated!
>
> Thanks!
>

12:55pm Tauren Mills said:
> On 5/14/07, Tauren Mills <tauren@groovee.com> wrote:
>>
>>  I used yum update on a newly installed dual cpu i386 FC6 system a
>>  couple days ago. My system was updated from Fedora Core
>>  2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went fine.
>>  After reboot, I got the following kernel panic:
>>
>>  Scanning logical volumes
>>    Reading all physical volumes. This may take a while...
>>    No volume groups found
>>  Activating logical volumnes
>>    Volume group "vg0" not found
Your kernel/initrd are looking for the "vg0" volume group.
>>
>>  /dev/md1 is LVM VolGroup00 and contains these logical volumes
>>          /dev/VolGroup00/lvdom0 (root)           3GB
>>          /dev/VolGroup00/lvswap0 (swap)          4GB
But you installed Fedora with the default "VolGroup00" name.

Did you try changing the name at some point?

../C

Curtis,

Thanks for the response.  Sorry for the confusion.  I did two separate
FC6 installations on two different machines.  One of the machines
already had some partitions in place that I didn''t remove during
install that used VolGroup00.  So I left that name as is.  The other
system I did a clean wipe and started from scratch, using vg0 as the
volgroup.  Evidentally I mixed up the text from the two machines when
I copy and pasted and sent the email to the list.

The bottom line is that the problem happens on both machines. One
using vg0 and the other using VolGroup00.

Thanks,
Tauren

On 5/19/07, Curtis Doty <Curtis@greenkey.net>
wrote:
> 12:55pm Tauren Mills said:
>
> > On 5/14/07, Tauren Mills <tauren@groovee.com> wrote:
> >>
> >>  I used yum update on a newly installed dual cpu i386 FC6 system a
> >>  couple days ago. My system was updated from Fedora Core
> >>  2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went
fine.
> >>  After reboot, I got the following kernel panic:
> >>
> >>  Scanning logical volumes
> >>    Reading all physical volumes. This may take a while...
> >>    No volume groups found
> >>  Activating logical volumnes
> >>    Volume group "vg0" not found
>
> Your kernel/initrd are looking for the "vg0" volume group.
>
> >>
> >>  /dev/md1 is LVM VolGroup00 and contains these logical volumes
> >>          /dev/VolGroup00/lvdom0 (root)           3GB
> >>          /dev/VolGroup00/lvswap0 (swap)          4GB
>
> But you installed Fedora with the default "VolGroup00" name.
>
> Did you try changing the name at some point?
>
> ../C
>
> --
> Fedora-xen mailing list
> Fedora-xen@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-xen
>

Tauren Mills wrote:
> Has anyone else had this problem?  I can''t be the only one who had
> updated to this version.  If you have updated and things are working
> fine, I''d like to know.  That way at least I''ll know it
is something
> on my end.
> 
> Thanks,
> Tauren
> 
> 
> On 5/14/07, Tauren Mills <tauren@groovee.com> wrote:
>> Hello,
>>
>> I used yum update on a newly installed dual cpu i386 FC6 system a
>> couple days ago. My system was updated from Fedora Core
>> 2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went fine.
...

no problems here with my AMD Athlon(tm) 64 X2 Dual Core and xen kernel
2948, but it''s a x86_64 installation.

you can compare both initrd''s from the xen kernels - assuming you still
have both - via:

- copy them to two different directories
- zcat initrd... | cpio -di
- compare the files "init" , esp. lines: mkrootdev ... and insmod ...
or
for last part compare the directories "lib".

or take a look at

https://bugzilla.redhat.com/bugzilla/buglist.cgi?product=Fedora+Core&version=fc6&component=kernel-xen&bug_status=NEW&bug_status=ASSIGNED&bug_status=NEEDINFO&bug_status=MODIFIED&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc
-- 
      ronald

Ronald,

Thanks for the help.  I copied the initrd files to different
directories and executed the commands you indicated.  Doing a diff
only reports this:

[root@pyrite test]# diff 2798/init 2948/init
55a56,59
> echo "Loading xor.ko module"
> insmod /lib/xor.ko
> echo "Loading raid456.ko module"
> insmod /lib/raid456.ko
It looks like there are two extra modules in 2948 that weren''t in 2798.

Also, I previously had edited grub.conf and changed root= to see if
that made any difference.  But using
root=/dev/mapper/VolGroup00-lvdom0 has the same result as
root=/dev/VolGroup00/lvdom0.  I did this because after the yum update,
the new kernel used the mapper syntax while the original kernel
didn''t.  At the moment I have it like this:

title Fedora Core (2.6.20-1.2948.fc6xen)
        root (hd0,0)
        kernel /xen.gz-2.6.20-1.2948.fc6
        module /vmlinuz-2.6.20-1.2948.fc6xen ro
root=/dev/VolGroup00/lvdom0 ide=nodma
        module /initrd-2.6.20-1.2948.fc6xen.img
title Fedora Core (2.6.18-1.2798.fc6xen)
        root (hd0,0)
        kernel /xen.gz-2.6.18-1.2798.fc6
        module /vmlinuz-2.6.18-1.2798.fc6xen ro
root=/dev/VolGroup00/lvdom0 ide=nodma
        module /initrd-2.6.18-1.2798.fc6xen.img

Thanks again for the help.

Tauren


On 5/19/07, Ronald Warsow <rwarsow@online.de>
wrote:
> Tauren Mills wrote:
> > Has anyone else had this problem?  I can''t be the only one
who had
> > updated to this version.  If you have updated and things are working
> > fine, I''d like to know.  That way at least I''ll know
it is something
> > on my end.
> >
> > Thanks,
> > Tauren
> >
> >
> > On 5/14/07, Tauren Mills <tauren@groovee.com> wrote:
> >> Hello,
> >>
> >> I used yum update on a newly installed dual cpu i386 FC6 system a
> >> couple days ago. My system was updated from Fedora Core
> >> 2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went
fine.
> ...
>
> no problems here with my AMD Athlon(tm) 64 X2 Dual Core and xen kernel
> 2948, but it''s a x86_64 installation.
>
> you can compare both initrd''s from the xen kernels - assuming you
still
> have both - via:
>
> - copy them to two different directories
> - zcat initrd... | cpio -di
> - compare the files "init" , esp. lines: mkrootdev ... and insmod
... or
> for last part compare the directories "lib".
>
> or take a look at
>
>
https://bugzilla.redhat.com/bugzilla/buglist.cgi?product=Fedora+Core&version=fc6&component=kernel-xen&bug_status=NEW&bug_status=ASSIGNED&bug_status=NEEDINFO&bug_status=MODIFIED&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc>
> --
>       ronald
>
> --
> Fedora-xen mailing list
> Fedora-xen@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-xen
>

Tauren Mills wrote:
...
> 
> root=/dev/VolGroup00/lvdom0.  I did this because after the yum update,
AFAIK, the init process runs scripts/tools like "vgscan" *after* the 
kernel has mounted it''s rootdev, etc. ....

and i believe *exactly afterwards*
/dev/mapper/VolGroup00-lvdom0 is the same then
/dev/VolGroup00/lvdom0

but i''m no lvm expert !
> the new kernel used the mapper syntax while the original kernel
> didn''t.  At the moment I have it like this:
> 
> title Fedora Core (2.6.20-1.2948.fc6xen)
>        root (hd0,0)
>        kernel /xen.gz-2.6.20-1.2948.fc6
>        module /vmlinuz-2.6.20-1.2948.fc6xen ro
shouldn''t it read kernel /boot/...
                   module /boot/...  ???

keying error ?
> root=/dev/VolGroup00/lvdom0 ide=nodma
>        module /initrd-2.6.20-1.2948.fc6xen.img
> title Fedora Core (2.6.18-1.2798.fc6xen)
>        root (hd0,0)
>        kernel /xen.gz-2.6.18-1.2798.fc6
>        module /vmlinuz-2.6.18-1.2798.fc6xen ro
> root=/dev/VolGroup00/lvdom0 ide=nodma
>        module /initrd-2.6.18-1.2798.fc6xen.img
did the non-xen 2.6.20.x.y kernels talk to your hd-controller/hd''s ?
with that hand keyed lvm-lines ?

you could compare the initrd''s of the non-xen-kernel and a xen-kernel 
(both version 2.6.20.x.y), also. - they shouldn''t be very different -


something prevents your xen-kernel talking to/seeing your hd-controller/hd !
and
weeks ago i read something on one of the fedora-{test,devel}-list
about initrd''s and the module raid456...


-- 
      ronald

aloa,
i had the same problem a few days ago after upgrading from fc5 to fc6 xen.
the problem seems to be a missing driver in the initrd.
i solved this creating a new initrd using
initrd --with=raid456 --with=lvm xen-xxxxx.img kernel
this works without any problems, i don''t know if this is the
"correct
way" to solve this, but it works ^^
don''t forget to edit your grub.conf, so that the new initrd file is
used!

greets jm

Tauren Mills schrieb:
> Has anyone else had this problem?  I can''t be the only one who had
> updated to this version.  If you have updated and things are working
> fine, I''d like to know.  That way at least I''ll know it
is something
> on my end.
>
> Thanks,
> Tauren
>
>
> On 5/14/07, Tauren Mills <tauren@groovee.com> wrote:
>> Hello,
>>
>> I used yum update on a newly installed dual cpu i386 FC6 system a
>> couple days ago. My system was updated from Fedora Core
>> 2.6.18-1.2798.fc6xen to 2.6.20-1.2948.fc6xen. The update went fine.
>> After reboot, I got the following kernel panic:
>>
>> Scanning logical volumes
>>   Reading all physical volumes. This may take a while...
>>   No volume groups found
>> Activating logical volumnes
>>   Volume group "vg0" not found
>> Creating root device.
>> Mounting root filesystem.
>> mount: could not fine filesystem ''/dev/root''
>> Setting up other filesystems
>> Setting up new root fs
>> setuproot: moving /dev failed: No such file or directory
>> no fstab.sys, mounting internal defaults
>> setuproot: error mounting /proc failed: No such file or directory
>> setuproot: error mounting /sys failed: No such file or directory
>> Switching to new root and running init.
>> unmounting old /dev
>> unmounting old /proc
>> unmounting old /sys
>> switchroot: mount failed: No such file or directory
>> Kernel panic - not syncing: Attempted to kill init!
>>
>> If I''d boot into the original kernel, it booted just fine.
>>
>> So, today I took a spare system and installed FC6 fresh from
>> installation disks. The very first thing I did after the installation
>> was completed was do a yum update. After reboot, the exact same
>> problem. Both systems don''t work with the latest XEN kernel.
>>
>> For reference, the only packages installed from the graphical
>> installer were these:
>>
>> Applications/Editors:
>>         vim-enhanced
>>
>> Applications/Text-based Internet:
>>         lynx
>>
>> Base System/Base:
>>         finger
>>         gnupg
>>         man-pages
>>         mdadm
>>         mlocate
>>         nfs-utils
>>         openssh-clients
>>         oopenssh-server
>>         rsync
>>         sudo
>>         telnet
>>         wget
>>         which
>>         yum
>> Base System/Virtualization:
>>         virt-manager
>>
>> Since the errors indicate LVM problems, here''s info about my
disk
>> layout. I''m configured using two 72GB scsi disks using raid1,
then lvm
>> on top. Disk partitioning:
>>
>> /dev/sda (72GB)
>>         /dev/sda1 (102MB)
>>         /dev/sda2 (69.9GB)
>> /dev/sdb (72GB)
>>         /dev/sdb1 (102MB)
>>         /dev/sdb2 (69.9GB)
>>
>> /dev/sda1 & /dev/sdb1 are mirrored to /dev/md0
>>         used by /boot partition (102MB)
>>
>> /dev/sda2 & /dev/sdb2 are mirrored to physical volume /dev/md1
>>         used in VolGroup00
>>
>> /dev/md1 is LVM VolGroup00 and contains these logical volumes
>>         /dev/VolGroup00/lvdom0 (root)           3GB
>>         /dev/VolGroup00/lvswap0 (swap)          4GB
>>
>> The rest of the space was left empty for xen guest partitions.
>>
>> I''ve edited grub.conf to use /dev/mapper vs /dev/VolGroup00,
but
>> nothing has booted with the new kernel.
>>
>> Is there a problem with the new kernel? Where should I go for a
>> solution? Should I just run the old kernel for now? If I''m not
in the
>> best forum for this, where should I go for help?
>>
>> Any help would be appreciated!
>>
>> Thanks!
>>
>
> -- 
> Fedora-xen mailing list
> Fedora-xen@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-xen