Begin forwarded message:> From: Michael Hale <mhale at transcomus.com> > Date: December 18, 2007 6:15:12 PM CST > To: zfs-discuss at opensolaris.org > Subject: zfs boot suddenly not working > > We have a machine that is configured with zfs boot , Nevada v67- we > have two pools, rootpool and datapool. It has been working ok since > June. Today it kernel panicked and now when we try to boot it up, > it gets to the grub screen, we select ZFS, and then there is a > kernel panic that flashes by too quickly for us to see and then it > reboots. > > If we boot to a nevada v77 DVD and if we boot to that, we can do a > zpool import and mount the zfs pools successfully. We scrubbed them > and didn''t find any errors. From the nevada v77 DVD we can see > everything ok. > > Here is our grub menu.lst > > title Solaris ZFS snv_67 X86 > kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS > module$ /platform/i86pc/$ISADIR/boot_archive > > First of all, is there a way to slow down that kernel panic so that > we can see what it is? Also, we suspect that maybe /platform/i86pm/ > boot_archive might have been damaged. Is there a way to regenerate > it? > -- > Michael Hale > mhale at transcomus.com > > >-- Michael Hale giftculture at gmail.com http://www.gift-culture.org -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20071218/765d1193/attachment.html>
Edit the kernel$ line and add ''-k'' at the end. That should drop you into the kernel debugger after the panic (typing ''$q'' will exit the debugger, and resume whatever it was doing -- in this case likely rebooting). On Dec 18, 2007 6:26 PM, Michael Hale <giftculture at gmail.com> wrote:> > > Begin forwarded message: > > From: Michael Hale <mhale at transcomus.com> > Date: December 18, 2007 6:15:12 PM CST > To: zfs-discuss at opensolaris.org > Subject: zfs boot suddenly not working > > We have a machine that is configured with zfs boot , Nevada v67- we have > two pools, rootpool and datapool. It has been working ok since June. Today > it kernel panicked and now when we try to boot it up, it gets to the grub > screen, we select ZFS, and then there is a kernel panic that flashes by too > quickly for us to see and then it reboots. > > If we boot to a nevada v77 DVD and if we boot to that, we can do a zpool > import and mount the zfs pools successfully. We scrubbed them and didn''t > find any errors. From the nevada v77 DVD we can see everything ok. > > Here is our grub menu.lst > > title Solaris ZFS snv_67 X86 > kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS > module$ /platform/i86pc/$ISADIR/boot_archive > > First of all, is there a way to slow down that kernel panic so that we can > see what it is? Also, we suspect that maybe /platform/i86pm/boot_archive > might have been damaged. Is there a way to regenerate it? > -- > Michael Hale > mhale at transcomus.com > > > > > > -- > Michael Hale > giftculture at gmail.com > http://www.gift-culture.org > > > > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >
After doing that, this is what we see: panic[cpu0]/thread=fffffffffbc257a0: cannot mount root path /pci at 0,0/ pci8086,25e3 at 3/pci8086,370 at 0/pci1028,1f03 at e/sd at 0,0:f fffffffffbc46790 genunix: rootconf+112 () fffffffffbc467e0 genunix:vfs_mountroot +65 () fffffffffbc46810 genunix:main+ce () fffffffffbc46820 unix:_locore_start+92 () panic: entering debugger /rootpool/rootfs/etc/zfs/zpool.cache was last updated dec 13 at 15:25 and has a size of 3880 bytes if we boot off the DVD (snv 77) /etc/zfs/zpool.cache has a size of 1604 On Dec 18, 2007, at 7:08 PM, Jason King wrote:> Edit the kernel$ line and add ''-k'' at the end. That should drop you > into the kernel debugger after the panic (typing ''$q'' will exit the > debugger, and resume whatever it was doing -- in this case likely > rebooting). > > > On Dec 18, 2007 6:26 PM, Michael Hale <giftculture at gmail.com> wrote: >> >> >> Begin forwarded message: >> >> From: Michael Hale <mhale at transcomus.com> >> Date: December 18, 2007 6:15:12 PM CST >> To: zfs-discuss at opensolaris.org >> Subject: zfs boot suddenly not working >> >> We have a machine that is configured with zfs boot , Nevada v67- we >> have >> two pools, rootpool and datapool. It has been working ok since >> June. Today >> it kernel panicked and now when we try to boot it up, it gets to >> the grub >> screen, we select ZFS, and then there is a kernel panic that >> flashes by too >> quickly for us to see and then it reboots. >> >> If we boot to a nevada v77 DVD and if we boot to that, we can do a >> zpool >> import and mount the zfs pools successfully. We scrubbed them and >> didn''t >> find any errors. From the nevada v77 DVD we can see everything ok. >> >> Here is our grub menu.lst >> >> title Solaris ZFS snv_67 X86 >> kernel$ /platform/i86pc/kernel/$ISADIR/unix -B $ZFS-BOOTFS >> module$ /platform/i86pc/$ISADIR/boot_archive >> >> First of all, is there a way to slow down that kernel panic so that >> we can >> see what it is? Also, we suspect that maybe /platform/i86pm/ >> boot_archive >> might have been damaged. Is there a way to regenerate it? >> -- >> Michael Hale >> mhale at transcomus.com >> >> >> >> >> >> -- >> Michael Hale >> giftculture at gmail.com >> http://www.gift-culture.org >> >> >> >> >> >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >>-- Michael Hale giftculture at gmail.com http://www.gift-culture.org
> panic[cpu0]/thread=fffffffffbc257a0: cannot mount root path /pci at 0,0/when booted from snv_77 type: zpool import rootpool zpool get bootfs rootpool mkdir /mnt mount -F zfs "the bootfs string" /mnt my guess is it will fail... so then do zfs list and find one that will mount, then zpool set bootfs=root/snv_77 rootpool grep zfs /mnt/etc/vfstab and verify it matches what you set bootfs to also take a peek at /rootpool/boot/grub/menu.lst but it will be fine.. Rob
the root pool mounts fine - if I do: zpool import rootpool zpool get bootfs rootpool mkdir /mnt mount -F zfs rootpool/rootfs it mounts fine /etc/zfs/zpool.cache exists a zpool get all rootpool gets us: size 19.9 G used 3.67G available 16.2G capacity 18% altroot - health ONLINE guid 1573491433247481682 version 6 botfs rootpool/rootfs delegation on autoreplace off cachefile - failmode wait we''ve scrubbed the pool, the config is a rootpool with two mirrors, c3t0d0s5 c3t3d0s5 On Dec 18, 2007, at 8:03 PM, Rob Logan wrote:> >> panic[cpu0]/thread=fffffffffbc257a0: cannot mount root path /pci at 0,0/ > > when booted from snv_77 type: > > zpool import rootpool > zpool get bootfs rootpool > mkdir /mnt > mount -F zfs "the bootfs string" /mnt > > my guess is it will fail... so then do > zfs list > > and find one that will mount, then > > zpool set bootfs=root/snv_77 rootpool > grep zfs /mnt/etc/vfstab > > and verify it matches what you set bootfs to > > also take a peek at /rootpool/boot/grub/menu.lst > but it will be fine.. > > Rob > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- Michael Hale giftculture at gmail.com http://www.gift-culture.org
> bootfs rootpool/rootfsdoes "grep zfs /mnt/etc/vfstab" look like: rootpool/rootfs - / zfs - no - (bet it doesn''t... edit like above and reboot) or second guess (well, third :-) is your theory that can be checked with: zpool import rootpool zpool import datapool mkdir /mnt mount -F zfs rootpool/rootfs /mnt tail /mnt/boot/solaris/filelist.ramdisk echo look for (no leading /) etc/zfs/zpool.cache cp /etc/zfs/zpool.cache /mnt/etc/zfs/zpool.cache /usr/sbin/bootadm update-archive -R /mnt reboot
On Dec 18, 2007, at 8:26 PM, Rob Logan wrote:> >> bootfs rootpool/rootfs > > does "grep zfs /mnt/etc/vfstab" look like: > > rootpool/rootfs - / zfs - no - > > (bet it doesn''t... edit like above and reboot)That is exactly what it looks like :^(> > > or second guess (well, third :-) is your theory that > can be checked with: > > zpool import rootpool > zpool import datapool > mkdir /mnt > mount -F zfs rootpool/rootfs /mnt > tail /mnt/boot/solaris/filelist.ramdisk > echo look for (no leading /) etc/zfs/zpool.cache > cp /etc/zfs/zpool.cache /mnt/etc/zfs/zpool.cache > /usr/sbin/bootadm update-archive -R /mnt > rebootWe''re trying this now. This seems to have worked! :^) I guess the zpool.cache in the bootimage got corrupted? -- Michael Hale giftculture at gmail.com http://www.gift-culture.org
> I guess the zpool.cache in the bootimage got corrupted?not on zfs :-) perhaps a path to a drive changed? Rob
On Dec 18, 2007, at 9:15 PM, Rob Logan wrote:> >> I guess the zpool.cache in the bootimage got corrupted? > not on zfs :-) perhaps a path to a drive changed?heh - probably. This is off topic but now this brings us to another problem... My fellow syadmin here at work was trying to get solaris 10 to talk to our openldap server. He ran ldapclient with a manual config to set up password authentication. Upon reboot, it displayed the hostname and then said: ldap nis domain name is and then would hang. At that point, it was taken into single user mode and he ran: ldap client uninit which wiped out the LDAP configuration, but now upon boot, the machine says: Hostname: mbox02 NIS domain name is and just hangs. If we try to boot into single user mode, it asks for the root password - we type it in and then it just hangs. We''ve waited several minutes now and it just seems to be locked up there I know this is off topic but does anybody have any ideas? -- Michael Hale giftculture at gmail.com http://www.gift-culture.org
I can think of two things to check: First, is there a ''bootfs'' line in your grub entry? I didn''t see it in the original email; not sure if it was left out or it simply isn''t present. If it''s not present, ensure the ''bootfs'' property is set on your pool. Secondly, ensure that there''s a zpool.cache entry in the filelist.ramdisk (if not, add it and re-run ''bootadm update-archive'') If that doesn''t do the trick, take a look at this page and see if there''s anything that''ll help: http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ Regards, markm -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20071218/10428b16/attachment.html>