Bill Hutchison
2009-Dec-04 21:57 UTC
[zfs-discuss] USB sticks show on one set of devices in zpool, different devices in format
Hello,
I had snv_111b running for a while on a HP DL160G5. With two 16GB USB sticks
comprising the mirrored rpool for boot. And four 1TB drives comprising another
pool, pool1, for data.
So that''s been working just fine for a few months. Yesterday I get it
into my mind to upgrade the OS to latest, then was snv_127. That worked, and
all was well. Also did an upgrade to the DL160G5''s BIOs firmware. All
was cool and running as snv_127 just fine. Upgraded zfs from 13 to 19.
See pool status post-upgrade:
root at arc:/# zpool status
pool: pool1
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
pool1 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
c7t1d0 ONLINE 0 0 0
c7t2d0 ONLINE 0 0 0
c7t3d0 ONLINE 0 0 0
c7t4d0 ONLINE 0 0 0
errors: No known data errors
pool: rpool
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
c2t0d0s0 ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0
errors: No known data errors
Today I went to activate the BE for the new snv_127 install that I''ve
been manually booting into, but "beadm activate..." will always fail,
see here:
root at arc:~# export BE_PRINT_ERR=true
root at arc:~# beadm activate opensolaris-snv127
be_do_installgrub: installgrub failed for device c2t0d0s0.
Unable to activate opensolaris-snv127.
Unknown external error.
So I tried the installgrub manually and get this:
root at arc:~# installgrub /boot/grub/stage1 /boot/grub/stage2
/dev/rdsk/c2t0d0s0
cannot open/stat device /dev/rdsk/c2t0d0s2
OK, wtf? The rpool status shows both of my USB sticks alive and well at
c2t0d0s0 and c1t0d0s0...
But when I run "format -e" I see this:
root at arc:/# format -e
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c7t1d0 <ATA-GB1000EAFJL-HPG1-931.51GB>
/pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 1,0
1. c7t2d0 <ATA-GB1000EAFJL-HPG1-931.51GB>
/pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 2,0
2. c7t3d0 <ATA-GB1000EAFJL-HPG1-931.51GB>
/pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 3,0
3. c7t4d0 <ATA-GB1000EAFJL-HPG1-931.51GB>
/pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 4,0
4. c8t0d0 <DEFAULT cyl 1958 alt 2 hd 255 sec 63>
/pci at 0,0/pci103c,31fe at 1d,7/storage at 8/disk at 0,0
5. c11t0d0 <Kingston-DataTraveler2.0-1.00 cyl 1958 alt 2 hd 255 sec
63>
/pci at 0,0/pci103c,31fe at 1d,7/storage at 6/disk at 0,0
Specify disk (enter its number): 4
selecting c8t0d0
[disk formatted]
/dev/dsk/c8t0d0s0 is part of active ZFS pool rpool. Please see zpool(1M).
It shows my two USB sticks of the rpool being at c8t0d0 and c11t0d0... !
How is this system even working? What do I need to do to clear this up...?
Thanks for your time,
-Bill
--
This message posted from opensolaris.org
Cindy Swearingen
2009-Dec-04 22:26 UTC
[zfs-discuss] USB sticks show on one set of devices in zpool, different devices in format
Hi Bill, I can''t comment on why your USB device names are changing, but I have seen BIOS upgrades do similar things to device names. If you must run a root pool on USB sticks, then I think you would have to boot from the LiveCD before running the BIOS upgrade. Maybe someone can comment. On Sun systems, we recommend importing the pool before running firmware upgrades, for example. The root cause of your beadm activate failure is that the device naming conventions changed in build 125 and beadm doesn''t understand a mirrored root pool because of these changes. This is a known problem described on this list and also in the ZFS troubleshooting wiki although I see that it refers to Live Upgrade only. I will fix this. It might be a good idea to review this wiki before attempting an upgrade. The beadm workaround is to detach the second disk in the mirrored root pool, run the beadm operation, then re-attach the second disk. Cindy On 12/04/09 14:57, Bill Hutchison wrote:> Hello, > > I had snv_111b running for a while on a HP DL160G5. With two 16GB USB sticks comprising the mirrored rpool for boot. And four 1TB drives comprising another pool, pool1, for data. > > So that''s been working just fine for a few months. Yesterday I get it into my mind to upgrade the OS to latest, then was snv_127. That worked, and all was well. Also did an upgrade to the DL160G5''s BIOs firmware. All was cool and running as snv_127 just fine. Upgraded zfs from 13 to 19. > > See pool status post-upgrade: > > root at arc:/# zpool status > pool: pool1 > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > pool1 ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > c7t1d0 ONLINE 0 0 0 > c7t2d0 ONLINE 0 0 0 > c7t3d0 ONLINE 0 0 0 > c7t4d0 ONLINE 0 0 0 > > errors: No known data errors > > pool: rpool > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > rpool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > c2t0d0s0 ONLINE 0 0 0 > c1t0d0s0 ONLINE 0 0 0 > > errors: No known data errors > > > Today I went to activate the BE for the new snv_127 install that I''ve been manually booting into, but "beadm activate..." will always fail, see here: > > root at arc:~# export BE_PRINT_ERR=true > root at arc:~# beadm activate opensolaris-snv127 > be_do_installgrub: installgrub failed for device c2t0d0s0. > Unable to activate opensolaris-snv127. > Unknown external error. > > So I tried the installgrub manually and get this: > > root at arc:~# installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c2t0d0s0 > cannot open/stat device /dev/rdsk/c2t0d0s2 > > OK, wtf? The rpool status shows both of my USB sticks alive and well at c2t0d0s0 and c1t0d0s0... > > But when I run "format -e" I see this: > > root at arc:/# format -e > Searching for disks...done > > > AVAILABLE DISK SELECTIONS: > 0. c7t1d0 <ATA-GB1000EAFJL-HPG1-931.51GB> > /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 1,0 > 1. c7t2d0 <ATA-GB1000EAFJL-HPG1-931.51GB> > /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 2,0 > 2. c7t3d0 <ATA-GB1000EAFJL-HPG1-931.51GB> > /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 3,0 > 3. c7t4d0 <ATA-GB1000EAFJL-HPG1-931.51GB> > /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 4,0 > 4. c8t0d0 <DEFAULT cyl 1958 alt 2 hd 255 sec 63> > /pci at 0,0/pci103c,31fe at 1d,7/storage at 8/disk at 0,0 > 5. c11t0d0 <Kingston-DataTraveler2.0-1.00 cyl 1958 alt 2 hd 255 sec 63> > /pci at 0,0/pci103c,31fe at 1d,7/storage at 6/disk at 0,0 > Specify disk (enter its number): 4 > selecting c8t0d0 > [disk formatted] > /dev/dsk/c8t0d0s0 is part of active ZFS pool rpool. Please see zpool(1M). > > > It shows my two USB sticks of the rpool being at c8t0d0 and c11t0d0... ! > > How is this system even working? What do I need to do to clear this up...? > > > Thanks for your time, > > -Bill
Cindy Swearingen
2009-Dec-04 22:31 UTC
[zfs-discuss] USB sticks show on one set of devices in zpool, different devices in format
On second thought...I think the changing device names are the root cause of your problem with these messages: be_do_installgrub: installgrub failed for device c2t0d0s0. cannot open/stat device /dev/rdsk/c2t0d0s2 After the device names are resolved, you will probably run into the b125 change in device names and the beadm activate will fail something like this: ERROR: Unable to determine the configuration of the current boot environment Cindy On 12/04/09 15:26, Cindy Swearingen wrote:> Hi Bill, > > I can''t comment on why your USB device names are changing, but I have > seen BIOS upgrades do similar things to device names. > > If you must run a root pool on USB sticks, then I think you would have > to boot from the LiveCD before running the BIOS upgrade. Maybe someone > can comment. On Sun systems, we recommend importing the pool before > running firmware upgrades, for example. > > The root cause of your beadm activate failure is that the device naming > conventions changed in build 125 and beadm doesn''t understand a mirrored > root pool because of these changes. > > This is a known problem described on this list and also in the ZFS > troubleshooting wiki although I see that it refers to Live Upgrade only. > I will fix this. It might be a good idea to review this wiki before > attempting an upgrade. > > The beadm workaround is to detach the second disk in the mirrored root > pool, run the beadm operation, then re-attach the second disk. > > Cindy > > > > On 12/04/09 14:57, Bill Hutchison wrote: >> Hello, >> >> I had snv_111b running for a while on a HP DL160G5. With two 16GB USB >> sticks comprising the mirrored rpool for boot. And four 1TB drives >> comprising another pool, pool1, for data. >> >> So that''s been working just fine for a few months. Yesterday I get it >> into my mind to upgrade the OS to latest, then was snv_127. That >> worked, and all was well. Also did an upgrade to the DL160G5''s BIOs >> firmware. All was cool and running as snv_127 just fine. Upgraded >> zfs from 13 to 19. >> >> See pool status post-upgrade: >> >> root at arc:/# zpool status >> pool: pool1 >> state: ONLINE >> scrub: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> pool1 ONLINE 0 0 0 >> raidz1-0 ONLINE 0 0 0 >> c7t1d0 ONLINE 0 0 0 >> c7t2d0 ONLINE 0 0 0 >> c7t3d0 ONLINE 0 0 0 >> c7t4d0 ONLINE 0 0 0 >> >> errors: No known data errors >> >> pool: rpool >> state: ONLINE >> scrub: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> rpool ONLINE 0 0 0 >> mirror-0 ONLINE 0 0 0 >> c2t0d0s0 ONLINE 0 0 0 >> c1t0d0s0 ONLINE 0 0 0 >> >> errors: No known data errors >> >> >> Today I went to activate the BE for the new snv_127 install that I''ve >> been manually booting into, but "beadm activate..." will always fail, >> see here: >> >> root at arc:~# export BE_PRINT_ERR=true >> root at arc:~# beadm activate opensolaris-snv127 >> be_do_installgrub: installgrub failed for device c2t0d0s0. >> Unable to activate opensolaris-snv127. >> Unknown external error. >> >> So I tried the installgrub manually and get this: >> >> root at arc:~# installgrub /boot/grub/stage1 /boot/grub/stage2 >> /dev/rdsk/c2t0d0s0 >> cannot open/stat device /dev/rdsk/c2t0d0s2 >> >> OK, wtf? The rpool status shows both of my USB sticks alive and well >> at c2t0d0s0 and c1t0d0s0... >> >> But when I run "format -e" I see this: >> >> root at arc:/# format -e >> Searching for disks...done >> >> >> AVAILABLE DISK SELECTIONS: >> 0. c7t1d0 <ATA-GB1000EAFJL-HPG1-931.51GB> >> /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 1,0 >> 1. c7t2d0 <ATA-GB1000EAFJL-HPG1-931.51GB> >> /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 2,0 >> 2. c7t3d0 <ATA-GB1000EAFJL-HPG1-931.51GB> >> /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 3,0 >> 3. c7t4d0 <ATA-GB1000EAFJL-HPG1-931.51GB> >> /pci at 0,0/pci8086,4021 at 1/pci103c,3229 at 0/sd at 4,0 >> 4. c8t0d0 <DEFAULT cyl 1958 alt 2 hd 255 sec 63> >> /pci at 0,0/pci103c,31fe at 1d,7/storage at 8/disk at 0,0 >> 5. c11t0d0 <Kingston-DataTraveler2.0-1.00 cyl 1958 alt 2 hd 255 >> sec 63> >> /pci at 0,0/pci103c,31fe at 1d,7/storage at 6/disk at 0,0 >> Specify disk (enter its number): 4 >> selecting c8t0d0 >> [disk formatted] >> /dev/dsk/c8t0d0s0 is part of active ZFS pool rpool. Please see zpool(1M). >> >> >> It shows my two USB sticks of the rpool being at c8t0d0 and c11t0d0... ! >> How is this system even working? What do I need to do to clear this >> up...? >> >> >> Thanks for your time, >> >> -Bill > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss