I''m having trouble booting with one of my zpools. It looks like this:
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c4d0 ONLINE 0 0 0
c4d1 ONLINE 0 0 0
c5d0 ONLINE 0 0 0
c5d1 ONLINE 0 0 0
logs
c11d0 ONLINE 0 0 0
I''m running OpenSolaris 2009.06 updated to build 118.
Basically the system won''t boot until I boot with a CD, zpool import -f
the pool and then zpool export it. Even that doesn''t really work.
When
I do the zpool import -f from the CD, the command runs forever (well, I
let it run for a good hour or so before I stopped it). The command
seems to be running in a loop. If I truss the process I see the following:
door_call(6, 0x0803D390) = 0
lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
close(6) = 0
resolvepath("/", "/", 1024) = 1
resolvepath("/", "/", 1024) = 1
open("/etc/dev/.devlink_db", O_RDONLY) = 6
fxstat(2, 6, 0x0803D870) = 0
mmap(0x00000000, 40, PROT_READ, MAP_SHARED, 6, 0) = 0xFE950000
mmap(0x00000000, 81920, PROT_READ, MAP_SHARED, 6, 45056) = 0xFE93B000
munmap(0xFE93B000, 81920) = 0
munmap(0xFE950000, 40) = 0
close(6) = 0
ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x0803EEF0) = 0
ioctl(3, ZFS_IOC_CREATE_MINOR, 0x0803D9B0) = 0
getppriv(PRIV_EFFECTIVE, {ffffffffffffffffffffffff}) = 0
open("/etc/devname_check_RDONLY", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 6
close(6) = 0
unlink("/etc/devname_check_RDONLY") = 0
xstat(2, "//etc/dev/.devfsadm_synch_door", 0x0803CF00) = 0
open("//etc/dev/.devfsadm_synch_door", O_RDONLY) = 6
lwp_sigmask(SIG_SETMASK, 0xFFBFFEFF, 0x0000FFF7) = 0xFFBFFEFF [0x0000FFFF]
door_call(6, 0x0803D390) = 0
lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
close(6) = 0
resolvepath("/", "/", 1024) = 1
resolvepath("/", "/", 1024) = 1
open("/etc/dev/.devlink_db", O_RDONLY) = 6
fxstat(2, 6, 0x0803D870) = 0
mmap(0x00000000, 40, PROT_READ, MAP_SHARED, 6, 0) = 0xFE950000
mmap(0x00000000, 81920, PROT_READ, MAP_SHARED, 6, 45056) = 0xFE93B000
munmap(0xFE93B000, 81920) = 0
munmap(0xFE950000, 40) = 0
close(6) = 0
ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x0803EEF0) = 0
ioctl(3, ZFS_IOC_CREATE_MINOR, 0x0803D9B0) = 0
getppriv(PRIV_EFFECTIVE, {ffffffffffffffffffffffff}) = 0
open("/etc/devname_check_RDONLY", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 6
close(6) = 0
unlink("/etc/devname_check_RDONLY") = 0
xstat(2, "//etc/dev/.devfsadm_synch_door", 0x0803CF00) = 0
open("//etc/dev/.devfsadm_synch_door", O_RDONLY) = 6
lwp_sigmask(SIG_SETMASK, 0xFFBFFEFF, 0x0000FFF7) = 0xFFBFFEFF [0x0000FFFF]
door_call(6, 0x0803D390) = 0
This sequence is repeated over and over. I realize now that I should
have done a pfiles on the process to figure out what the file
descriptors were mapping to. I can do that if it will help diagnose this.
If I kill the zpool import at this point, all of my filesystems are
there and mounted and everything seems to be fine. I''m going to scrub
the pool over night tonight, but I''d appreciate any suggestions as to
how to fix this problem in the future.
Steve Green
Hi Stephen, Have you got many zvols (or snapshots of zvols) in your pool? You could be running into CR 6761786 and/or 6693210. On Thu, 27 Aug 2009, Stephen Green wrote:> I''m having trouble booting with one of my zpools. It looks like this: > > pool: tank > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > raidz1 ONLINE 0 0 0 > c4d0 ONLINE 0 0 0 > c4d1 ONLINE 0 0 0 > c5d0 ONLINE 0 0 0 > c5d1 ONLINE 0 0 0 > logs > c11d0 ONLINE 0 0 0 > > I''m running OpenSolaris 2009.06 updated to build 118. > > Basically the system won''t boot until I boot with a CD, zpool import -f the > pool and then zpool export it. Even that doesn''t really work. When I do the > zpool import -f from the CD, the command runs forever (well, I let it run for > a good hour or so before I stopped it). The command seems to be running in a > loop. If I truss the process I see the following: > > door_call(6, 0x0803D390) = 0 > lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF] > close(6) = 0 > resolvepath("/", "/", 1024) = 1 > resolvepath("/", "/", 1024) = 1 > open("/etc/dev/.devlink_db", O_RDONLY) = 6 > fxstat(2, 6, 0x0803D870) = 0 > mmap(0x00000000, 40, PROT_READ, MAP_SHARED, 6, 0) = 0xFE950000 > mmap(0x00000000, 81920, PROT_READ, MAP_SHARED, 6, 45056) = 0xFE93B000 > munmap(0xFE93B000, 81920) = 0 > munmap(0xFE950000, 40) = 0 > close(6) = 0 > ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x0803EEF0) = 0 > ioctl(3, ZFS_IOC_CREATE_MINOR, 0x0803D9B0) = 0 > getppriv(PRIV_EFFECTIVE, {ffffffffffffffffffffffff}) = 0 > open("/etc/devname_check_RDONLY", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 6 > close(6) = 0 > unlink("/etc/devname_check_RDONLY") = 0 > xstat(2, "//etc/dev/.devfsadm_synch_door", 0x0803CF00) = 0 > open("//etc/dev/.devfsadm_synch_door", O_RDONLY) = 6 > lwp_sigmask(SIG_SETMASK, 0xFFBFFEFF, 0x0000FFF7) = 0xFFBFFEFF [0x0000FFFF] > door_call(6, 0x0803D390) = 0 > lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF] > close(6) = 0 > resolvepath("/", "/", 1024) = 1 > resolvepath("/", "/", 1024) = 1 > open("/etc/dev/.devlink_db", O_RDONLY) = 6 > fxstat(2, 6, 0x0803D870) = 0 > mmap(0x00000000, 40, PROT_READ, MAP_SHARED, 6, 0) = 0xFE950000 > mmap(0x00000000, 81920, PROT_READ, MAP_SHARED, 6, 45056) = 0xFE93B000 > munmap(0xFE93B000, 81920) = 0 > munmap(0xFE950000, 40) = 0 > close(6) = 0 > ioctl(3, ZFS_IOC_SNAPSHOT_LIST_NEXT, 0x0803EEF0) = 0 > ioctl(3, ZFS_IOC_CREATE_MINOR, 0x0803D9B0) = 0 > getppriv(PRIV_EFFECTIVE, {ffffffffffffffffffffffff}) = 0 > open("/etc/devname_check_RDONLY", O_WRONLY|O_CREAT|O_TRUNC, 0644) = 6 > close(6) = 0 > unlink("/etc/devname_check_RDONLY") = 0 > xstat(2, "//etc/dev/.devfsadm_synch_door", 0x0803CF00) = 0 > open("//etc/dev/.devfsadm_synch_door", O_RDONLY) = 6 > lwp_sigmask(SIG_SETMASK, 0xFFBFFEFF, 0x0000FFF7) = 0xFFBFFEFF [0x0000FFFF] > door_call(6, 0x0803D390) = 0 > > This sequence is repeated over and over. I realize now that I should have > done a pfiles on the process to figure out what the file descriptors were > mapping to. I can do that if it will help diagnose this. > > If I kill the zpool import at this point, all of my filesystems are there and > mounted and everything seems to be fine. I''m going to scrub the pool over > night tonight, but I''d appreciate any suggestions as to how to fix this > problem in the future. > > Steve Green > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >Regards, markm
Mark J Musante wrote:> > Hi Stephen, > > Have you got many zvols (or snapshots of zvols) in your pool? You could > be running into CR 6761786 and/or 6693210.There are four volumes on that pool: stgreen at blue:/tank/tivo/videos$ zfs list -t volume NAME USED AVAIL REFER MOUNTPOINT rpool/dump 6.00G 350G 6.00G - rpool/swap 6.00G 355G 618M - tank/iscsi/mac-backup 119G 1.73T 119G - tank/iscsi/macup 423G 2.02T 118G - tank/iscsi/macup-newer 30.6G 1.73T 143G - tank/iscsi/macup-recover 25.1M 1.73T 115G - These are the iscsi targets that I use for backing up the mac. They are auto-snapshotted (Time machine + ZFS Time slider!):> stgreen at blue:/tank/iscsi$ zfs list -rt snapshot tank/iscsi | wc > 6243 31215 555636So, yeah, there''s a lot of snapshots. I''ll have a look at those bugs. I guess I should turn of the auto snapshots and clear out the old ones, but those snapshots saved my behind when the wife''s mac went crazy... Thanks! Steve -- Stephen Green // Stephen.Green at sun.com Principal Investigator \\ http://blogs.sun.com/searchguy The AURA Project // Voice: +1 781-442-0926 Sun Microsystems Labs \\ Fax: +1 781-442-0399