I''m trying to import a pool I just exported but I can''t, even
-f doesn''t help. Every time I try I''m getting an error:
"cannot import ''rc-pool'': one or more devices is
currently unavailable"
Now I suspect the reason it''s not happy is that the pool used to have a
ZIL :)
However I know the pool works fine without the ZIL as my ZIL was a ramdisk and
the server''s been rebooted a good few times since doing that. There
have been no problems mounting the pool and each time after booting if I wanted
a ramdisk ZIL again I just created a new one and ran zpool replace to add it to
the pool.
The reason I exported the pool was because my ramdisk was called
''test'' and I wanted to replace it with one called
''rc-pool-zil''. I didn''t have enough RAM to have two
disks online at once so to save a reboot I exported the pool, deleted the
''test'' ramdisk and created my new one.
The only problem is that I can''t import the pool any more.
I''m beginning to think I should have rebooted instead...
The output of the relevant commands is:
# zpool import
pool: rc-pool
id: 11547678520047091246
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
see: http://www.sun.com/msg/ZFS-8000-6X
config:
rc-pool UNAVAIL missing device
mirror ONLINE
c1t1d0 ONLINE
c2t0d0 ONLINE
c1t2d0 ONLINE
mirror DEGRADED
c2t1d0 UNAVAIL cannot open
c1t3d0 ONLINE
c2t2d0 ONLINE
mirror ONLINE
c1t4d0 ONLINE
c2t3d0 ONLINE
c1t5d0 ONLINE
mirror ONLINE
c2t4d0 ONLINE
c1t6d0 ONLINE
c2t5d0 ONLINE
mirror ONLINE
c1t7d0 ONLINE
c2t6d0 ONLINE
c2t7d0 ONLINE
Additional devices are known to be part of this pool, though their
exact configuration cannot be determined.
# zpool import -f rc-pool
cannot import ''rc-pool'': one or more devices is currently
unavailable
Even after a reboot I get the same error. Creating a ramdisk again with the
original name doesn''t seem to do anything either. Checking the Sun
error message it says a top level device isn''t available and the pool
can''t be opened. Well, the only thing it could possibly be is the ZIL,
which seems odd since ZFS could certainly mount the pool fine without it on
boot. Can ZFS really not import a pool if the ZIL is missing?
If so, doesn''t that make this a potential issue when people do start
using separate media for the ZIL? If the ZIL hardware fails for any reason do
you loose access to your entire pool? What if you''re using battery
backed nvram but power is off long enough to wipe the device? What if a server
fails and you need to move the disks to a new chassis?
Can anybody help me get this pool back online?
thanks,
Ross
This message posted from opensolaris.org
Richard Elling
2008-Aug-04 16:24 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
Ross wrote:> I''m trying to import a pool I just exported but I can''t, even -f doesn''t help. Every time I try I''m getting an error: > "cannot import ''rc-pool'': one or more devices is currently unavailable" > > Now I suspect the reason it''s not happy is that the pool used to have a ZIL :) >Correct. What you want is CR 6707530, log device failure needs some work http://bugs.opensolaris.org/view_bug.do?bug_id=6707530 which Neil has been working on, scheduled for b96.> However I know the pool works fine without the ZIL as my ZIL was a ramdisk and the server''s been rebooted a good few times since doing that. There have been no problems mounting the pool and each time after booting if I wanted a ramdisk ZIL again I just created a new one and ran zpool replace to add it to the pool. > > The reason I exported the pool was because my ramdisk was called ''test'' and I wanted to replace it with one called ''rc-pool-zil''. I didn''t have enough RAM to have two disks online at once so to save a reboot I exported the pool, deleted the ''test'' ramdisk and created my new one. > > The only problem is that I can''t import the pool any more. I''m beginning to think I should have rebooted instead... >It still would have failed.> The output of the relevant commands is: > > # zpool import > pool: rc-pool > id: 11547678520047091246 > state: UNAVAIL > status: One or more devices are missing from the system. > action: The pool cannot be imported. Attach the missing > devices and try again. > see: http://www.sun.com/msg/ZFS-8000-6X > config: > > rc-pool UNAVAIL missing device > mirror ONLINE > c1t1d0 ONLINE > c2t0d0 ONLINE > c1t2d0 ONLINE > mirror DEGRADED > c2t1d0 UNAVAIL cannot open > c1t3d0 ONLINE > c2t2d0 ONLINE > mirror ONLINE > c1t4d0 ONLINE > c2t3d0 ONLINE > c1t5d0 ONLINE > mirror ONLINE > c2t4d0 ONLINE > c1t6d0 ONLINE > c2t5d0 ONLINE > mirror ONLINE > c1t7d0 ONLINE > c2t6d0 ONLINE > c2t7d0 ONLINE > > Additional devices are known to be part of this pool, though their > exact configuration cannot be determined. > > # zpool import -f rc-pool > cannot import ''rc-pool'': one or more devices is currently unavailable > > Even after a reboot I get the same error. Creating a ramdisk again with the original name doesn''t seem to do anything either. Checking the Sun error message it says a top level device isn''t available and the pool can''t be opened. Well, the only thing it could possibly be is the ZIL, which seems odd since ZFS could certainly mount the pool fine without it on boot. Can ZFS really not import a pool if the ZIL is missing? >Yes. Pending the above fix. Prior to the fix, the separate ZIL log was considered to be another top-level vdev. This is why it refuses to import, as it believes it is missing a top-level vdev instead of a (potentially empty) ZIL.> If so, doesn''t that make this a potential issue when people do start using separate media for the ZIL? If the ZIL hardware fails for any reason do you loose access to your entire pool? What if you''re using battery backed nvram but power is off long enough to wipe the device? What if a server fails and you need to move the disks to a new chassis? > > Can anybody help me get this pool back online? >If you look in the archives, there was a way to work around this discussed a few months ago. It isn''t pretty, and the real fix is in the above CR. -- richard
Neil Perrin
2008-Aug-04 16:42 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
Richard Elling wrote:> Ross wrote: >> I''m trying to import a pool I just exported but I can''t, even -f doesn''t help. Every time I try I''m getting an error: >> "cannot import ''rc-pool'': one or more devices is currently unavailable" >> >> Now I suspect the reason it''s not happy is that the pool used to have a ZIL :) >> > > Correct. What you want is CR 6707530, log device failure needs some work > http://bugs.opensolaris.org/view_bug.do?bug_id=6707530 > which Neil has been working on, scheduled for b96.Actually no. That CR mentioned the problem and talks about splitting out the bug, as it''s really a separate problem. I''ve just done that and here''s the new CR which probably won''t be visible immediately to you: 6733267 Allow a pool to be imported with a missing slog Here''s the Description: ------------------- This CR is being broken out from 6707530 "log device failure needs some work" When Separate Intent logs (slogs) were designed they were given equal status in the pool device tree. This was because they can contain committed changes to the pool. So if one is missing it is assumed to be important to the integrity of the application(s) that wanted the data committed synchronously, and thus a pool cannot be imported with a missing slog. However, we do allow a pool to be missing a slog on boot up if it''s in the /etc/zfs/zpool.cache file. So this sends a mixed message. We should allow a pool to be imported without a slog if -f is used and to not import without "-f" but perhaps with a better error message. It''s the guidsum check that actually rejects imports with missing devices. We could have a separate guidsum for the main pool devices (non slog/cache). -----------------------
Ross Smith
2008-Aug-05 07:33 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
Just a thought, before I go and wipe this zpool, is there any way to manually recreate the /etc/zfs/zpool.cache file? Ross> Date: Mon, 4 Aug 2008 10:42:43 -0600> From: Neil.Perrin at Sun.COM> Subject: Re: [zfs-discuss] Zpool import not working - I broke my pool...> To: Richard.Elling at Sun.COM; myxiplx at hotmail.com> CC: zfs-discuss at opensolaris.org> > > > Richard Elling wrote:> > Ross wrote:> >> I''m trying to import a pool I just exported but I can''t, even -f doesn''t help. Every time I try I''m getting an error:> >> "cannot import ''rc-pool'': one or more devices is currently unavailable"> >>> >> Now I suspect the reason it''s not happy is that the pool used to have a ZIL :)> >> > > > > Correct. What you want is CR 6707530, log device failure needs some work> > http://bugs.opensolaris.org/view_bug.do?bug_id=6707530> > which Neil has been working on, scheduled for b96.> > Actually no. That CR mentioned the problem and talks about splitting out> the bug, as it''s really a separate problem. I''ve just done that and here''s> the new CR which probably won''t be visible immediately to you:> > 6733267 Allow a pool to be imported with a missing slog> > Here''s the Description:> > -------------------> This CR is being broken out from 6707530 "log device failure needs some work"> > When Separate Intent logs (slogs) were designed they were given equal status in the pool device tree.> This was because they can contain committed changes to the pool.> So if one is missing it is assumed to be important to the integrity of the> application(s) that wanted the data committed synchronously, and thus> a pool cannot be imported with a missing slog.> However, we do allow a pool to be missing a slog on boot up if> it''s in the /etc/zfs/zpool.cache file. So this sends a mixed message.> > We should allow a pool to be imported without a slog if -f is used> and to not import without "-f" but perhaps with a better error message.> > It''s the guidsum check that actually rejects imports with missing devices.> We could have a separate guidsum for the main pool devices (non slog/cache).> -----------------------> _________________________________________________________________ Win a voice over part with Kung Fu Panda & Live Search?? and?? 100?s of Kung Fu Panda prizes to win with Live Search http://clk.atdmt.com/UKM/go/107571439/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080805/18bb576b/attachment.html>
Richard Elling
2008-Aug-05 14:59 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
Ross Smith wrote:> Just a thought, before I go and wipe this zpool, is there any way to > manually recreate the /etc/zfs/zpool.cache file?Do you have a copy in a snapshot? ZFS for root is awesome! -- richard> > Ross > > > Date: Mon, 4 Aug 2008 10:42:43 -0600 > > From: Neil.Perrin at Sun.COM > > Subject: Re: [zfs-discuss] Zpool import not working - I broke my pool... > > To: Richard.Elling at Sun.COM; myxiplx at hotmail.com > > CC: zfs-discuss at opensolaris.org > > > > > > > > Richard Elling wrote: > > > Ross wrote: > > >> I''m trying to import a pool I just exported but I can''t, even -f > doesn''t help. Every time I try I''m getting an error: > > >> "cannot import ''rc-pool'': one or more devices is currently > unavailable" > > >> > > >> Now I suspect the reason it''s not happy is that the pool used to > have a ZIL :) > > >> > > > > > > Correct. What you want is CR 6707530, log device failure needs > some work > > > http://bugs.opensolaris.org/view_bug.do?bug_id=6707530 > > > which Neil has been working on, scheduled for b96. > > > > Actually no. That CR mentioned the problem and talks about splitting out > > the bug, as it''s really a separate problem. I''ve just done that and > here''s > > the new CR which probably won''t be visible immediately to you: > > > > 6733267 Allow a pool to be imported with a missing slog > > > > Here''s the Description: > > > > ------------------- > > This CR is being broken out from 6707530 "log device failure needs > some work" > > > > When Separate Intent logs (slogs) were designed they were given > equal status in the pool device tree. > > This was because they can contain committed changes to the pool. > > So if one is missing it is assumed to be important to the integrity > of the > > application(s) that wanted the data committed synchronously, and thus > > a pool cannot be imported with a missing slog. > > However, we do allow a pool to be missing a slog on boot up if > > it''s in the /etc/zfs/zpool.cache file. So this sends a mixed message. > > > > We should allow a pool to be imported without a slog if -f is used > > and to not import without "-f" but perhaps with a better error message. > > > > It''s the guidsum check that actually rejects imports with missing > devices. > > We could have a separate guidsum for the main pool devices (non > slog/cache). > > ----------------------- > > > > > ------------------------------------------------------------------------ > Get Hotmail on your mobile from Vodafone Try it Now! > <http://clk.atdmt.com/UKM/go/107571435/direct/01/>
Ross Smith
2008-Aug-05 15:23 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
No, but that''s a great idea! I''m on a UFS root at the moment, will have a look at using ZFS next time I re-install.> Date: Tue, 5 Aug 2008 07:59:35 -0700> From: Richard.Elling at Sun.COM> Subject: Re: [zfs-discuss] Zpool import not working - I broke my pool...> To: myxiplx at hotmail.com> CC: Neil.Perrin at Sun.COM; zfs-discuss at opensolaris.org> > Ross Smith wrote:> > Just a thought, before I go and wipe this zpool, is there any way to > > manually recreate the /etc/zfs/zpool.cache file?> > Do you have a copy in a snapshot? ZFS for root is awesome!> -- richard> > > > > Ross> >> > > Date: Mon, 4 Aug 2008 10:42:43 -0600> > > From: Neil.Perrin at Sun.COM> > > Subject: Re: [zfs-discuss] Zpool import not working - I broke my pool...> > > To: Richard.Elling at Sun.COM; myxiplx at hotmail.com> > > CC: zfs-discuss at opensolaris.org> > >> > >> > >> > > Richard Elling wrote:> > > > Ross wrote:> > > >> I''m trying to import a pool I just exported but I can''t, even -f > > doesn''t help. Every time I try I''m getting an error:> > > >> "cannot import ''rc-pool'': one or more devices is currently > > unavailable"> > > >>> > > >> Now I suspect the reason it''s not happy is that the pool used to > > have a ZIL :)> > > >>> > > >> > > > Correct. What you want is CR 6707530, log device failure needs > > some work> > > > http://bugs.opensolaris.org/view_bug.do?bug_id=6707530> > > > which Neil has been working on, scheduled for b96.> > >> > > Actually no. That CR mentioned the problem and talks about splitting out> > > the bug, as it''s really a separate problem. I''ve just done that and > > here''s> > > the new CR which probably won''t be visible immediately to you:> > >> > > 6733267 Allow a pool to be imported with a missing slog> > >> > > Here''s the Description:> > >> > > -------------------> > > This CR is being broken out from 6707530 "log device failure needs > > some work"> > >> > > When Separate Intent logs (slogs) were designed they were given > > equal status in the pool device tree.> > > This was because they can contain committed changes to the pool.> > > So if one is missing it is assumed to be important to the integrity > > of the> > > application(s) that wanted the data committed synchronously, and thus> > > a pool cannot be imported with a missing slog.> > > However, we do allow a pool to be missing a slog on boot up if> > > it''s in the /etc/zfs/zpool.cache file. So this sends a mixed message.> > >> > > We should allow a pool to be imported without a slog if -f is used> > > and to not import without "-f" but perhaps with a better error message.> > >> > > It''s the guidsum check that actually rejects imports with missing > > devices.> > > We could have a separate guidsum for the main pool devices (non > > slog/cache).> > > -----------------------> > >> >> >> > ------------------------------------------------------------------------> > Get Hotmail on your mobile from Vodafone Try it Now! > > <http://clk.atdmt.com/UKM/go/107571435/direct/01/>>_________________________________________________________________ Get Hotmail on your mobile from Vodafone http://clk.atdmt.com/UKM/go/107571435/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080805/6627ca45/attachment.html>
Ross Smith
2008-Aug-06 09:04 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
Hmm... got a bit more information for you to add to that bug I think. Zpool import also doesn''t work if you have mirrored log devices and either one of them is offline. I created two ramdisks with: # ramdiskadm -a rc-pool-zil-1 256m # ramdiskadm -a rc-pool-zil-2 256m And added them to the pool with: # zpool add rc-pool log mirror /dev/ramdisk/rc-pool-zil-1 /dev/ramdisk/rc-pool-zil-2 I can reboot fine, the pool imports ok without the ZIL and I have a script that recreates the ramdisks and adds them back to the pool:#!/sbin/shstate="$1"case "$state" in''start'') echo ''Starting Ramdisks'' /usr/sbin/ramdiskadm -a rc-pool-zil-1 256m /usr/sbin/ramdiskadm -a rc-pool-zil-2 256m echo ''Attaching to ZFS ZIL'' /usr/sbin/zpool replace test /dev/ramdisk/rc-pool-zil-1 /usr/sbin/zpool replace test /dev/ramdisk/rc-pool-zil-2 ;;''stop'') ;;esac However, if I export the pool, and delete one ramdisk to check that the mirroring works fine, the import fails: # zpool export rc-pool # ramdiskadm -d rc-pool-zil-1 # zpool import rc-pool cannot import ''rc-pool'': one or more devices is currently unavailable Ross> Date: Mon, 4 Aug 2008 10:42:43 -0600> From: Neil.Perrin at Sun.COM> Subject: Re: [zfs-discuss] Zpool import not working - I broke my pool...> To: Richard.Elling at Sun.COM; myxiplx at hotmail.com> CC: zfs-discuss at opensolaris.org> > > > Richard Elling wrote:> > Ross wrote:> >> I''m trying to import a pool I just exported but I can''t, even -f doesn''t help. Every time I try I''m getting an error:> >> "cannot import ''rc-pool'': one or more devices is currently unavailable"> >>> >> Now I suspect the reason it''s not happy is that the pool used to have a ZIL :)> >> > > > > Correct. What you want is CR 6707530, log device failure needs some work> > http://bugs.opensolaris.org/view_bug.do?bug_id=6707530> > which Neil has been working on, scheduled for b96.> > Actually no. That CR mentioned the problem and talks about splitting out> the bug, as it''s really a separate problem. I''ve just done that and here''s> the new CR which probably won''t be visible immediately to you:> > 6733267 Allow a pool to be imported with a missing slog> > Here''s the Description:> > -------------------> This CR is being broken out from 6707530 "log device failure needs some work"> > When Separate Intent logs (slogs) were designed they were given equal status in the pool device tree.> This was because they can contain committed changes to the pool.> So if one is missing it is assumed to be important to the integrity of the> application(s) that wanted the data committed synchronously, and thus> a pool cannot be imported with a missing slog.> However, we do allow a pool to be missing a slog on boot up if> it''s in the /etc/zfs/zpool.cache file. So this sends a mixed message.> > We should allow a pool to be imported without a slog if -f is used> and to not import without "-f" but perhaps with a better error message.> > It''s the guidsum check that actually rejects imports with missing devices.> We could have a separate guidsum for the main pool devices (non slog/cache).> ----------------------->_________________________________________________________________ Get Hotmail on your mobile from Vodafone http://clk.atdmt.com/UKM/go/107571435/direct/01/ -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20080806/b7d33952/attachment.html>
Neil Perrin
2008-Aug-06 17:15 UTC
[zfs-discuss] Zpool import not working - I broke my pool...
Ross, Thanks, I have updated the bug with this info. Neil. Ross Smith wrote:> Hmm... got a bit more information for you to add to that bug I think. > > Zpool import also doesn''t work if you have mirrored log devices and > either one of them is offline. > > I created two ramdisks with: > # ramdiskadm -a rc-pool-zil-1 256m > # ramdiskadm -a rc-pool-zil-2 256m > > And added them to the pool with: > # zpool add rc-pool log mirror /dev/ramdisk/rc-pool-zil-1 > /dev/ramdisk/rc-pool-zil-2 > > I can reboot fine, the pool imports ok without the ZIL and I have a > script that recreates the ramdisks and adds them back to the pool: > #!/sbin/sh > state="$1" > case "$state" in > ''start'') > echo ''Starting Ramdisks'' > /usr/sbin/ramdiskadm -a rc-pool-zil-1 256m > /usr/sbin/ramdiskadm -a rc-pool-zil-2 256m > echo ''Attaching to ZFS ZIL'' > /usr/sbin/zpool replace test /dev/ramdisk/rc-pool-zil-1 > /usr/sbin/zpool replace test /dev/ramdisk/rc-pool-zil-2 > ;; > ''stop'') > ;; > esac > > However, if I export the pool, and delete one ramdisk to check that the > mirroring works fine, the import fails: > # zpool export rc-pool > # ramdiskadm -d rc-pool-zil-1 > # zpool import rc-pool > cannot import ''rc-pool'': one or more devices is currently unavailable > > Ross > > > > Date: Mon, 4 Aug 2008 10:42:43 -0600 > > From: Neil.Perrin at Sun.COM > > Subject: Re: [zfs-discuss] Zpool import not working - I broke my pool... > > To: Richard.Elling at Sun.COM; myxiplx at hotmail.com > > CC: zfs-discuss at opensolaris.org > > > > > > > > Richard Elling wrote: > > > Ross wrote: > > >> I''m trying to import a pool I just exported but I can''t, even -f > doesn''t help. Every time I try I''m getting an error: > > >> "cannot import ''rc-pool'': one or more devices is currently > unavailable" > > >> > > >> Now I suspect the reason it''s not happy is that the pool used to > have a ZIL :) > > >> > > > > > > Correct. What you want is CR 6707530, log device failure needs some > work > > > http://bugs.opensolaris.org/view_bug.do?bug_id=6707530 > > > which Neil has been working on, scheduled for b96. > > > > Actually no. That CR mentioned the problem and talks about splitting out > > the bug, as it''s really a separate problem. I''ve just done that and > here''s > > the new CR which probably won''t be visible immediately to you: > > > > 6733267 Allow a pool to be imported with a missing slog > > > > Here''s the Description: > > > > ------------------- > > This CR is being broken out from 6707530 "log device failure needs > some work" > > > > When Separate Intent logs (slogs) were designed they were given equal > status in the pool device tree. > > This was because they can contain committed changes to the pool. > > So if one is missing it is assumed to be important to the integrity > of the > > application(s) that wanted the data committed synchronously, and thus > > a pool cannot be imported with a missing slog. > > However, we do allow a pool to be missing a slog on boot up if > > it''s in the /etc/zfs/zpool.cache file. So this sends a mixed message. > > > > We should allow a pool to be imported without a slog if -f is used > > and to not import without "-f" but perhaps with a better error message. > > > > It''s the guidsum check that actually rejects imports with missing > devices. > > We could have a separate guidsum for the main pool devices (non > slog/cache). > > ----------------------- > > > > > ------------------------------------------------------------------------ > Find out how to make Messenger your very own TV! Try it Now! > <http://clk.atdmt.com/UKM/go/101719648/direct/01/>
Any updates on this ? I created a pool in 5/08, then added a slog device which sadly failed. I can no longer mount the pool, it gives a "cannot import ''mypool'': one or more devices is currently unavailable". I have tried it with the latest OpenSolaris pre-release (2008.11, based on Nevada build 99) and still no luck. Please advise, James -- This message posted from opensolaris.org
If you have any backups of your boot volume, I found that the pool can be mounted on boot provided it''s still listed in your /etc/zfs/zpool.cache file. I''ve moved to OpenSolaris now purely so I can take snapshots of my boot volume and backup that file. The relevant bug you need fixing is this one, but I''ve no idea how long it might take before that fix is done. http://bugs.opensolaris.org/view_bug.do?bug_id=6733267 -- This message posted from opensolaris.org