Hi there,
We''ve been working with ZFS at an initial setup stage hoping to
eventually integrate with Sun Cluster 3.2 and create a failover fs. Somehow
between my two machines I managed to get the file system mounted on both. On
reboot of both machines I can now no longer import my ZFS file systems. ZFS
reports they''re online (which they were, albeit before the reboot) but
when I perform an import I''m told the pool "may be active".
When trying a zpool import -f the machine kernel panics and instantly reboots.
I''m at a loss as to how to mount this file system properly again.
Please help! :-|
[root at solaris1 ~]$ zpool import
pool: ax150s
id: 15868755553787799685
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
The pool may be active on on another system, but can be imported using
the ''-f'' flag.
config:
ax150s ONLINE
mirror ONLINE
c4t6006016071851800B86C8EE05831DB11d0 ONLINE
c4t6006016031E0180032F8E9868E30DB11d0 ONLINE
mirror ONLINE
c4t6006016071851800CA1D94EF5831DB11d0 ONLINE
c4t6006016031E0180026057F9B8E30DB11d0 ONLINE
mirror ONLINE
c4t6006016031E018003810E7AC8E30DB11d0 ONLINE
c4t60060160718518009A7926FF5831DB11d0 ONLINE
mirror ONLINE
c4t6006016031E01800AC7E34918E30DB11d0 ONLINE
c4t600601607185180010A65FE75831DB11d0 ONLINE
mirror ONLINE
c4t6006016031E018005A9B74A48E30DB11d0 ONLINE
c4t600601607185180064063BF85831DB11d0 ONLINE
[root at solaris1 ~]$ zpool import ax150s
cannot import ''ax150s'': pool may be in use from other system
use ''-f'' to import anyway
[root at solaris1 ~]$ zpool import -f ax150s
Read from remote host solaris1: Connection reset by peer
Connection to solaris1 closed.
[stuart at solarishead ~]$
Any pointers muchly appreciated! :-|
Stuart
This message posted from opensolaris.org
Stuart Low wrote:> Hi there, > > We''ve been working with ZFS at an initial setup stage hoping to eventually integrate with Sun Cluster 3.2 and create a failover fs. Somehow between my two machines I managed to get the file system mounted on both. On reboot of both machines I can now no longer import my ZFS file systems. ZFS reports they''re online (which they were, albeit before the reboot) but when I perform an import I''m told the pool "may be active". > > When trying a zpool import -f the machine kernel panics and instantly reboots. I''m at a loss as to how to mount this file system properly again. Please help! :-| > > [root at solaris1 ~]$ zpool import > pool: ax150s > id: 15868755553787799685 > state: ONLINE > action: The pool can be imported using its name or numeric identifier. > The pool may be active on on another system, but can be imported using > the ''-f'' flag....> Any pointers muchly appreciated! :-|Did you try a zpool export on either or both machines? James C. McPherson
Well I would, if it let me. :) [root at solaris1 ~]$ zpool export ax150s cannot open ''ax150s'': no such pool [root at solaris1 ~]$ By it''s own admission it''s Online but it can''t find it within it''s pool list? :-| Stuart This message posted from opensolaris.org
Stuart Low wrote:> Well I would, if it let me. :) > > [root at solaris1 ~]$ zpool export ax150s > cannot open ''ax150s'': no such pool > [root at solaris1 ~]$ > > By it''s own admission it''s Online but it can''t find it within it''s pool list? :-|Darn. What about a "zpool export -f ax150s" ? James
Nada. [root at solaris1 ~]$ zpool export -f ax150s cannot open ''ax150s'': no such pool [root at solaris1 ~]$ I wonder if it''s possible to force the pool to be marked as inactive? Ideally all I want to do is get it back online then scrub it for errors. :-| Stuart This message posted from opensolaris.org
Stuart Low wrote:> Nada. > > [root at solaris1 ~]$ zpool export -f ax150s > cannot open ''ax150s'': no such pool > [root at solaris1 ~]$ > > I wonder if it''s possible to force the pool to be marked as inactive? Ideally all I want to do is get it back online then scrub it for errors. :-|At this point it might be reasonable to copy your /etc/zfs/zpool.cache to /etc/zfs/zpool.cache.BORKED (on both machines), and then on one box run zpool import ax150s. Then if that works, export it on that box, then try the same op on the other. Failing that ... might just have to wait for TimF or somebody else from TeamZFS to get online. James
I thought that might work too but having tried the move of zpool.cache alas same problem. :( Stuart This message posted from opensolaris.org
Hi Stuart, Stuart Low wrote:> Hi there, > > We''ve been working with ZFS at an initial setup stage hoping to > eventually integrate with Sun Cluster 3.2 and create a failover fs. > Somehow between my two machines I managed to get the file system > mounted on both. On reboot of both machines I can now no longer > import my ZFS file systems. ZFS reports they''re online (which they > were, albeit before the reboot) but when I perform an import I''m told > the pool "may be active". >Just curious: Are you seeing this problem with SC3.2, or just plain Solaris? Thanks, Zoram -- Zoram Thanga, Sun Cluster Development.
Heya, SC3.1 until we can get our hands on SC3.2 beta. Realistically the Cluster itself is operating independent of the ZFS pools (we do manual failover). Stu This message posted from opensolaris.org
Ok. Can you post the panic stack backtrace? Perhaps it''s a known problem... Stuart Low wrote:> Heya, > > SC3.1 until we can get our hands on SC3.2 beta. Realistically the Cluster itself is operating independent of the ZFS pools (we do manual failover). > > Stu > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-- Zoram Thanga, Sun Cluster Development.
Stuart, Is it possible that you ran ''zpool import'' on node1 and then failed it over to the other node which ran ''zpool import'' on node2? If so, then the pool configuration was automatically added to zpool.cache so that the pool could be automatically loaded upon reboot. This may result in the pool being imported on both node at the same time. What you need to do is run ''zpool import -R <altroot>'' instead as this prevent the pool from being added to the cache. You should also ensure that the zpool.cache file does not exist on either node and that the import is driven by your failover scripts only. Thanks, George Stuart Low wrote:> Hi there, > > We''ve been working with ZFS at an initial setup stage hoping to eventually integrate with Sun Cluster 3.2 and create a failover fs. Somehow between my two machines I managed to get the file system mounted on both. On reboot of both machines I can now no longer import my ZFS file systems. ZFS reports they''re online (which they were, albeit before the reboot) but when I perform an import I''m told the pool "may be active". > > When trying a zpool import -f the machine kernel panics and instantly reboots. I''m at a loss as to how to mount this file system properly again. Please help! :-| > > [root at solaris1 ~]$ zpool import > pool: ax150s > id: 15868755553787799685 > state: ONLINE > action: The pool can be imported using its name or numeric identifier. > The pool may be active on on another system, but can be imported using > the ''-f'' flag. > config: > > ax150s ONLINE > mirror ONLINE > c4t6006016071851800B86C8EE05831DB11d0 ONLINE > c4t6006016031E0180032F8E9868E30DB11d0 ONLINE > mirror ONLINE > c4t6006016071851800CA1D94EF5831DB11d0 ONLINE > c4t6006016031E0180026057F9B8E30DB11d0 ONLINE > mirror ONLINE > c4t6006016031E018003810E7AC8E30DB11d0 ONLINE > c4t60060160718518009A7926FF5831DB11d0 ONLINE > mirror ONLINE > c4t6006016031E01800AC7E34918E30DB11d0 ONLINE > c4t600601607185180010A65FE75831DB11d0 ONLINE > mirror ONLINE > c4t6006016031E018005A9B74A48E30DB11d0 ONLINE > c4t600601607185180064063BF85831DB11d0 ONLINE > > > > [root at solaris1 ~]$ zpool import ax150s > cannot import ''ax150s'': pool may be in use from other system > use ''-f'' to import anyway > [root at solaris1 ~]$ zpool import -f ax150s > Read from remote host solaris1: Connection reset by peer > Connection to solaris1 closed. > [stuart at solarishead ~]$ > > Any pointers muchly appreciated! :-| > > Stuart > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Stuart, Can you send the output of ''zpool status -v'' from both nodes? Thanks, George Stuart Low wrote:> Nada. > > [root at solaris1 ~]$ zpool export -f ax150s > cannot open ''ax150s'': no such pool > [root at solaris1 ~]$ > > I wonder if it''s possible to force the pool to be marked as inactive? Ideally all I want to do is get it back online then scrub it for errors. :-| > > Stuart > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Stuart, Given that the pool was imported on both nodes simultaneously may have corrupted it beyond repair. I''m assuming that the "same problem" is a system panic? If so, can you send the panic string from that node? Thanks, George Stuart Low wrote:> I thought that might work too but having tried the move of zpool.cache alas same problem. :( > > Stuart > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[root at solaris1 ~]$ zpool status -v no pools available [root at solaris1 ~]$ [root at solaris2 ~]$ zpool status -v no pools available [root at solaris2 ~]$ It''s like it''s "not there" but when I do a zpool import it reports it as there and available just that I need to use -f. Use -f gives me instareboot. :) Stuart This message posted from opensolaris.org
Stuart, Issuing a ''zpool import'' will show all the pools which are accessible for import and that''s why you are seeing them. The fact that a forced import gives results in a panic is indicative of pool corruption that resulted from being imported on more than one host. Thanks, George Stuart Low wrote:> [root at solaris1 ~]$ zpool status -v > no pools available > [root at solaris1 ~]$ > > [root at solaris2 ~]$ zpool status -v > no pools available > [root at solaris2 ~]$ > > It''s like it''s "not there" but when I do a zpool import it reports it as > there and available just that I need to use -f. Use -f gives me > instareboot. :) > > Stuart > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss