zfs discuss - Jun 2011 - zpool import crashs SX11 trying to recovering a corrupted zpool

Hi 

I got following problem: duing a controller (LSI MegaRAID 9261-8i) outage I goot
a Solaris Express 11 zpool corrupted.

It is a whole 1,3 TB rpool zpool, RAID5 made by controller. 

Changing damaged controller the new one reports it be OPTIMAL. 

Important, zpool has got dedup enabled. 




If I try to: 

- boot it normally 
- rollback some ZILs with <zpool import -fFX> (launched by SXCD in rescue
mode)
- starting re-install from SXCD 

I got everytime system crash and instantelly reboot. 



Then I tried to check/manipulate it from OpenIndiana v148, and v151b, CDs in
rescue mode.

Trying <zpool import -fFX>, it reported zpool is present, but it has got a
newer formating version.

Launching zdb, it reporting me that label and other zpool information seem OK. 


But Solaris Express 11 has got zpool version 31, OpenIndiana (version 151 beta
too) reachs only version 28.



If standard SX11 is crashing seeing this zpool, and OI can''t manipulate
this newer zpool version, how can try to fix it via <zpool import -fFZ> (o
via whatever other tools)?

Exist a newer (patched) version of SXCD different from standard one downloadable
from Oracle web site?

Exist any "indipendent" Solaris distribution implementing zpool v31? 

Otherwise, have you got some other workaround to fix it? 




Thank you very much 

Stefano
-- 
This message posted from opensolaris.org

Did you try a read-only import as well? I THINK it goes like this:

zpool import -o ro -o cachefile=none -F -f badpool

Did you manage to capture any error output? For example, is it an option for you
to set up a serial console and copy-paste the error text from the serial
terminal on another machine?

As far as I know there are no other releases of Solaris 11 yet, and since the
code is now developed behind closed doors (due to be opened after Solaris 11 OS
public GA release), no other implementations of ZFS match its v31 features.

In part (another part being licensing) this is why the open community at large
shunned the S11X release - its ZFS version not yet interoperable (unlike the
options with OpenIndiana on different kernels and FreeBSD all supporting zpool
v28, not so sure Linux-FUSE - but maybe it is close too), and you can''t
look at the source code to see what might go wrong, or build a debug version
which would not kernel-panic.

Speaking of which, you might try to enforce dumping the kernel memory to a
dedicated volume/slice/partition so as to stacktrace it with the kernel
debugger. Maybe that would yield something...

//Jim
-- 
This message posted from opensolaris.org

Thank you very much Jim for your suggestions.


Trying any kind of import (including importing Read-Only) on SX11 will lead,
everytime, a system panic with following error:

panic[cpu12]/thread=ffffff02de5e20c0: assertion failed: zap_count(os, object,
&count) == 0, file: ../../common/fs/zfs/ddt_zap.c, line: 142

BTW, I saw ddt_zap.c is part of ZFS dedup implementation.



Trying with lastest OpenIndiana, I got:

cannot import ''rpool'': pool is formatted using a newer ZFS
version



You are right, there are no ZFS implementation newer than version 28 (outside
v31 on SX11).


So, now I got 2 possibilities:
- a complete scratch
- waiting for SX12!!!
(Probably I''m going to scratch .....)


Anyway I learnt some lessons from this experience:
- trust more in ZFS than in HW RAID
- OI is better than SX11


"Long life to ZFS!!!", not very impressed by SX11 ........


Stefano
-- 
This message posted from opensolaris.org