Hi all,
It would appear that I have a zpool corruption issue to deal with...
pool is exported, but upon trying to import it, server panics. Are
there any tools available on a zpool that is in an exported state?
I''ve got a separate test bed in which I''m trying to recreate,
but I
keep getting messages to the effect of need to import the pool first.
Suggestions?
thanks
Jay
Hardware:
Two
Sun-Fire T2000’s running
Sol10 8/07 s10s_u4wos_12b (SPARC) under control of Veritas Cluster.
Sun
StorageTek 6140 storage array.
Highlevel
Configuration:
6140
is setup in RAID5.
Several volumes were created on it, each mapping to a different zpool.
ZFS
Configuration:
Single
storage volume for each
zpool, no mirrors, no raidz groups.
Events
leading up to failure:
Both
nodes of the cluster
experienced reboots as a result of recovery efforts due to a power
outage at
the site. After the cluster was online and functioning, in order to
balance the load between the two T2000’s, the “home” zpool
was exported from one T2000 and then imported to the other T2000. Upon
trying to import on the second T2000, there was a bad checksum error
and then
the system did a dump and rebooted. Any subsequent attempts to import
the
“home” zpool have ended in the same.
A zfs
support person from Sun had us
try adding a few entries to /etc/system to see if we could then import,
but we
still had the same results.
Just
to be 100% clear, there was no
hardware failure in any of the storage subsystems. The 6140 is in
perfect
health. This is in my belief a ZFS issue, possibly caused by a failure
in
the Veritas Cluster in which two nodes tried to write to the same zpool
simultaneously. If not that, perhaps some random driver error or
bug… I don’t know. All theories aside, here is the output of
the error we see every time the `zpool import home` command fails:
panic[cpu8]/thread=2a100b69cc0:
ZFS: bad checksum (read on off 0: zio 60015f26a40 [L0
packed
nvlist] 4000L/400P DVA[0]=<0:4000b7000:400>
DVA[1]=<0:4600063800:400> fletcher4 lzjb BE contiguous
birth=3865837
fill=1 cksum=6e858ee992:2cad0296498c:cae1d40ad3d38:2e1daaef734ef08):
error 50
000002a100b69880
zfs:zio_done+29c (0, 10, 60015f26a98, 60012e062c0, 60015f26a40, 0)
%l0-3: 0000060015d04040 00000600160ffac0 00000000707de1c0
0000000000000032
%l4-7: 0000000000000001 0000000000000002 0000000000000032
0000060015c05118
000002a100b69940
zfs:zio_vdev_io_assess+178 (60015f26a40, 8000, 10, 0, 0, 10)
%l0-3: 0000000000010000 0000060015c05120 0000000000000000
0000000000000032
%l4-7: 0000060015c02888 0000000000000002 0000000000000000
0000060015c05118
000002a100b69a00
genunix:taskq_thread+1a4 (60015c05150, 60015c050f8, 50001, 5751b2acf88,
2a100b69aca, 2a100b69ac8)
%l0-3:
0000000000010000 0000060015c05120 0000060015c05128 0000060015c0512a
%l4-7: 0000060015c02888 0000000000000002 0000000000000000
0000060015c05118
syncing
file systems... 6 done
dumping
to
/dev/dsk/c0t0d0s1, offset 65536, content: kernel
100%
done:
100532 pages dumped, compression ratio 6.53, dump succeeded
rebooting...
SC
Alert:
Host System has Reset
SC
Alert:
Host system has shut down.
What
we’re looking for at this
point is any method to recover data from this zpool. Even a partial
one
would be better than nothing.
--
Thanks,
Jay
______
/_____/\ Jay Hamilton
/____ \\ \ Systems Support Engineer
/_____\ \\ / Sun Microsystems, Inc.
/_____/ \/ / / Sun Service
/_____/ / \//\ 500 El Dorado Blvd ubrm05-420
\_____\//\ / / Broomfield, CO. 80021
\_____/ / /\ / Accessline: (720) 895-3026 (x43026)
\_____/ \\ \ Email: jay.hamilton@sun.com
\_____\ \\ Pager: jay.hamilton-pager@sun.com
\_____\/
________________________________________________________________________
Submit, View and Update Service Tickets at
http://www.sun.com/service/online
This email may contain confidential and privileged material for the sole
use of the intended recipient. Any review or distribution by others is
strictly prohibited. If you are not the intended recipient please
contact the sender and delete all copies.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss