zfs discuss - Mar 2009 - Recovering data from a corrupted zpool

Hi there,

Is there a way to get as much data as possible off an existing slightly
corrupted zpool?  I have a 2 disk stripe which I''m moving to new
storage.  I
will be moving it to a ZFS Mirror, however at the moment I''m having
problems
with ZFS Panic''ing the system during a send | recv.

I don''t know exactly how much data is valid.  Everything appears to run
as
expected and applications aren''t crashing.

Doing an $( ls -lR | grep -i "IO Error" ) returns roughly 10-15 files
which
are affected.    Luckily, these files ls is returning aren''t super
critical.

Is it possible to tell ZFS to do a emergency "copy as much valid data off
this file system"?

I''ve tried disabling checkums on the corrupted source zpool.   But even
still, once ZFS runs into an error the zpool is FAULTED and the kernel
panic''s and the system crashes.   Is it possible to tell the zpool to
ignore
any errors and continue without faulting the zpool?

We have a backup of the data, which is 2 months old.  Is it slightly
possible to bring this backup online, and ''sync as much as it
can'' between
the two volumes?  Could this just be a rsync job?

Thanks



[root]# zpool status -v apps
  pool: apps
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://www.sun.com/msg/ZFS-8000-8A
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        apps        ONLINE       0     0     120
          c1t1d0    ONLINE       0     0     60
          c1t2d0    ONLINE       0     0     0
          c1t3d0    ONLINE       0     0     60

errors: Permanent errors have been detected in the following files:

        apps:<0x0>
        <0x1d2>:<0x0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090327/4f768b9a/attachment.html>

2009/3/27 Matthew Angelo <bangers at gmail.com>:
> Doing an $( ls -lR | grep -i "IO Error" ) returns roughly 10-15
files which
> are affected.
If ls works then tar, cpio, etc. should work.

Matthew Angelo wrote:
> Hi there,
>
> Is there a way to get as much data as possible off an existing 
> slightly corrupted zpool?  I have a 2 disk stripe which I''m moving
to
> new storage.  I will be moving it to a ZFS Mirror, however at the 
> moment I''m having problems with ZFS Panic''ing the system
during a send
> | recv.
set your failmode to continue.  If you don''t have a failmode parameter,
then you''ll want to try a later version of Solaris.

I blogged about some techniques to deal with borken files a while back...
http://blogs.sun.com/relling/entry/holy_smokes_a_holey_file
http://blogs.sun.com/relling/entry/dd_tricks_for_holey_files
http://blogs.sun.com/relling/entry/more_on_holey_files

NB: if it is the receive that is panicing, then please file a bug.
 -- richard
>
> I don''t know exactly how much data is valid.  Everything appears
to
> run as expected and applications aren''t crashing.
>
> Doing an $( ls -lR | grep -i "IO Error" ) returns roughly 10-15
files
> which are affected.    Luckily, these files ls is returning aren''t
> super critical.
>
> Is it possible to tell ZFS to do a emergency "copy as much valid data 
> off this file system"?  
>
> I''ve tried disabling checkums on the corrupted source zpool.   But
> even still, once ZFS runs into an error the zpool is FAULTED and the 
> kernel panic''s and the system crashes.   Is it possible to tell
the
> zpool to ignore any errors and continue without faulting the zpool?
>
> We have a backup of the data, which is 2 months old.  Is it slightly 
> possible to bring this backup online, and ''sync as much as it
can''
> between the two volumes?  Could this just be a rsync job?
>
> Thanks
>
>
>
> [root]# zpool status -v apps
>   pool: apps
>  state: ONLINE
> status: One or more devices has experienced an error resulting in data
>         corruption.  Applications may be affected.
> action: Restore the file in question if possible.  Otherwise restore the
>         entire pool from backup.
>    see: http://www.sun.com/msg/ZFS-8000-8A
>  scrub: none requested
> config:
>
>         NAME        STATE     READ WRITE CKSUM
>         apps        ONLINE       0     0     120
>           c1t1d0    ONLINE       0     0     60
>           c1t2d0    ONLINE       0     0     0
>           c1t3d0    ONLINE       0     0     60
>
> errors: Permanent errors have been detected in the following files:
>
>         apps:<0x0>
>         <0x1d2>:<0x0>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>