zfs discuss - Oct 2012 - Directory is not accessible

Hi

I have raidz pool with one directory, which is not accessible. It only gives
"Input/output error" when trying to access it. Is there any way to fix
that?

nas4free:/tankki/media# zpool get version tankki
NAME    PROPERTY  VALUE    SOURCE
tankki  version   15       local

nas4free:/tankki/media# zpool status -v
  pool: tankki
 state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
        entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: scrub in progress since Sun Oct  7 21:18:19 2012
        494G scanned out of 5.92T at 261M/s, 6h4m to go
        0 repaired, 8.15% done
config:

        NAME        STATE     READ WRITE CKSUM
        tankki      ONLINE       0     0 3.62K
          raidz1-0  ONLINE       0     0 14.5K
            ada5p2  ONLINE       0     0     0
            ada2p2  ONLINE       0     0     0
            ada4p2  ONLINE       0     0     0
            ada3p2  ONLINE       0     0     0
            ada0p2  ONLINE       0     0     0
            ada1p2  ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        tankki/media:<0x0>

nas4free:/tankki/media# ls
.windows                        Talo
001                             Thumbs.db
ChromeStandaloneSetup.exe       Video
Compaq                          Web Sites
Dokumentit                      clonezilla
Kuvat                           home
Lontoo                          password-export-2012-06-28.xml
Software

nas4free:/tankki/media# cd Dokumentit
Dokumentit: Input/output error.
nas4free:/tankki/media#

On Sun, Oct 7, 2012 at 12:59 PM, Sami Tuominen <sami.tuominen at tut.fi>
wrote:
> Hi
>
> I have raidz pool with one directory, which is not accessible. It only
gives
> "Input/output error" when trying to access it. Is there any way
to fix that?
[...]
> nas4free:/tankki/media# zpool status -v
>   pool: tankki
>  state: ONLINE
> status: One or more devices has experienced an error resulting in data
>         corruption.  Applications may be affected.
> action: Restore the file in question if possible.  Otherwise restore the
>         entire pool from backup.
>    see: http://illumos.org/msg/ZFS-8000-8A
>   scan: scrub in progress since Sun Oct  7 21:18:19 2012
>         494G scanned out of 5.92T at 261M/s, 6h4m to go
>         0 repaired, 8.15% done
> config:
>
>         NAME        STATE     READ WRITE CKSUM
>         tankki      ONLINE       0     0 3.62K
>           raidz1-0  ONLINE       0     0 14.5K
>             ada5p2  ONLINE       0     0     0
>             ada2p2  ONLINE       0     0     0
>             ada4p2  ONLINE       0     0     0
>             ada3p2  ONLINE       0     0     0
>             ada0p2  ONLINE       0     0     0
>             ada1p2  ONLINE       0     0     0
>
> errors: Permanent errors have been detected in the following files:
>
>         tankki/media:<0x0>
It''s as it says it is: the error is "permanent" in that ZFS
has done
what it could to recover the data from parity information and ditto
blocks. Sometimes the error is only in the current version of a
file/directory, so you can recover the data from a snapshot.

> nas4free:/tankki/media# cd Dokumentit
> Dokumentit: Input/output error.
> nas4free:/tankki/media#
Do you have a snapshot that you can navigate to and determine if the
directory appears intact?


Jan

>>
>>         NAME        STATE     READ WRITE CKSUM
>>         tankki      ONLINE       0     0 3.62K
>>           raidz1-0  ONLINE       0     0 14.5K
>>             ada5p2  ONLINE       0     0     0
>>             ada2p2  ONLINE       0     0     0
>>             ada4p2  ONLINE       0     0     0
>>             ada3p2  ONLINE       0     0     0
>>             ada0p2  ONLINE       0     0     0
>>             ada1p2  ONLINE       0     0     0
>>
>> errors: Permanent errors have been detected in the following files:
>>
>>         tankki/media:<0x0>
> 
> It''s as it says it is: the error is "permanent" in that
ZFS has done what it could to recover the data
> from parity information and ditto blocks. Sometimes the error is only in
the current version of a
>  file/directory, so you can recover the data from a snapshot.
Unfortunately there aren''t any snapshots.
The version of zpool is 15. Is it safe to upgrade that?
Is zpool clear -F supported or of any use here?

Sami
> Jan

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Sami Tuominen
> 
> Unfortunately there aren''t any snapshots.
> The version of zpool is 15. Is it safe to upgrade that?
> Is zpool clear -F supported or of any use here?
The only thing that will be of use to restore your data will be a backup.

To forget about the lost data and make the error message go away, simply rm the
bad directory (and/or its parent).

You''re probably wondering, you have redundancy and no faulted devices,
so how could this happen?  There are a few possible explanations, but
they''re all going to have one thing in common:  At some point,
something got corrupted before it was written corrupted and the redundant copy
also written corrupted.  It might be you had a CPU error, or some parity error
in non-ECC ram, or a bus glitch or bad firmware in the HBA, for example.  The
fact remains, something was written corrupted, and the redundant copy was also
written corrupted.  All you can do is restore from a snapshot, restore from a
backup, or accept it for what it is and make the error go away.

Sorry to hear it...