zfs discuss - Aug 2008 - Checksum error: which of my files have failed scrubbing?

ZFS has detected that my root filesystem has a small number of errors.  Is there
a way to tell which specific files have been corrupted?


sbox:~$ zpool status -x
  pool: rpool
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using ''zpool clear'' or replace the device with
''zpool replace''.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: scrub completed after 0h10m with 2 errors on Sun Aug  3 00:16:33 2008
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     4
          c4t0d0s0  ONLINE       0     0     4
 
 
This message posted from opensolaris.org

soren wrote:
> ZFS has detected that my root filesystem has a small number of errors.  Is
there a way to tell which specific files have been corrupted?
>   
After a scrub a zpool status -v should give you a list of files with 
unrecoverable errors.


Bob

> soren wrote:
> > ZFS has detected that my root filesystem has a
> small number of errors.  Is there a way to tell which
> specific files have been corrupted?
>
> After a scrub a zpool status -v should give you a
> list of files with 
> unrecoverable errors.
Hmm, I just tried that.  Perhaps "No known data errors" means that my
files are OK.  In that case I wonder what the checksum failure was from.


sbox:~$ zpool status -xv
  pool: rpool
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using ''zpool clear'' or replace the device with
''zpool replace''.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: scrub completed after 0h10m with 2 errors on Sun Aug  3 00:16:33 2008
config:

        NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     4
          c4t0d0s0  ONLINE       0     0     4

errors: No known data errors
 
 
This message posted from opensolaris.org

Possibly metadata. Since that''s however redundant due to ditto blocks  
(2 or 3 copies depending on importance), it was repaired during the  
scrub.

--
Via iPhone 3G

On 05-ao?t-08, at 21:11, soren <opensolaris at spam.monkeydyne.com> wrote:
>> soren wrote:
>>> ZFS has detected that my root filesystem has a
>> small number of errors.  Is there a way to tell which
>> specific files have been corrupted?
>>
>> After a scrub a zpool status -v should give you a
>> list of files with
>> unrecoverable errors.
>
> Hmm, I just tried that.  Perhaps "No known data errors" means
that
> my files are OK.  In that case I wonder what the checksum failure  
> was from.
>
>
> sbox:~$ zpool status -xv
>  pool: rpool
> state: ONLINE
> status: One or more devices has experienced an unrecoverable error.   
> An
>        attempt was made to correct the error.  Applications are  
> unaffected.
> action: Determine if the device needs to be replaced, and clear the  
> errors
>        using ''zpool clear'' or replace the device with
''zpool replace''.
>   see: http://www.sun.com/msg/ZFS-8000-9P
> scrub: scrub completed after 0h10m with 2 errors on Sun Aug  3  
> 00:16:33 2008
> config:
>
>        NAME        STATE     READ WRITE CKSUM
>        rpool       ONLINE       0     0     4
>          c4t0d0s0  ONLINE       0     0     4
>
> errors: No known data errors
>
>
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Soren,

At this point, I''d like to know what fmdump -eV says about your disk so
you can determine whether it should be replaced or not.

Cindy

soren wrote:
>>soren wrote:
>>
>>>ZFS has detected that my root filesystem has a
>>
>>small number of errors.  Is there a way to tell which
>>specific files have been corrupted?
>>
>>After a scrub a zpool status -v should give you a
>>list of files with 
>>unrecoverable errors.
> 
> 
> Hmm, I just tried that.  Perhaps "No known data errors" means
that my files are OK.  In that case I wonder what the checksum failure was from.
> 
> 
> sbox:~$ zpool status -xv
>   pool: rpool
>  state: ONLINE
> status: One or more devices has experienced an unrecoverable error.  An
>         attempt was made to correct the error.  Applications are
unaffected.
> action: Determine if the device needs to be replaced, and clear the errors
>         using ''zpool clear'' or replace the device with
''zpool replace''.
>    see: http://www.sun.com/msg/ZFS-8000-9P
>  scrub: scrub completed after 0h10m with 2 errors on Sun Aug  3 00:16:33
2008
> config:
> 
>         NAME        STATE     READ WRITE CKSUM
>         rpool       ONLINE       0     0     4
>           c4t0d0s0  ONLINE       0     0     4
> 
> errors: No known data errors
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

> At this point, I''d like to know what fmdump -eV says about your
disk so
> you can determine whether it should be replaced or not.
TIME                           CLASS
Aug 03 2008 00:09:27.559200662 ereport.fs.zfs.checksum
nvlist version: 0
	class = ereport.fs.zfs.checksum
	ena = 0xaaf142e8b2d00001
	detector = (embedded nvlist)
	nvlist version: 0
		version = 0x0
		scheme = zfs
		pool = 0xb8e9e54aadb6b8e
		vdev = 0xe6f603b3ce48c037
	(end detector)

	pool = rpool
	pool_guid = 0xb8e9e54aadb6b8e
	pool_context = 0
	pool_failmode = wait
	vdev_guid = 0xe6f603b3ce48c037
	vdev_type = disk
	vdev_path = /dev/dsk/c4t0d0s0
	vdev_devid = id1,sd at f0000000048852163000040380000/a
	parent_guid = 0xb8e9e54aadb6b8e
	parent_type = root
	zio_err = 50
	zio_offset = 0x7b88a000
	zio_size = 0xb000
	zio_objset = 0x0
	zio_object = 0x0
	zio_level = 0
	zio_blkid = 0x3e
	__ttl = 0x1
	__tod = 0x489559a7 0x2154b996

Aug 03 2008 00:09:27.559200258 ereport.fs.zfs.checksum
nvlist version: 0
	class = ereport.fs.zfs.checksum
	ena = 0xaaf142e8b2d00001
	detector = (embedded nvlist)
	nvlist version: 0
		version = 0x0
		scheme = zfs
		pool = 0xb8e9e54aadb6b8e
		vdev = 0xe6f603b3ce48c037
	(end detector)

	pool = rpool
	pool_guid = 0xb8e9e54aadb6b8e
	pool_context = 0
	pool_failmode = wait
	vdev_guid = 0xe6f603b3ce48c037
	vdev_type = disk
	vdev_path = /dev/dsk/c4t0d0s0
	vdev_devid = id1,sd at f0000000048852163000040380000/a
	parent_guid = 0xb8e9e54aadb6b8e
	parent_type = root
	zio_err = 50
	zio_offset = 0x7b88a000
	zio_size = 0xb000
	zio_objset = 0x0
	zio_object = 0x0
	zio_level = 0
	zio_blkid = 0x3e
	__ttl = 0x1
	__tod = 0x489559a7 0x2154b802

Aug 03 2008 00:09:27.559200135 ereport.fs.zfs.data
nvlist version: 0
	class = ereport.fs.zfs.data
	ena = 0xaaf142e8b2d00001
	detector = (embedded nvlist)
	nvlist version: 0
		version = 0x0
		scheme = zfs
		pool = 0xb8e9e54aadb6b8e
	(end detector)

	pool = rpool
	pool_guid = 0xb8e9e54aadb6b8e
	pool_context = 0
	pool_failmode = wait
	zio_err = 50
	zio_objset = 0x0
	zio_object = 0x0
	zio_level = 0
	zio_blkid = 0x3e
	__ttl = 0x1
	__tod = 0x489559a7 0x2154b787

Aug 03 2008 00:13:11.874835614 ereport.fs.zfs.checksum
nvlist version: 0
	class = ereport.fs.zfs.checksum
	ena = 0xae34e6f82e200401
	detector = (embedded nvlist)
	nvlist version: 0
		version = 0x0
		scheme = zfs
		pool = 0xb8e9e54aadb6b8e
		vdev = 0xe6f603b3ce48c037
	(end detector)

	pool = rpool
	pool_guid = 0xb8e9e54aadb6b8e
	pool_context = 0
	pool_failmode = wait
	vdev_guid = 0xe6f603b3ce48c037
	vdev_type = disk
	vdev_path = /dev/dsk/c4t0d0s0
	vdev_devid = id1,sd at f0000000048852163000040380000/a
	parent_guid = 0xb8e9e54aadb6b8e
	parent_type = root
	zio_err = 50
	zio_offset = 0x125502000
	zio_size = 0x2000
	zio_objset = 0x0
	zio_object = 0x0
	zio_level = 0
	zio_blkid = 0x8
	__ttl = 0x1
	__tod = 0x48955a87 0x3424ee9e

Aug 03 2008 00:13:11.874836090 ereport.fs.zfs.checksum
nvlist version: 0
	class = ereport.fs.zfs.checksum
	ena = 0xae34e6f82e200401
	detector = (embedded nvlist)
	nvlist version: 0
		version = 0x0
		scheme = zfs
		pool = 0xb8e9e54aadb6b8e
		vdev = 0xe6f603b3ce48c037
	(end detector)

	pool = rpool
	pool_guid = 0xb8e9e54aadb6b8e
	pool_context = 0
	pool_failmode = wait
	vdev_guid = 0xe6f603b3ce48c037
	vdev_type = disk
	vdev_path = /dev/dsk/c4t0d0s0
	vdev_devid = id1,sd at f0000000048852163000040380000/a
	parent_guid = 0xb8e9e54aadb6b8e
	parent_type = root
	zio_err = 50
	zio_offset = 0x125502000
	zio_size = 0x2000
	zio_objset = 0x0
	zio_object = 0x0
	zio_level = 0
	zio_blkid = 0x8
	__ttl = 0x1
	__tod = 0x48955a87 0x3424f07a

Aug 03 2008 00:13:11.874835624 ereport.fs.zfs.data
nvlist version: 0
	class = ereport.fs.zfs.data
	ena = 0xae34e6f82e200401
	detector = (embedded nvlist)
	nvlist version: 0
		version = 0x0
		scheme = zfs
		pool = 0xb8e9e54aadb6b8e
	(end detector)

	pool = rpool
	pool_guid = 0xb8e9e54aadb6b8e
	pool_context = 0
	pool_failmode = wait
	zio_err = 50
	zio_objset = 0x0
	zio_object = 0x0
	zio_level = 0
	zio_blkid = 0x8
	__ttl = 0x1
	__tod = 0x48955a87 0x3424eea8
 
 
This message posted from opensolaris.org

On Tue, 2008-08-05 at 12:11 -0700, soren wrote:
> > soren wrote:
> > > ZFS has detected that my root filesystem has a
> > small number of errors.  Is there a way to tell which
> > specific files have been corrupted?
> >
> > After a scrub a zpool status -v should give you a
> > list of files with 
> > unrecoverable errors.
> 
> Hmm, I just tried that.  Perhaps "No known data errors" means
that my files are OK.  In that case I wonder what the checksum failure was from.
If this is build 94 and you have one or more unmounted filesystems, 
(such as alternate boot environments), these errors are false positives.
There is no actual error; the scrubber misinterpreted the end of an
intent log block chain as a checksum error.

the bug id is:

6727872 zpool scrub: reports checksum errors for pool with zfs and
unplayed ZIL

This bug is fixed in build 95.  One workaround is to mount the
filesystems and then unmount them to apply the intent log changes.

					- Bill

Aha, that''s the problem.  I just upgraded to build 94, and I have
alternate boot environments.

======================================================
> Hmm, I just tried that.  Perhaps "No known data errors" means
that my files are OK.  In > that case I wonder what the checksum failure was
from.
If this is build 94 and you have one or more unmounted filesystems, 
(such as alternate boot environments), these errors are false positives.
There is no actual error; the scrubber misinterpreted the end of an
intent log block chain as a checksum error.

the bug id is:

6727872 zpool scrub: reports checksum errors for pool with zfs and
unplayed ZIL

This bug is fixed in build 95.  One workaround is to mount the
filesystems and then unmount them to apply the intent log changes.

					- Bill
 
 
This message posted from opensolaris.org