Is there a way to fsck the spacemap? Does scrub helps for this? -- This message posted from opensolaris.org
This is new for me:
$ zpool status
pool: rpool
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using ''zpool clear'' or replace the device with
''zpool replace''.
see: http://www.sun.com/msg/ZFS-8000-9P
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror-0 ONLINE 0 0 2
c7t0d0s0 ONLINE 0 0 2
c7t1d0s0 ONLINE 0 0 2
errors: No known data errors
I never saw that before.
It means the CRC error is on the two disks.
So did ZFS deleted the bad blocks or what did it?
--
This message posted from opensolaris.org
On 09/19/10 08:11 AM, Stephan Ferraro wrote:> This is new for me: > > $ zpool status > pool: rpool > state: ONLINE > status: One or more devices has experienced an unrecoverable error. An > attempt was made to correct the error. Applications are unaffected. > action: Determine if the device needs to be replaced, and clear the errors > using ''zpool clear'' or replace the device with ''zpool replace''. > see: http://www.sun.com/msg/ZFS-8000-9P > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > rpool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 2 > c7t0d0s0 ONLINE 0 0 2 > c7t1d0s0 ONLINE 0 0 2 > > errors: No known data errors > > I never saw that before. > It means the CRC error is on the two disks. > So did ZFS deleted the bad blocks or what did it? >ZFS handled the errors, some common to both drives in you system caused it. See the thread from an hour before this, "ZFS checksum errors (ZFS-8000-8A)" -- Ian.
On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote:> Is there a way to fsck the spacemap? > Does scrub helps for this?No, because issues that you see are internal inconsistencies with unclear nature. Though as actual issue varies from one inctance to another this is likely some random corruption. regards victor
On Sep 19, 2010, at 12:11 AM, Stephan Ferraro wrote:> This is new for me: > > $ zpool status > pool: rpool > state: ONLINE > status: One or more devices has experienced an unrecoverable error. An > attempt was made to correct the error. Applications are unaffected. > action: Determine if the device needs to be replaced, and clear the errors > using ''zpool clear'' or replace the device with ''zpool replace''. > see: http://www.sun.com/msg/ZFS-8000-9P > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > rpool ONLINE 0 0 0 > mirror-0 ONLINE 0 0 2 > c7t0d0s0 ONLINE 0 0 2 > c7t1d0s0 ONLINE 0 0 2 > > errors: No known data errors > > I never saw that before. > It means the CRC error is on the two disks. > So did ZFS deleted the bad blocks or what did it?This strongly suggests that you need to look for issue in some common component. It may be hardware like memory, caches, CPU, system busses or storage controllers, as you disks are apparently on the same controller. It may also be some software that corrupts memory contents. To troubleshoot HW issues you may consider running memtest86 or Oracle VTS for example. Setting kmem_flags to e.g. 0xf would be helpful to catch software issue.. regards victor
Am 19.09.2010 um 18:59 schrieb Victor Latushkin:> On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: > >> Is there a way to fsck the spacemap? >> Does scrub helps for this? > > No, because issues that you see are internal inconsistencies with unclear nature. > > Though as actual issue varies from one inctance to another this is likely some random corruption.The error repeated in endless loop because the server rebooted in endless loop and each time when it booted up ZFS it showed this ASSERTION error of the corrupt spacemap every time in the same memory location. -- Best regards, Stephan FERRARO Director & Member - Ferraro Ltd. - IT Linux & UNIX solutions Rechtliche Informationen: ------------------------- Name der Gesellschaft: FERRARO Ltd. Gesellschafter und Gesch?ftsf?hrer: Stephan Ferraro Registergericht: Amtsgericht Stuttgart Registernummer: HRB 724764 Umsatzsteuer-Indentifikationsnummer: DE814799566
On Sep 19, 2010, at 9:06 PM, Stephan Ferraro wrote:> Am 19.09.2010 um 18:59 schrieb Victor Latushkin: > >> On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: >> >>> Is there a way to fsck the spacemap? >>> Does scrub helps for this? >> >> No, because issues that you see are internal inconsistencies with unclear nature. >> >> Though as actual issue varies from one inctance to another this is likely some random corruption. > > > The error repeated in endless loop because the server rebooted in endless loop and each time when it booted up ZFS it showed this ASSERTION error of the corrupt spacemap every time in the same memory location.Once it is there you''ll trip various assertions each time affected space map is loaded. By random corruption I mean that it is different each time you encounter it another time after fixing it. victor
Am 19.09.2010 um 19:24 schrieb Victor Latushkin:> > On Sep 19, 2010, at 9:06 PM, Stephan Ferraro wrote: > >> Am 19.09.2010 um 18:59 schrieb Victor Latushkin: >> >>> On Sep 19, 2010, at 12:08 AM, Stephan Ferraro wrote: >>> >>>> Is there a way to fsck the spacemap? >>>> Does scrub helps for this? >>> >>> No, because issues that you see are internal inconsistencies with unclear nature. >>> >>> Though as actual issue varies from one inctance to another this is likely some random corruption. >> >> >> The error repeated in endless loop because the server rebooted in endless loop and each time when it booted up ZFS it showed this ASSERTION error of the corrupt spacemap every time in the same memory location. > > Once it is there you''ll trip various assertions each time affected space map is loaded. By random corruption I mean that it is different each time you encounter it another time after fixing it.I admit that hardware is probably the most cheapest you can get. I rent a server at Hetzner hosting service which provide a rescue system of OpenSolaris snv_111b. -- Best regards, Stephan FERRARO Director & Member - Ferraro Ltd. - IT Linux & UNIX solutions Rechtliche Informationen: ------------------------- Name der Gesellschaft: FERRARO Ltd. Gesellschafter und Gesch?ftsf?hrer: Stephan Ferraro Registergericht: Amtsgericht Stuttgart Registernummer: HRB 724764 Umsatzsteuer-Indentifikationsnummer: DE814799566