Brian Leonard
2010-Jun-29 15:16 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
Hi,
I have a zpool which is currently reporting that the
"<metadata>:<0x13>" file is corrupt:
bleonard at opensolaris:~$ pfexec zpool status -xv external
pool: external
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://www.sun.com/msg/ZFS-8000-8A
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
external ONLINE 0 0 0
c0t0d0p0 ONLINE 0 0 0
errors: Permanent errors have been detected in the following files:
<metadata>:<0x13>
Otherwise, as you can see, the pool is online. As it''s unclear to me
how to restore the "<metadata>:<0x13>" file, is my only
option for correcting this error to destroy and recreate the pool?
Thanks,
Brian
--
This message posted from opensolaris.org
Cindy Swearingen
2010-Jun-29 15:51 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
Hi Brian, You might try running a scrub on this pool. Is this an external USB device? Thanks, Cindy On 06/29/10 09:16, Brian Leonard wrote:> Hi, > > I have a zpool which is currently reporting that the "<metadata>:<0x13>" file is corrupt: > > bleonard at opensolaris:~$ pfexec zpool status -xv external > pool: external > state: ONLINE > status: One or more devices has experienced an error resulting in data > corruption. Applications may be affected. > action: Restore the file in question if possible. Otherwise restore the > entire pool from backup. > see: http://www.sun.com/msg/ZFS-8000-8A > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > external ONLINE 0 0 0 > c0t0d0p0 ONLINE 0 0 0 > > errors: Permanent errors have been detected in the following files: > > <metadata>:<0x13> > > Otherwise, as you can see, the pool is online. As it''s unclear to me how to restore the "<metadata>:<0x13>" file, is my only option for correcting this error to destroy and recreate the pool? > > Thanks, > Brian
W Brian Leonard
2010-Jun-29 17:15 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
Hi Cindy, The scrub didn''t help and yes, this is an external USB device. Thanks, Brian Cindy Swearingen wrote:> Hi Brian, > > You might try running a scrub on this pool. > > Is this an external USB device? > > Thanks, > > Cindy > > On 06/29/10 09:16, Brian Leonard wrote: >> Hi, >> >> I have a zpool which is currently reporting that the >> "<metadata>:<0x13>" file is corrupt: >> >> bleonard at opensolaris:~$ pfexec zpool status -xv external >> pool: external >> state: ONLINE >> status: One or more devices has experienced an error resulting in data >> corruption. Applications may be affected. >> action: Restore the file in question if possible. Otherwise restore the >> entire pool from backup. >> see: http://www.sun.com/msg/ZFS-8000-8A >> scrub: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> external ONLINE 0 0 0 >> c0t0d0p0 ONLINE 0 0 0 >> >> errors: Permanent errors have been detected in the following files: >> >> <metadata>:<0x13> >> >> Otherwise, as you can see, the pool is online. As it''s unclear to me >> how to restore the "<metadata>:<0x13>" file, is my only option for >> correcting this error to destroy and recreate the pool? >> >> Thanks, >> Brian-- W Brian Leonard Principal Product Manager 860.206.6093 http://blogs.sun.com/observatory
Cindy Swearingen
2010-Jun-29 19:31 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
Hi Brian, Because the pool is still online and the metadata is redundant, maybe these errors were caused by a brief hiccup from the USB device''s physical connection. You might try: # zpool clear external c0t0d0p0 Then, run a scrub: # zpool scrub external If the above fails, then please identify the Solaris release and what events preceded this problem. Thanks, Cindy On 06/29/10 11:15, W Brian Leonard wrote:> Hi Cindy, > > The scrub didn''t help and yes, this is an external USB device. > > Thanks, > Brian > > Cindy Swearingen wrote: >> Hi Brian, >> >> You might try running a scrub on this pool. >> >> Is this an external USB device? >> >> Thanks, >> >> Cindy >> >> On 06/29/10 09:16, Brian Leonard wrote: >>> Hi, >>> >>> I have a zpool which is currently reporting that the >>> "<metadata>:<0x13>" file is corrupt: >>> >>> bleonard at opensolaris:~$ pfexec zpool status -xv external >>> pool: external >>> state: ONLINE >>> status: One or more devices has experienced an error resulting in data >>> corruption. Applications may be affected. >>> action: Restore the file in question if possible. Otherwise restore the >>> entire pool from backup. >>> see: http://www.sun.com/msg/ZFS-8000-8A >>> scrub: none requested >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> external ONLINE 0 0 0 >>> c0t0d0p0 ONLINE 0 0 0 >>> >>> errors: Permanent errors have been detected in the following files: >>> >>> <metadata>:<0x13> >>> >>> Otherwise, as you can see, the pool is online. As it''s unclear to me >>> how to restore the "<metadata>:<0x13>" file, is my only option for >>> correcting this error to destroy and recreate the pool? >>> >>> Thanks, >>> Brian >
W Brian Leonard
2010-Jun-29 20:13 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
Interesting, this time it worked! Does specifying the device to clear cause the command to behave differently? I had assumed w/out the device specification, the clear would just apply to all devices in the pool (which are just the one). Thanks, Brian Cindy Swearingen wrote:> Hi Brian, > > Because the pool is still online and the metadata is redundant, maybe > these errors were caused by a brief hiccup from the USB device''s > physical connection. You might try: > > # zpool clear external c0t0d0p0 > > Then, run a scrub: > > # zpool scrub external > > If the above fails, then please identify the Solaris release and what > events preceded this problem. > > Thanks, > > Cindy > > > > > On 06/29/10 11:15, W Brian Leonard wrote: >> Hi Cindy, >> >> The scrub didn''t help and yes, this is an external USB device. >> >> Thanks, >> Brian >> >> Cindy Swearingen wrote: >>> Hi Brian, >>> >>> You might try running a scrub on this pool. >>> >>> Is this an external USB device? >>> >>> Thanks, >>> >>> Cindy >>> >>> On 06/29/10 09:16, Brian Leonard wrote: >>>> Hi, >>>> >>>> I have a zpool which is currently reporting that the >>>> "<metadata>:<0x13>" file is corrupt: >>>> >>>> bleonard at opensolaris:~$ pfexec zpool status -xv external >>>> pool: external >>>> state: ONLINE >>>> status: One or more devices has experienced an error resulting in data >>>> corruption. Applications may be affected. >>>> action: Restore the file in question if possible. Otherwise >>>> restore the >>>> entire pool from backup. >>>> see: http://www.sun.com/msg/ZFS-8000-8A >>>> scrub: none requested >>>> config: >>>> >>>> NAME STATE READ WRITE CKSUM >>>> external ONLINE 0 0 0 >>>> c0t0d0p0 ONLINE 0 0 0 >>>> >>>> errors: Permanent errors have been detected in the following files: >>>> >>>> <metadata>:<0x13> >>>> >>>> Otherwise, as you can see, the pool is online. As it''s unclear to >>>> me how to restore the "<metadata>:<0x13>" file, is my only option >>>> for correcting this error to destroy and recreate the pool? >>>> >>>> Thanks, >>>> Brian >>-- W Brian Leonard Principal Product Manager 860.206.6093 http://blogs.sun.com/observatory
Cindy Swearingen
2010-Jun-29 20:38 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
I reviewed the zpool clear syntax (looking at my own docs) and didn''t remember that a one-device pool probably doesn''t need the device specified. For pools with many devices, you might want to just clear the errors on a particular device. USB sticks for pools are problemmatic. It would be good to know what caused these errors to try to prevent them in the future. We know that USB devices don''t generate/fabricate device IDs so they are prone to problems when moving/changing/re-inserting but without more info, its hard to tell what happened. cs On 06/29/10 14:13, W Brian Leonard wrote:> Interesting, this time it worked! Does specifying the device to clear > cause the command to behave differently? I had assumed w/out the device > specification, the clear would just apply to all devices in the pool > (which are just the one). > > Thanks, > Brian > > Cindy Swearingen wrote: >> Hi Brian, >> >> Because the pool is still online and the metadata is redundant, maybe >> these errors were caused by a brief hiccup from the USB device''s >> physical connection. You might try: >> >> # zpool clear external c0t0d0p0 >> >> Then, run a scrub: >> >> # zpool scrub external >> >> If the above fails, then please identify the Solaris release and what >> events preceded this problem. >> >> Thanks, >> >> Cindy >> >> >> >> >> On 06/29/10 11:15, W Brian Leonard wrote: >>> Hi Cindy, >>> >>> The scrub didn''t help and yes, this is an external USB device. >>> >>> Thanks, >>> Brian >>> >>> Cindy Swearingen wrote: >>>> Hi Brian, >>>> >>>> You might try running a scrub on this pool. >>>> >>>> Is this an external USB device? >>>> >>>> Thanks, >>>> >>>> Cindy >>>> >>>> On 06/29/10 09:16, Brian Leonard wrote: >>>>> Hi, >>>>> >>>>> I have a zpool which is currently reporting that the >>>>> "<metadata>:<0x13>" file is corrupt: >>>>> >>>>> bleonard at opensolaris:~$ pfexec zpool status -xv external >>>>> pool: external >>>>> state: ONLINE >>>>> status: One or more devices has experienced an error resulting in data >>>>> corruption. Applications may be affected. >>>>> action: Restore the file in question if possible. Otherwise >>>>> restore the >>>>> entire pool from backup. >>>>> see: http://www.sun.com/msg/ZFS-8000-8A >>>>> scrub: none requested >>>>> config: >>>>> >>>>> NAME STATE READ WRITE CKSUM >>>>> external ONLINE 0 0 0 >>>>> c0t0d0p0 ONLINE 0 0 0 >>>>> >>>>> errors: Permanent errors have been detected in the following files: >>>>> >>>>> <metadata>:<0x13> >>>>> >>>>> Otherwise, as you can see, the pool is online. As it''s unclear to >>>>> me how to restore the "<metadata>:<0x13>" file, is my only option >>>>> for correcting this error to destroy and recreate the pool? >>>>> >>>>> Thanks, >>>>> Brian >>> >
W Brian Leonard
2010-Jun-29 21:10 UTC
[zfs-discuss] Permanet errors detected in <metadata>:<0x13>
Well, I was doing a ZFS send / receive to backup a large (60 GB) of data, which never completed. A zpool clear at that point just hung and I had to reboot the system, after which it appeared to come up clean. As soon as I tried the backup again I noticed the pool reported the error you see below - but the backup did complete as the pool remained online. Thanks for your help Cindy, Brian Cindy Swearingen wrote:> > > I reviewed the zpool clear syntax (looking at my own docs) and didn''t > remember that a one-device pool probably doesn''t need the device > specified. For pools with many devices, you might want to just clear > the errors on a particular device. > > USB sticks for pools are problemmatic. It would be good to know what > caused these errors to try to prevent them in the future. > > We know that USB devices don''t generate/fabricate device IDs so they > are prone to problems when moving/changing/re-inserting but without > more info, its hard to tell what happened. > > cs > > On 06/29/10 14:13, W Brian Leonard wrote: >> Interesting, this time it worked! Does specifying the device to clear >> cause the command to behave differently? I had assumed w/out the >> device specification, the clear would just apply to all devices in >> the pool (which are just the one). >> >> Thanks, >> Brian >> >> Cindy Swearingen wrote: >>> Hi Brian, >>> >>> Because the pool is still online and the metadata is redundant, maybe >>> these errors were caused by a brief hiccup from the USB device''s >>> physical connection. You might try: >>> >>> # zpool clear external c0t0d0p0 >>> >>> Then, run a scrub: >>> >>> # zpool scrub external >>> >>> If the above fails, then please identify the Solaris release and what >>> events preceded this problem. >>> >>> Thanks, >>> >>> Cindy >>> >>> >>> >>> >>> On 06/29/10 11:15, W Brian Leonard wrote: >>>> Hi Cindy, >>>> >>>> The scrub didn''t help and yes, this is an external USB device. >>>> >>>> Thanks, >>>> Brian >>>> >>>> Cindy Swearingen wrote: >>>>> Hi Brian, >>>>> >>>>> You might try running a scrub on this pool. >>>>> >>>>> Is this an external USB device? >>>>> >>>>> Thanks, >>>>> >>>>> Cindy >>>>> >>>>> On 06/29/10 09:16, Brian Leonard wrote: >>>>>> Hi, >>>>>> >>>>>> I have a zpool which is currently reporting that the >>>>>> "<metadata>:<0x13>" file is corrupt: >>>>>> >>>>>> bleonard at opensolaris:~$ pfexec zpool status -xv external >>>>>> pool: external >>>>>> state: ONLINE >>>>>> status: One or more devices has experienced an error resulting in >>>>>> data >>>>>> corruption. Applications may be affected. >>>>>> action: Restore the file in question if possible. Otherwise >>>>>> restore the >>>>>> entire pool from backup. >>>>>> see: http://www.sun.com/msg/ZFS-8000-8A >>>>>> scrub: none requested >>>>>> config: >>>>>> >>>>>> NAME STATE READ WRITE CKSUM >>>>>> external ONLINE 0 0 0 >>>>>> c0t0d0p0 ONLINE 0 0 0 >>>>>> >>>>>> errors: Permanent errors have been detected in the following files: >>>>>> >>>>>> <metadata>:<0x13> >>>>>> >>>>>> Otherwise, as you can see, the pool is online. As it''s unclear to >>>>>> me how to restore the "<metadata>:<0x13>" file, is my only option >>>>>> for correcting this error to destroy and recreate the pool? >>>>>> >>>>>> Thanks, >>>>>> Brian >>>> >>-- W Brian Leonard Principal Product Manager 860.206.6093 http://blogs.sun.com/observatory