Hi all I''m seeing some rather bad resilver times for a pool of WD Green drives (I know, bad drives, but leave that). Does resilver go through the whole pool or just the VDEV in question? -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
----- Original Message -----> On 10/17/10 04:54 AM, Roy Sigurd Karlsbakk wrote: > > Hi all > > > > I''m seeing some rather bad resilver times for a pool of WD Green > > drives (I know, bad drives, but leave that). Does resilver go > > through the whole pool or just the VDEV in question? > > > > > The vdev only. All the data required to reconstruct a device in a vdev > is stored on the other devices.That''s what I thought, but then root at urd:~# zpool status pool: dpool state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scrub: resilver in progress for 8h46m, 2.31% done, 370h47m to go config: NAME STATE READ WRITE CKSUM dpool ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 c7t2d0 ONLINE 0 0 0 c7t3d0 ONLINE 0 0 0 c7t4d0 ONLINE 0 0 0 c7t5d0 ONLINE 0 0 0 c7t6d0 ONLINE 0 0 0 c7t7d0 ONLINE 0 0 0 c8t0d0 ONLINE 0 0 0 raidz2-1 ONLINE 0 0 0 c8t1d0 ONLINE 0 0 0 c8t2d0 ONLINE 0 0 0 c8t3d0 ONLINE 0 0 0 c8t4d0 ONLINE 0 0 0 c8t5d0 ONLINE 0 0 0 c8t6d0 ONLINE 0 0 0 c8t7d0 ONLINE 0 0 0 raidz2-2 ONLINE 0 0 0 c9t0d0 ONLINE 0 0 0 c9t1d0 ONLINE 0 0 0 c9t2d0 ONLINE 0 0 0 c9t3d0 ONLINE 0 0 0 spare-4 ONLINE 0 0 0 c9t4d0 ONLINE 0 0 0 c9t7d0 ONLINE 0 0 0 43.5G resilvered c9t5d0 ONLINE 0 0 0 c9t6d0 ONLINE 0 0 0 raidz2-4 ONLINE 0 0 0 c14t9d0 ONLINE 0 0 0 c14t10d0 ONLINE 0 0 0 c14t11d0 ONLINE 0 0 0 c14t12d0 ONLINE 0 0 0 c14t13d0 ONLINE 0 0 0 c14t14d0 ONLINE 0 0 0 c14t15d0 ONLINE 0 0 0 c14t16d0 ONLINE 0 0 0 c14t17d0 ONLINE 0 0 0 c14t18d0 ONLINE 0 0 0 c14t19d0 ONLINE 0 0 0 c14t20d0 ONLINE 0 0 0 logs mirror-3 ONLINE 0 0 0 c10d1s0 ONLINE 0 0 0 c11d0s0 ONLINE 0 0 0 cache c10d1s1 ONLINE 0 0 0 c11d0s1 ONLINE 0 0 0 spares c9t7d0 INUSE currently in use errors: No known data errors -- Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Roy Sigurd Karlsbakk > > > The vdev only.Right on. Furthermore, as shown in the "zpool status," a 7-disk raidz2 is certainly a reasonable vdev configuration.> scrub: resilver in progress for 8h46m, 2.31% done, 370h47m to goOuch. I''ll just say this much: During the resilver, be sure to disable autosnapshots and scrubs and "zfs sends." Do everything you can to reduce workload on the system. Would it help to delete old snapshots? I''m not sure, but I think it probably would. The time to resilver is determined by how many slabs (stripes, blocks, not sure if there''s a good or correct terminology here) ... how many slabs exist inside that vdev. All 6 good disks will seek & read their piece of the slab, parity is calculated and written to the resilvering disk. Repeat for all slabs in the vdev. I think if you destroy snaps, it will reduce the number of slabs that need to be processed. In the future, consider using either (a) mirrors instead of raidzN, or (b) disks with higher spindle speeds and lower seek times. If your HBA supports WriteBack, you might improve resilver speed, by enabling WB on the disk which is resilvering. But you should consider that temporary, and go back to WriteThrough after the resilver is completed.
On Oct 16, 2010, at 8:54 AM, Roy Sigurd Karlsbakk wrote:> Hi all > > I''m seeing some rather bad resilver times for a pool of WD Green drives (I know, bad drives, but leave that). Does resilver go through the whole pool or just the VDEV in question?Resilvers are done in time order. The metadata is traversed starting with the first txg and moving forward to the current txg. The good news is that only data is resilvered. The bad news for HDD fans is that HDDs do not like random workloads. -- richard -- OpenStorage Summit, October 25-27, Palo Alto, CA http://nexenta-summit2010.eventbrite.com USENIX LISA ''10 Conference November 8-16 ZFS and performance consulting http://www.RichardElling.com
On 10/17/10 12:37 PM, Roy Sigurd Karlsbakk wrote:> ----- Original Message ----- > >> On 10/17/10 04:54 AM, Roy Sigurd Karlsbakk wrote: >> >>> Hi all >>> >>> I''m seeing some rather bad resilver times for a pool of WD Green >>> drives (I know, bad drives, but leave that). Does resilver go >>> through the whole pool or just the VDEV in question? >>> >>> >>> >> The vdev only. All the data required to reconstruct a device in a vdev >> is stored on the other devices. >> > That''s what I thought, but then > > root at urd:~# zpool status > pool: dpool > state: ONLINE > status: One or more devices is currently being resilvered. The pool will > continue to function, possibly in a degraded state. > action: Wait for the resilver to complete. > scrub: resilver in progress for 8h46m, 2.31% done, 370h47m to go >I''m not sure what that''s supposed to prove. Run zpool iostat -v to see where the activity is. -- Ian.