zfs discuss - Nov 2008 - Resilvering is SLOW!. Not progressing at all

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi, everybody. I started a resilvering with a 150 GB ZFS pool 16 hours
ago. The resilvering is not progressing at all:

"""
                 capacity     operations    bandwidth
pool           used  avail   read  write   read  write
- ------------  -----  -----  -----  -----  -----  -----
datos          107G  50.5G     32     95   170K   569K
  mirror       107G  50.5G     79    235   418K  1.37M
    c5t1d0s4      -      -     22     66  1.22M  1.12M
    c5t0d0s0      -      -      1    160  63.3K  1.65M
- ------------  -----  -----  -----  -----  -----  -----

[root at tesalia /]# zpool status
  pool: datos
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
 scrub: resilver in progress for 0h0m, 0.03% done, 44h30m to go
config:

        NAME          STATE     READ WRITE CKSUM
        datos         ONLINE       0     0     0
          mirror      ONLINE       0     0     0
            c5t1d0s4  ONLINE       0     0     0
            c5t0d0s0  ONLINE       0     0     0

errors: No known data errors
"""

The pool is live and in production, but the IO traffic is fairly low. I
wrote a dtrace script to see synchronous IO traffic, and it is very
sparse (2-3 fdsyncs per second, at most).

What can I do?. Taking days to resilver 150 GB is going to be a problem.
And no, I am not taking snapshots while resilvering (another problem).

Solaris 10 Update 6 here, ZFS boot.

What can I do?.

- --
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQCVAwUBSRqoHZlgi5GaxT1NAQKWoAQAheRdEtswfcFhYJsRGmr8Z/1i9muZ6uOH
jBZAEC+DOfp2OK6g4P337ewpGg6nmzBqN/OpDRIdT/H2TXuBe+nReIjd5bFukRbt
d5NrO8m41CkSUJ11W9nFNldOBuK+lQcmGep2XyKL3NPUrwH65uyUTO4HVmOHG5x3
vf/n0OeuRkU=dbuw
-----END PGP SIGNATURE-----

> Hi, everybody. I started a resilvering with a 150 GB ZFS pool 16 hours
> ago. The resilvering is not progressing at all:
[...]
>
> [root at tesalia /]# zpool status
Try to run "zpool status" as a non-root user and see if the resilver
shows any progress.  Sometimes, "zpool status" as root seems to
cause the resilver to start again from the beginning.  

The one bug I could find in bugs.opensolaris.org was closed as
"not reproducible" so I am not really sure what the status of this
problem is.


Regards -- Volker
-- 
------------------------------------------------------------------------
Volker A. Brandt                  Consulting and Support for Sun Solaris
Brandt & Brandt Computer GmbH                   WWW: http://www.bb-c.de/
Am Wiesenpfad 6, 53340 Meckenheim                     Email: vab at bb-c.de
Handelsregister: Amtsgericht Bonn, HRB 10513              Schuhgr??e: 45
Gesch?ftsf?hrer: Rainer J. H. Brandt und Volker A. Brandt