Hello zfs-discuss, One of a disk started to behave strangely. Apr 11 16:07:42 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:07:42 thumper-9.srv port 6: device reset Apr 11 16:07:42 thumper-9.srv scsi: [ID 107833 kern.warning] WARNING: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1/disk at 6,0 (sd27): Apr 11 16:07:42 thumper-9.srv Error for Command: read(10) Error Level: Retryable Apr 11 16:07:42 thumper-9.srv scsi: [ID 107833 kern.notice] Requested Block: 41122898 Error Block: 41122898 Apr 11 16:07:42 thumper-9.srv scsi: [ID 107833 kern.notice] Vendor: ATA Serial Number: Apr 11 16:07:42 thumper-9.srv scsi: [ID 107833 kern.notice] Sense Key: No Additional Sense Apr 11 16:07:42 thumper-9.srv scsi: [ID 107833 kern.notice] ASC: 0x0 (no additional sense info), ASCQ: 0x0, FRU: 0x0 Apr 11 16:07:42 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:07:42 thumper-9.srv port 6: device reset Apr 11 16:07:43 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:07:43 thumper-9.srv port 6: link lost Apr 11 16:07:43 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:07:43 thumper-9.srv port 6: link established Apr 11 16:07:43 thumper-9.srv marvell88sx: [ID 812950 kern.warning] WARNING: marvell88sx2: error on port 6: Apr 11 16:07:43 thumper-9.srv marvell88sx: [ID 517869 kern.info] device disconnected Apr 11 16:07:43 thumper-9.srv marvell88sx: [ID 517869 kern.info] device connected Apr 11 16:07:43 thumper-9.srv marvell88sx: [ID 517869 kern.info] SError interrupt Apr 11 16:08:42 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:08:42 thumper-9.srv port 6: device reset Apr 11 16:08:43 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:08:43 thumper-9.srv port 6: device reset Apr 11 16:08:43 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:08:43 thumper-9.srv port 6: link lost Apr 11 16:08:43 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:08:43 thumper-9.srv port 6: link established Apr 11 16:08:43 thumper-9.srv marvell88sx: [ID 812950 kern.warning] WARNING: marvell88sx2: error on port 6: Apr 11 16:08:43 thumper-9.srv marvell88sx: [ID 517869 kern.info] device disconnected Apr 11 16:08:43 thumper-9.srv marvell88sx: [ID 517869 kern.info] device connected Apr 11 16:08:43 thumper-9.srv marvell88sx: [ID 517869 kern.info] SError interrupt Apr 11 16:09:44 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:09:44 thumper-9.srv port 6: device reset Apr 11 16:09:44 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:09:44 thumper-9.srv port 6: device reset Apr 11 16:09:45 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:09:45 thumper-9.srv port 6: link lost Apr 11 16:09:45 thumper-9.srv sata: [ID 801593 kern.notice] NOTICE: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1: Apr 11 16:09:45 thumper-9.srv port 6: link established Apr 11 16:09:45 thumper-9.srv marvell88sx: [ID 812950 kern.warning] WARNING: marvell88sx2: error on port 6: Apr 11 16:09:45 thumper-9.srv marvell88sx: [ID 517869 kern.info] device disconnected Apr 11 16:09:45 thumper-9.srv marvell88sx: [ID 517869 kern.info] device connected Apr 11 16:09:45 thumper-9.srv marvell88sx: [ID 517869 kern.info] SError interrupt Apr 11 16:09:45 thumper-9.srv scsi: [ID 107833 kern.warning] WARNING: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1/disk at 6,0 (sd27): Apr 11 16:09:45 thumper-9.srv Error for Command: read(10) Error Level: Retryable Apr 11 16:09:45 thumper-9.srv scsi: [ID 107833 kern.notice] Requested Block: 41122898 Error Block: 41122898 Apr 11 16:09:45 thumper-9.srv scsi: [ID 107833 kern.notice] Vendor: ATA Serial Number: Apr 11 16:09:45 thumper-9.srv scsi: [ID 107833 kern.notice] Sense Key: No Additional Sense Apr 11 16:09:45 thumper-9.srv scsi: [ID 107833 kern.notice] ASC: 0x0 (no additional sense info), ASCQ: 0x0, FRU: 0x0 zpool status, df -h, ls, etc. all hung so operator just removed the disk. HotSpare kicked in and started resilvering but starts all over again (sometimes 1-3% is done and it restarts, sometimes 20%). No snapshots are on a system and no new snapshots or file systems are created or destroyed however there''re IOs to the pool from nfsd (not much writes). bash-3.00# uname -a SunOS thumper-9.srv 5.10 Generic_125101-04 i86pc i386 i86pc bash-3.00# pool: thumper-9 state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scrub: resilver in progress, 0.35% done, 8h19m to go config: NAME STATE READ WRITE CKSUM thumper-9 DEGRADED 0 0 0 raidz2 ONLINE 0 0 0 c0t0d0 ONLINE 0 0 0 c1t0d0 ONLINE 0 0 0 c4t0d0 ONLINE 0 0 0 c6t0d0 ONLINE 0 0 0 c7t0d0 ONLINE 0 0 0 c0t1d0 ONLINE 0 0 0 c1t1d0 ONLINE 0 0 0 c5t1d0 ONLINE 0 0 0 c6t1d0 ONLINE 0 0 0 c7t1d0 ONLINE 0 0 0 c0t2d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c5t2d0 ONLINE 0 0 0 c6t2d0 ONLINE 0 0 0 c7t2d0 ONLINE 0 0 0 c0t4d0 ONLINE 0 0 0 c1t4d0 ONLINE 0 0 0 c4t4d0 ONLINE 0 0 0 c6t4d0 ONLINE 0 0 0 c7t4d0 ONLINE 0 0 0 c0t3d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 raidz2 ONLINE 0 0 0 c4t3d0 ONLINE 0 0 0 c5t3d0 ONLINE 0 0 0 c6t3d0 ONLINE 0 0 0 c7t3d0 ONLINE 0 0 0 c0t5d0 ONLINE 0 0 0 c1t5d0 ONLINE 0 0 0 c4t5d0 ONLINE 0 0 0 c5t5d0 ONLINE 0 0 0 c6t5d0 ONLINE 0 0 0 c7t5d0 ONLINE 0 0 0 c0t6d0 ONLINE 0 0 0 raidz2 DEGRADED 0 0 0 c1t6d0 ONLINE 0 0 0 replacing DEGRADED 0 0 0 c4t6d0 UNAVAIL 0 0 0 cannot open c4t2d0 ONLINE 0 0 0 c5t6d0 ONLINE 0 0 0 c6t6d0 ONLINE 0 0 0 c7t6d0 ONLINE 0 0 0 c0t7d0 ONLINE 0 0 0 c1t7d0 ONLINE 0 0 0 c4t7d0 ONLINE 0 0 0 c5t7d0 ONLINE 0 0 0 c6t7d0 ONLINE 0 0 0 c7t7d0 ONLINE 0 0 0 spares c4t1d0 AVAIL c4t2d0 INUSE currently in use errors: No known data errors In logs I keep getting: Apr 11 21:46:09 thumper-9.srv scsi: [ID 107833 kern.warning] WARNING: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1/disk at 6,0 (sd27): Apr 11 21:46:09 thumper-9.srv offline or reservation conflict Apr 11 21:46:09 thumper-9.srv scsi: [ID 107833 kern.warning] WARNING: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1/disk at 6,0 (sd27): Apr 11 21:46:09 thumper-9.srv offline or reservation conflict Apr 11 21:47:10 thumper-9.srv scsi: [ID 107833 kern.warning] WARNING: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1/disk at 6,0 (sd27): Apr 11 21:47:10 thumper-9.srv offline or reservation conflict Apr 11 21:47:10 thumper-9.srv scsi: [ID 107833 kern.warning] WARNING: /pci at 1,0/pci1022,7458 at 3/pci11ab,11ab at 1/disk at 6,0 (sd27): Apr 11 21:47:10 thumper-9.srv offline or reservation conflict Exporting/importing the pool doesn''t help for those messages. I rebooted server. It helped to silence above log entries. I also stopped nfsd so no IOs are issued to the pool except resilvering. bash-3.00# fsstat zfs 1 new name name attr attr lookup rddir read read write write file remov chng get set ops ops ops bytes ops bytes 0 4.50K 1 487K 0 75.3K 65 40.8K 981M 53 233K zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs 0 0 0 0 0 0 0 0 0 0 0 zfs ^C bash-3.00# bash-3.00# zpool iostat thumper-9 1 capacity operations bandwidth pool used avail read write read write ---------- ----- ----- ----- ----- ----- ----- thumper-9 761G 19.3T 575 346 8.96M 920K thumper-9 761G 19.3T 437 186 7.92M 281K thumper-9 761G 19.3T 452 259 4.07M 371K thumper-9 761G 19.3T 592 321 9.43M 460K thumper-9 761G 19.3T 421 196 8.25M 254K thumper-9 761G 19.3T 550 201 22.6M 290K ^C bash-3.00# But: bash-3.00# iostat -xnz 1 [removed first output] extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 25.1 0.0 904.7 0.0 0.0 0.1 0.8 5.9 2 15 c4t0d0 20.1 0.0 711.0 0.0 0.0 0.1 0.0 5.9 0 12 c6t0d0 23.1 0.0 776.3 0.0 0.0 0.1 1.5 6.2 3 14 c1t0d0 22.1 0.0 585.6 0.0 0.0 0.1 0.3 6.1 1 14 c6t1d0 20.1 0.0 584.6 0.0 0.0 0.1 0.0 4.3 0 9 c5t1d0 23.1 0.0 776.3 0.0 0.0 0.1 0.9 6.1 2 14 c1t1d0 13.0 0.0 261.9 0.0 0.0 0.1 0.0 7.1 0 9 c6t2d0 0.0 203.7 0.0 1583.6 0.0 0.0 0.1 0.2 1 3 c4t2d0 14.0 0.0 198.7 0.0 0.0 0.1 0.0 4.2 0 6 c1t2d0 10.0 0.0 69.7 0.0 0.0 0.1 0.0 6.4 0 6 c5t2d0 17.1 0.0 709.5 0.0 0.0 0.1 0.8 7.0 1 12 c6t3d0 17.1 0.0 646.3 0.0 0.0 0.1 1.0 5.4 2 9 c4t3d0 20.1 0.0 328.7 0.0 0.0 0.1 0.0 2.9 0 6 c1t3d0 17.1 0.0 710.0 0.0 0.0 0.1 0.0 6.8 0 12 c5t3d0 16.1 0.0 262.9 0.0 0.0 0.1 0.0 4.8 0 8 c4t4d0 14.0 0.0 134.5 0.0 0.0 0.1 0.0 5.9 0 8 c6t4d0 16.1 0.0 199.2 0.0 0.0 0.1 1.2 5.9 2 9 c1t4d0 19.1 0.0 583.6 0.0 0.0 0.1 0.0 4.6 0 9 c4t5d0 22.1 0.0 585.1 0.0 0.0 0.1 0.2 3.8 1 8 c1t5d0 18.1 0.0 455.1 0.0 0.0 0.1 0.6 4.7 1 8 c6t5d0 20.1 0.0 647.3 0.0 0.0 0.1 0.9 5.6 2 11 c0t0d0 20.1 0.0 711.0 0.0 0.0 0.1 0.1 6.0 0 12 c7t0d0 52.2 87.3 2278.1 44.7 0.3 0.2 2.4 1.2 12 16 c1t6d0 53.2 76.3 2257.5 38.1 0.4 0.2 3.0 1.2 13 16 c6t6d0 24.1 0.0 840.5 0.0 0.0 0.1 0.2 5.5 0 13 c0t1d0 22.1 0.0 585.1 0.0 0.0 0.1 0.1 6.0 0 13 c7t1d0 15.1 0.0 453.6 0.0 0.0 0.1 0.8 5.2 1 8 c5t5d0 53.2 84.3 2255.5 42.1 0.4 0.2 2.6 1.1 12 16 c4t7d0 53.2 85.3 2257.5 43.2 0.4 0.2 3.2 1.2 13 17 c1t7d0 54.2 91.3 2269.6 45.7 0.3 0.1 2.3 0.9 10 14 c6t7d0 22.1 0.0 712.0 0.0 0.0 0.1 0.0 5.5 0 12 c0t2d0 48.2 73.3 2198.8 36.6 0.2 0.1 1.9 1.0 7 12 c5t6d0 13.0 0.0 197.7 0.0 0.0 0.1 0.0 4.9 0 6 c7t2d0 54.2 88.3 2270.1 44.2 0.3 0.1 1.9 0.8 9 12 c5t7d0 18.1 0.0 200.7 0.0 0.0 0.1 0.5 4.3 1 8 c0t3d0 19.1 0.0 647.3 0.0 0.0 0.1 0.2 5.2 0 10 c7t3d0 13.0 0.0 70.2 0.0 0.0 0.1 0.0 5.9 0 8 c0t4d0 14.0 0.0 71.3 0.0 0.0 0.1 0.0 5.0 0 7 c7t4d0 20.1 0.0 584.1 0.0 0.0 0.1 0.3 4.7 1 9 c0t5d0 17.1 0.0 454.6 0.0 0.0 0.1 0.0 5.6 0 10 c7t5d0 18.1 0.0 582.6 0.0 0.0 0.1 0.0 6.9 0 12 c0t6d0 53.2 78.3 2257.5 39.1 0.4 0.2 2.9 1.2 12 16 c7t6d0 53.2 87.3 2257.5 43.7 0.3 0.1 2.1 1.0 9 15 c0t7d0 54.2 88.3 2269.1 45.2 0.3 0.1 2.3 1.0 11 15 c7t7d0 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 21.0 0.0 645.5 0.0 0.0 0.1 0.2 5.1 0 11 c4t0d0 24.0 0.0 710.5 0.0 0.0 0.1 0.0 3.7 0 9 c6t0d0 27.0 0.0 648.5 0.0 0.0 0.1 0.4 4.5 1 12 c1t0d0 39.0 0.0 908.9 0.0 0.0 0.2 0.1 3.9 0 15 c6t1d0 28.0 0.0 712.5 0.0 0.0 0.1 1.4 4.5 3 13 c5t1d0 21.0 0.0 709.0 0.0 0.0 0.1 1.0 5.1 2 11 c1t1d0 12.0 0.0 196.5 0.0 0.0 0.1 0.0 5.9 0 7 c6t2d0 0.0 263.0 0.0 1574.9 0.0 0.0 0.1 0.1 2 4 c4t2d0 16.0 0.0 262.0 0.0 0.0 0.1 0.0 4.3 0 7 c1t2d0 19.0 0.0 454.0 0.0 0.0 0.1 0.0 5.3 0 10 c5t2d0 19.0 0.0 772.5 0.0 0.0 0.1 0.6 5.2 1 10 c6t3d0 24.0 0.0 774.0 0.0 0.0 0.1 0.7 4.2 2 10 c4t3d0 15.0 0.0 325.0 0.0 0.0 0.1 0.0 3.8 0 6 c1t3d0 20.0 0.0 835.5 0.0 0.0 0.1 0.9 4.9 2 10 c5t3d0 13.0 0.0 134.0 0.0 0.0 0.1 0.0 6.6 0 9 c4t4d0 14.0 0.0 198.0 0.0 0.0 0.1 0.0 4.8 0 7 c6t4d0 12.0 0.0 324.0 0.0 0.0 0.1 0.0 6.9 0 8 c1t4d0 21.0 0.0 962.9 0.0 0.0 0.1 0.6 6.5 1 14 c4t5d0 27.0 0.0 1029.4 0.0 0.0 0.2 0.9 6.2 2 17 c1t5d0 20.0 0.0 708.5 0.0 0.0 0.1 1.2 6.7 2 13 c6t5d0 25.0 0.0 584.0 0.0 0.0 0.1 0.0 4.0 0 10 c0t0d0 29.0 0.0 840.5 0.0 0.0 0.1 1.1 4.8 3 14 c7t0d0 55.0 119.0 2276.4 60.0 0.4 0.2 2.3 1.1 15 20 c1t6d0 54.0 102.0 2251.9 51.5 0.3 0.1 1.7 0.9 8 14 c6t6d0 22.0 0.0 710.0 0.0 0.0 0.1 1.1 5.3 2 12 c0t1d0 32.0 0.0 968.4 0.0 0.0 0.2 0.5 5.1 1 16 c7t1d0 18.0 0.0 707.5 0.0 0.0 0.1 0.6 7.5 1 14 c5t5d0 54.0 124.0 2312.4 62.0 0.3 0.1 1.7 0.8 11 15 c4t7d0 54.0 112.0 2311.4 56.0 0.3 0.2 1.8 1.0 12 16 c1t7d0 53.0 110.0 2248.4 55.5 0.4 0.2 2.3 1.1 14 18 c6t7d0 24.0 0.0 647.0 0.0 0.0 0.1 0.0 4.5 0 11 c0t2d0 54.0 120.0 2251.9 60.5 0.3 0.2 2.0 1.0 11 17 c5t6d0 6.0 0.0 3.5 0.0 0.0 0.0 0.0 5.9 0 4 c7t2d0 56.0 113.0 2251.9 57.0 0.5 0.2 2.9 1.1 16 19 c5t7d0 13.0 0.0 260.5 0.0 0.0 0.1 0.0 4.1 0 5 c0t3d0 18.0 0.0 645.0 0.0 0.0 0.1 0.2 6.1 0 11 c7t3d0 8.0 0.0 258.5 0.0 0.0 0.1 0.0 7.1 0 6 c0t4d0 9.0 0.0 131.5 0.0 0.0 0.1 1.0 7.0 1 6 c7t4d0 26.0 0.0 839.0 0.0 0.0 0.1 0.4 5.5 1 14 c0t5d0 22.0 0.0 773.0 0.0 0.0 0.1 1.0 6.6 2 14 c7t5d0 24.0 0.0 774.0 0.0 0.0 0.1 0.4 5.5 1 13 c0t6d0 53.0 117.0 2247.4 59.0 0.4 0.2 2.3 1.1 15 19 c7t6d0 54.0 115.0 2311.9 58.5 0.4 0.2 2.5 1.2 15 21 c0t7d0 55.0 120.0 2277.4 61.0 0.3 0.2 1.9 0.9 12 16 c7t7d0 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 21.0 0.0 518.5 0.0 0.0 0.1 0.0 4.7 0 10 c4t0d0 18.0 0.0 453.5 0.0 0.0 0.1 0.4 4.6 1 8 c6t0d0 24.0 0.0 710.5 0.0 0.0 0.1 0.2 5.5 0 13 c1t0d0 30.0 0.0 840.5 0.0 0.0 0.1 0.4 4.6 1 14 c6t1d0 33.0 0.0 778.5 0.0 0.0 0.1 0.0 3.5 0 12 c5t1d0 39.0 0.0 1035.5 0.0 0.0 0.2 0.0 3.9 0 15 c1t1d0 11.0 0.0 259.5 0.0 0.0 0.1 0.0 5.6 0 6 c6t2d0 0.0 210.0 0.0 908.0 0.0 0.0 0.1 0.1 1 3 c4t2d0 14.0 0.0 197.5 0.0 0.0 0.1 0.0 3.9 0 6 c1t2d0 12.0 0.0 260.0 0.0 0.0 0.1 0.0 4.6 0 5 c5t2d0 24.0 0.0 965.0 0.0 0.0 0.1 0.0 5.7 0 14 c6t3d0 21.0 0.0 963.0 0.0 0.0 0.1 0.5 5.7 1 12 c4t3d0 11.0 0.0 196.0 0.0 0.0 0.1 0.0 5.1 0 6 c1t3d0 24.0 0.0 1028.5 0.0 0.0 0.2 0.3 7.1 1 17 c5t3d0 17.0 0.0 390.0 0.0 0.0 0.1 0.0 4.0 0 7 c4t4d0 18.0 0.0 326.5 0.0 0.0 0.1 0.5 5.3 1 10 c6t4d0 18.0 0.0 454.0 0.0 0.0 0.1 0.0 4.6 0 8 c1t4d0 18.0 0.0 517.5 0.0 0.0 0.1 0.7 5.7 1 10 c4t5d0 22.0 0.0 709.5 0.0 0.0 0.1 0.1 5.6 0 12 c1t5d0 26.0 0.0 965.5 0.0 0.0 0.1 0.4 5.3 1 14 c6t5d0 24.0 0.0 774.5 0.0 0.0 0.1 0.4 4.5 1 11 c0t0d0 29.0 0.0 776.5 0.0 0.0 0.1 0.3 5.0 1 14 c7t0d0 39.0 108.0 1419.0 56.0 0.4 0.2 2.8 1.5 18 22 c1t6d0 39.0 115.0 1496.5 59.5 0.7 0.3 4.3 1.9 21 29 c6t6d0 41.0 0.0 1100.0 0.0 0.0 0.2 0.1 4.0 0 16 c0t1d0 27.0 0.0 903.0 0.0 0.0 0.2 0.9 6.4 2 17 c7t1d0 22.0 0.0 773.5 0.0 0.0 0.1 0.8 5.5 2 12 c5t5d0 41.0 115.0 1498.0 60.0 0.5 0.3 3.0 1.7 19 26 c4t7d0 41.0 125.0 1632.0 64.0 0.6 0.3 3.7 2.0 23 33 c1t7d0 41.0 110.0 1498.5 57.0 0.6 0.2 4.0 1.6 21 25 c6t7d0 23.0 0.0 711.0 0.0 0.0 0.1 0.3 5.9 1 14 c0t2d0 38.0 112.0 1546.5 58.0 0.6 0.2 4.0 1.6 20 25 c5t6d0 12.0 0.0 387.0 0.0 0.0 0.1 0.0 5.5 0 7 c7t2d0 45.0 113.0 1641.0 59.0 0.5 0.3 3.4 1.7 21 27 c5t7d0 9.0 0.0 195.0 0.0 0.0 0.0 0.0 3.4 0 3 c0t3d0 23.0 0.0 901.0 0.0 0.0 0.1 0.2 5.4 1 12 c7t3d0 21.0 0.0 582.0 0.0 0.0 0.1 0.0 4.7 0 10 c0t4d0 12.0 0.0 260.0 0.0 0.0 0.0 0.0 4.2 0 5 c7t4d0 24.0 0.0 965.0 0.0 0.0 0.1 0.1 6.1 0 15 c0t5d0 20.0 0.0 772.5 0.0 0.0 0.1 0.0 6.3 0 13 c7t5d0 19.0 0.0 898.5 0.0 0.0 0.1 0.4 6.6 1 13 c0t6d0 42.0 112.0 1561.5 57.5 0.3 0.2 2.3 1.3 14 21 c7t6d0 37.0 112.0 1432.5 59.0 0.6 0.3 3.9 1.7 18 25 c0t7d0 34.0 109.0 1430.5 57.0 0.4 0.2 2.6 1.6 16 23 c7t7d0 ^C Why there''re writes to other disks in a pool except hotspare disk c4t2d0? And why resilver is so slow... there''s still ~97% idle cpu and no applications are running. nfsd started again and I watch resilver progress. So far so good - it''s slowly progressing in a right direction :) (55% done right now) -- Best regards, Robert mailto:rmilkowski at task.gda.pl http://milek.blogspot.com