On Jan 5, 2013, at 9:42 AM, Russ Poyner <rpoyner at engr.wisc.edu> wrote:
> I''m configuring a box with 24x 3Tb consumer SATA drives, and
wondering about the best way to configure the pool. The customer wants capacity
on the cheap, and I want something I can service without sweating too much about
data loss. Due to capacity concerns raid 10 is probably out, which leaves
various raidz choices
You should consider space, data dependability as measured by Mean Time to Data
Loss (MTTDL), and performance.
For MTTDL[1] model, let''s use 700k hours MTBF for the disks and 168
hours for
recovery (48 hours logistical + 120 hours resilver of a full disk)
For performance, lets hope for 7,200 rpms and about 80 IOPS for small, random
reads
with 100% cache miss.
>
> 1. A stripe of four 6 disk raidz2
Option 1
space ~= 4 * (6 - 2) * 3TB = 48 TB
MTTDL[1] = 8.38e+5 years or 0.000119% Annualized Failure Rate (AFR)
small, random read performance, best of worst case = 4 * (6/4) * 80 IOPS = 480
IOPS
> 2. A stripe of two 11 disk raidz3 with 2 hot spares.
Option 2
space ~= 2 * (11 - 3) * 3TB = 48 TB
MTTDL[1] = 3.62e+7 years or 0.000003% AFR
small, random read performance, best of worst case = 2 * (11/8) * 80 IOPS =
220 IOPS
Option 2a (no hot spares)
space ~= 2 * (12 - 3) * 3TB = 54 TB
MTTDL[2] = 1.90e+7 years or 0.000005% AFR
small, random read performance, best of worst case = 2 * (12/9) * 80 IOPS =
213 IOPS
>
> Other, better ideas?
There are thousands of permutations you could consider :-)
For 24-bay systems with double parity or better, we also see a 3x8-disk as a
common configuration. Offhand, I''d say we see more 4x6-disk and
3x8-disk
configs than any configs with more than 10 disks per set.
>
> My questions are
>
> A. How long will resilvering take with these layouts when the disks start
dying?
It depends on the concurrent workload. By default resilvers are throttled and
give
way to other workload. In general, for double or triple parity RAID, you
don''t need
to worry too much on a per-disk basis. The conditions you need to worry about
are
where the failure cause is common to all disks, such as a controller, fans,
cabling,
or power because they are more likely than a triple failure of disks (as clearly
shown
by the MTTDL[1] model results above)
>
> B. Should I prefer hot spares or additional parity drives, and why?
In general, addional parity is better than hot spares. You get more performance
and better data dependability.
>
> The box is a supermicro with 36 bays controlled through a single LSI
9211-8i. There is a separate intel 320 ssd for the OS. The purpose is to backup
data from the customer''s windows workstations. I''m leaning
toward using BackupPC for the backups since it seems to combine good efficiency
with a fairly customer-friendly web interface.
Sounds like a good plan.
-- richard
>
> I''m running FreeBSD 9, after having failed to get the plugin jail
working in FreeNAS, also for personal reasons I find csh easier to use than the
FreeNAS web interface. My impression is that FreeBSD combines a mature OS with
the second oldest/best (after Illumos) free implementation of zfs.
>
> Thanks in advance
> Russ Poyner
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
--
Richard.Elling at RichardElling.com
+1-760-896-4422
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20130105/225285ec/attachment.html>