On Mon, May 9, 2011 at 8:33 AM, Darren Honeyball <mlist at spod.net>
wrote:> I''m just mulling over the best configuration for this system - our
work load is mostly writing millions of small files (around 50k) with occasional
reads & we need to keep as much space as possible.
If space is a priority, then raidz or raidz2 are probably the best
bets. If you''re going to have a lot of random iops, then mirrors are
best.
You have some control over the performance : space ratio with raidz by
adjusting the width of the radiz vdevs. For instance, mirrors will
provide 34TB of space and best random iops. 24 x 3-disk raidz vdevs
will have 48TB of space but still have pretty strong random iops
performance. 13 x 5-disk raidz vdevs will give 52TB of space at the
lost of lower random iops.
Testing will help you find the best configuration for your environment.
> HP''s recommendations for configuring the MDS 600 with ZFS is to
let the P212 do the raid functions (raid 1+0 is recommended here) by configuring
each half of the MDS 600 as a single logical drive (35 drives) & then use a
basic zfs pool on top to provide the zfs functionality - to me this would seem
to loose a lot of the error checking functions of zfs?
If you configured the two logical drives as a mirror in ZFS, then
you''d still have full protection. Your overhead would be really high
though - 3/4 of your original capacity would be used for data
protection if I understand the recommendation correctly. (You''d use
1/2 of the original capacity for RAID1 in the MDS, then 1/2 of the
remaining for the ZFS mirror.) You could use non-redundant pool in ZFS
to reduce the overhead, but you sacrifice the self-healing properties
of ZFS when you do that.
> Another option is to use raidz and let zfs handle the smart stuff - as the
P212 doesn''t support a true dumb JBOD function I''d need to
create each drive as a single raid 0 logical drive - are there any drawback to
doing this? Or would it be better to create slightly larger logical drives using
say 2 physical drives per logical drive?
Single-device logical drives are required when you can''t configure a
card or device as JBOD, and I believe its usually the recommended
solution. Once you have the LUNs created, you can use ZFS to create
mirrors or raidz vdevs.
> I''m planning on having 2 hot spares - one in each side of the MDS
600, is it also worth using a dedicated ZIL spindle or 2?
It would depend on your workload. (How''s that for helpful?)
If you''re experiencing a lot of synchronous writes, then a ZIL will
help. If you aren''t seeing a lot of sync writes, then a ZIL
won''t
help. The ZIL doesn''t have to be very large, since it''s
flushed on a
regular basis. From the Best Practices guide:
"For a target throughput of X MB/sec and given that ZFS pushes
transaction groups every 5 seconds (and have 2 outstanding), we also
expect the ZIL to not grow beyond X MB/sec * 10 sec. So to service
100MB/sec of synchronous writes, 1 GB of log device should be
sufficient."
If the MDS has a non-volatile cache, there should be little or no need
to use a ZIL.
However, some reports have shown ZFS with a ZIL to be faster than
using non-volatile cache. You should test performance using your
workload.
> Is it worth tweaking zfs_nocacheflush or zfs_vdev_max_pending?
As I mentioned above, if the MDS has a non-volatile cache, then
setting zfs_nocacheflush might help performance.
If you''re exporting one LUN per device then you shouldn''t need
to
adjust the max_pending. If you''re exporting larger RAID10 luns from
the MDS, then increasing the value might help for read workloads.
-B
--
Brandon High : bhigh at freaks.com