First I''d like to note that contrary to the nomenclature there
isn''t any one "SAN" product that all operates the same. There
are a number of different vendor provided solutions that use a FC SAN to deliver
luns to hosts, and they each have their own limitations. Forgive my pedanticism
please.
>On Sun, Oct 28, 2012 at 04:43:34PM +0700, Fajar A. Nugraha wrote:
> On Sat, Oct 27, 2012 at 9:16 PM, Edward Ned Harvey
> (opensolarisisdeadlongliveopensolaris)
> <opensolarisisdeadlongliveopensolaris at nedharvey.com> wrote:
> >> From: zfs-discuss-bounces at opensolaris.org
[mailto:zfs-discuss-()
> >> bounces at opensolaris.org] On Behalf Of Fajar A. Nugraha
> >>
> >> So my
> >> suggestion is actually just present one huge 25TB LUN to zfs and
let
> >> the SAN handle redundancy.
>
> > create a bunch of 1-disk volumes and let ZFS handle them as if
they''re JBOD.
>
> Last time I use IBM''s enterprise storage (which was, admittedly, a
> long time ago) you can''t even do that. And looking at
Morris'' mail
> address, it should be revelant :)
>
> ... or probably it''s just me who haven''t found how to do
that. Which
> why I suggested just use whatever the SAN can present :)
Many vendors don''t allow the hosts to control the disk - or only do it
on their lower end storage. Usually because the higher end stuff has
''intelligent'' caching, extra processors, etc etc, to handle
the storage workload in their own way.
>
>You are entering the uncharted waters of ``multi-level disk
>>management'''' here. Both ZFS and the SAN use redundancy
and error-
>checking to ensure data integrity. Both of them also do automatic
>replacement of failing disks. A good SAN will present LUNs that
>>behave as perfectly reliable virtual disks, guaranteed to be error
>free. Almost all of the time, ZFS will find no errors. If ZFS does
>find an error, there''s no nice way to recover. Most commonly, this
>happens when the SAN is powered down or rebooted while the ZFS host
>is still running.
That''s been my experience - ZFS plays nice, we do scrubs monthly to
check for any data problems, which we have to get fixed from backups. The
biggest messes happen when all the paths go offline for whatever reason while
the host is still running. We have redundant networks of SAN switches and our
storage arrays don''t get powered down if we can help it, so that
doesn''t come up too often, usually only when someone is playing with
the cables in an unfriendly way.
All that being said, the real limitations probably end up being on your
hardware. For example, one vendor''s tier one solution only handles
synchronous mirroring on 4TB luns or smaller - so if you''re using that
vendor''s software and doing synchronous mirroring, you''d want
to do a bunch of 4TB or less luns. Some vendors do
''wide-striping'' across pools of disk on the back end, but some
don''t. If yours doesn''t, and you can''t present the
disks directly, you''ll want to find out what layout on the back end is
optimized for the best performance for your particular workload via ZFS
(database, digital media streaming, etc), and set it up that way. That will
probably tell you what LUN sizes to use.
On your host side, there''s also the consideration of ssd/scsi queuing.
If you''re running on only one LUN, you''re limiting your IOPS
to only one IO queue over your FC paths, and if you have that throttled (per
many storage vendors recommendations about ssd:ssd_max_throttle and
zfs:zfs_vdev_max_pending), then one LUN will throttle your IOPS back on your
host. That might also motivate you to split into multiple LUNS so your OS
doesn''t end up bottle-necking your IO before it even gets to your SAN
HBA.
Summary - my experience on FC SANs (previous and ongoing) is that ZFS is great
in that it doesn''t tell me what LUN sizes are the best to use.
It''s a combination of what my storage array limitations and strengths
are, as well as my OS configuration and application workload that usually end up
telling me if I should use one big lun, or twenty small ones. And when I get it
wrong (because what percentage of times does an app admin/DBA accurately guess
their IOPS and storage bandwidth needs), ZFS usually lets me fix it, sometimes
even non-disruptive to uptime.
Cheers,
Brian
--
-Gary Mills- -refurb- -Winnipeg, Manitoba, Canada-
_______________________________________________
zfs-discuss mailing list
zfs-discuss at opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss