zfs discuss - Mar 2012 - Any recommendations on Perc H700 controller on Dell Rx10 ?

Hi folks:

At work, I have an R510, and R610 and an R710 - all with the H700 PERC
controller.

Based on experiments, it seems like there is no way to bypass the PERC
controller - it seems like one can only access the individual disks if
they are set up in RAID0 each.

This brings me to ask some questions:
a. Is it fine (in terms of an intelligent controller coming in the way
of ZFS) to have the PERC controllers present each drive as RAID0
drives ?
b. Would there be any errors in terms of PERC doing things that ZFS is
not aware of and this causing any issues later  ?

-- Sriram

Le 10/03/2012 10:29, Sriram Narayanan a ?crit :
> Hi folks:
> 
> At work, I have an R510, and R610 and an R710 - all with the H700 PERC
> controller.
> 
> Based on experiments, it seems like there is no way to bypass the PERC
> controller - it seems like one can only access the individual disks if
> they are set up in RAID0 each.
> 
> This brings me to ask some questions:
> a. Is it fine (in terms of an intelligent controller coming in the way
> of ZFS) to have the PERC controllers present each drive as RAID0
> drives ?
> b. Would there be any errors in terms of PERC doing things that ZFS is
> not aware of and this causing any issues later  ?
> 
> -- Sriram
Hi,
can''t answer for this precise Raid controller, but i had the same
problem with  3Ware raid controllers ( 2 different flavor of 9500S PCIX).

It seems to be mandatory to use each disk as a single raid0 array in
order to be able to export the disk for ZFS.It''s the only way to use
cache of the controller.

That way, it works like a charm.

Furthermore, when a disk died (HW failure, certainly on a mechanical
part, according to the weird sound produced), ZFS did the job.
zfs offline
zfs detach
zfs attach
zfs scrub.

_BUT_, i don''t know about disk errors being silently hidden by the
controller to ZFS!
I''d have thought that, because of the CopyOnWrite mechanism, it may be
impossible to the controller to do such a awful job; am i wrong ??

BTW, i have to spend a few buckets for a SAN (ZFS,iSCSI on FreeBSD). Are
H700 good for the job ? Especially on FreeBSD (if you tested it of course)?

Thanks.
Best regards.

On 03/10/12 09:29, Sriram Narayanan wrote:
> Hi folks:
>
> At work, I have an R510, and R610 and an R710 - all with the H700 PERC
> controller.
>
> Based on experiments, it seems like there is no way to bypass the PERC
> controller - it seems like one can only access the individual disks if
> they are set up in RAID0 each.
>
> This brings me to ask some questions:
> a. Is it fine (in terms of an intelligent controller coming in the way
> of ZFS) to have the PERC controllers present each drive as RAID0
> drives ?
> b. Would there be any errors in terms of PERC doing things that ZFS is
> not aware of and this causing any issues later  ?
I had to produce a ZFS hybrid storage pool performance demo, and was 
initially given a system with a RAID-only controller (different from 
yours, but same idea).

I created the demo with it, but disabled the RAID''s cache as that
wasn''t
what I wanted in the picture. Meanwhile, I ordered the non-RAID version 
of the card.

When it came and I swapped it in. A couple of issues...

ZFS doesn''t recognise any of the disks obviously, because they have 
propitiatory RAID headers on them, so they have to be created again from 
scratch. (That was no big deal in this case, and if it had been, I could 
have done a zfs send and receive to somewhere else temporarily.)

The performance went up, a tiny bit for the spinning disks, and by 50% 
for the SSDs, so the RAID controller was seriously limiting the IOPs of 
the SSDs in particular. This was when SSDs were relatively new, and the 
controllers may not have been designed with SSDs in mind. That''s likely
to be somewhat different nowadays, but I don''t have any data to show 
that either way.

-- 
Andrew Gabriel

In message <CANiY96a2t1n2m0KPMdx2RBODogjOFetOTH2qe4roT6x-Xn8e3A at
mail.gmail.com>
, Sriram Narayanan writes:
>At work, I have an R510, and R610 and an R710 - all with the H700 PERC
>controller.
BTW with some effort, Dell''s sales critter will sell you the 
H200 LSI SAS HBA as a replacement for the H700 LSI MegaRAID
controller for those boxes.
>Based on experiments, it seems like there is no way to bypass the PERC
>controller - it seems like one can only access the individual disks if
>they are set up in RAID0 each.
Did you try deleting all of the RAID0 Virtual Disks and
then enabling JBOD?
# MegaCli -AdpSetProp -EnableJBOD -1 -aALL
# MegaCli -PDMakeJBOD -PhysDrv[E0:S0,E1:S1,...] -aALL

John
groenveld at acm.org

> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss-
> bounces at opensolaris.org] On Behalf Of Sriram Narayanan
> 
> Based on experiments, it seems like there is no way to bypass the PERC
> controller - it seems like one can only access the individual disks if
> they are set up in RAID0 each.
The first consideration to remember is:  When your disks are set up by one
of these controllers, you won''t be able to move these disks to a
generic
controller-less system and expect them to work.  You are dependent on the
raid hardware (or compatible competitors) to make the disks accessible.
Just keep this in mind whenever your system warranty renewal period is
approaching.  Also, if you need to move these disks to a new system that has
a compatible controller, you must configure them exactly the same as they
are on this system - but don''t initialize.

The second consideration you should think about is the writeback cache.  If
you have any SSD dedicated log device, you actually get the best performance
by DISABLING the writeback on all disks.  The second best configuration is
without SSD, you enable writeback on all disks.  And obviously the worst
configuration, you have no SSD and no writeback.