CentOS - Mar 2013 - [OT/HW] hardware raid -- comment/experience with 3Ware

Greetings,

I am looking for a hardware raid card that supports up to 4 SATA II
hard disks with hot swap (compatible raid cage)

I have short listed two LSI/3Ware cards:
1. 9750-4i
2. 9650SE-4LPML

Both appear to be well supported in Linux.

I would appreciate your personal experience with the CLI tools
provided by LSI.
Can they be configured to send email for disk failures or SMART errors?
Is there a Web interface for monitoring?

Any preference between 1 and 2 above.

Thanks for your time and suggestions.
-- Arun Khan

On 03/06/2013 08:35 AM, Arun Khan wrote:
>
> Both appear to be well supported in Linux.
They are.
> I would appreciate your personal experience with the CLI tools
> provided by LSI.
> Can they be configured to send email for disk failures or SMART errors?
Not the CLI tools.  You'll need to run 3dm2.
> Is there a Web interface for monitoring?
Yes, that is also provided by 3dm2.
> Any preference between 1 and 2 above.
Based on about 10 years of running a hundred or so systems with 3ware 
controllers, I would say that you're better off with an LSI MegaRAID 
card, or with Linux software RAID.  3ware cards themselves have been the 
most problematic component of any system I've run in my entire 
professional career (starting in 1996).  Even very recent cards fail in 
a wide variety of ways, and there is no guarantee that if your array 
fails using a controller that you buy now that you'll be able to connect 
it to a controller that you buy later.

At this point, I deploy almost exclusively systems running Linux with 
KVM on top of software RAID.  While I lose the battery backed write 
cache (which is great for performance unless you sustain enough writes 
to fill it completely, at which point the system grinds nearly to a 
halt), I gain a consistent set of management tools and the ability to 
move a disk array to any hardware that accepts the same form factor 
disk.  The reliability of my systems has improved significantly since I 
moved to software RAID.

On 2013-03-06, Arun Khan <knura9 at gmail.com>
wrote:
>
> I am looking for a hardware raid card that supports up to 4 SATA II
> hard disks with hot swap (compatible raid cage)
>
> I have short listed two LSI/3Ware cards:
> 1. 9750-4i
> 2. 9650SE-4LPML
>
> I would appreciate your personal experience with the CLI tools
> provided by LSI.
> Can they be configured to send email for disk failures or SMART errors?
Yes, with the 3dm2 monitor.  There was a bug (which may still exist)
where using the hostname of the SMTP server did not work, so you should
try the IP address if you try using a DNS name and it doesn't work.  You
may also want to adjust the EmailSeverity option in 3dm.conf; I use
EmailSeverity 3 which sends me information messages like array
verifications and BBU charging events in addition to disk failures and
SMART errors.  I believe the default setting only sends out errors.
> Is there a Web interface for monitoring?
3dm2 can also provide a web server, though I've never used it.

The command line tools (tw_cli) are okay, but they don't have a fabulous
API.  In particular the output is basically human readable text, which
means you need to do your own text parsing if you want to use the output
in automated tools like nagios.
> Any preference between 1 and 2 above.
I would have a slight preference for the 9750 series.  You get a faster
controller for a very similar price.

The only other thing I would recommend is that you not rely on the
reshape feature of these 3ware cards.  My first test many years ago
actually destroyed the array, and I aborted my second test last year
because the reshape was probably going to take many weeks.  (This was,
IIRC, a reshape from 4 3TB disks to 5.  Linux md can do this reshape in
1-2 days.)  If you need to add space you should use tools like LVM
instead.

The other standard warnings about using these controllers apply (e.g.,
if using the write cache, have a BBU on the card and a UPS on the
server; do regular verifies on your redundant arrays; RAID is not a
backup system).

--keith




-- 
kkeller at wombat.san-francisco.ca.us

On 03/06/2013 10:56 AM, Keith Keller wrote:
> The other standard warnings about using these controllers apply (e.g.,
> if using the write cache, have a BBU on the card
A 3ware card will not enable write caching unless a BBU is present.