CentOS - Sep 2005 - Software RAID setup ideas and an apology.

The apology first.  If I've offended anyone, or just annoyed anyone, I 
apologize for that.  I sometimes can go 'off the deep end' when I'm
being
sarcastic (an old habit from my news.admin days when I ran a Usenet site in 
the early 90's), and I'll try to not go into news.admin mode here in the
future.

Now, the question.  I have a few ideas about how to do this, but am not sure 
of the ideal way of doing it.  It has to be a pretty common thing to do, so 
I'll ask it here.

I have a server (Dell PE 1600SC) with two SCSI host adapters: a U320 LSI/MPT 
for internal drives only, and an Adaptec 39160 with two internal and two 
external U160 channels.  I have set up a software RAID config with LVM2 on 
top (CentOS 4) with four 36GB 15K RPM Ultra 3 drives.  The internal drives 
are set up with /dev/sda and /dev/sdb partitioned with 100MB /boot (for sda) 
and 100MB /boot2 (backup on sdb), and the rest of each drive as mirrors on a 
RAID1.  /dev/sdc and sdd have each a single RAID partition and are likewise 
mirrors in a RAID1.  So I have /dev/md0 and /dev/md1, which are then two 
parts of a physical volume with the various logical volumes under that.  This 
works and performs well, all things considered.

Now, the 39160 HBA is attached through one of its two channels to a donated 
Cisco Storage Array 12 with 12 18GB drives in it.  Unfortunately, the second 
channel of the SA12 isn't working properly (it automatically configures
split
bus for the array if the second channel is activated), so I'm limited to a 
single channel with 12 drives on it.  I have the hardware portion of this 
working, all drives are recognized, etc both at the Adaptec BIOS level and at 
the Linux kernel level (/dev/sde through /dev/sdp).

My question is, what is the opinion of those on the list with experience in 
this area as to the 'best' method of joining these drives.  I would like
to
be able to lose two or more drives and still have redundancy; I had thought 
about pairing drives in RAID1's and LVM over the array; and I had thought 
about a big RAID5.  But I don't want to shoot performance through the foot, 
either.  I don't have a good hardware RAID controller for this box, so
that's
not at this point an option (I have some hardware RAID controllers, but the 
ones that work ok with this array have much slower SCSI busses, and the boot 
up behavior of this particular array causes the good RAID controllers I do 
have with Ultra 3 interfaces to go crazy (the array powers up when it 
receives signal from the host adapter: not each drive, but the WHOLE ARRAY 
powers up on command; the DPT controllers I have simply refuse to recognize 
any drives at all on the bus, regardless of spinup delay (I set it to 60 
seconds and still no joy; the 39160 recognizes all the drives just fine))). 

Maybe one of those slower controllers might be better than software RAID1; but 
at this point I'm open to suggestions.
-- 
Lamar Owen
Director of Information Technology
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC  28772
(828)862-5554
www.pari.edu

On Sat, 2005-09-10 at 09:16, Lamar Owen wrote:
> My question is, what is the opinion of those on the list with experience in
> this area as to the 'best' method of joining these drives.  I would
like to
> be able to lose two or more drives and still have redundancy; I had thought
> about pairing drives in RAID1's and LVM over the array; and I had
thought
> about a big RAID5.  But I don't want to shoot performance through the
foot,
> either.
I don't know about 'best', but I have several machines with 6 drives
on one controller configured in RAID1 pairs and don't consider
the performance to be a problem.  I started this way back in
the RH 7.x era and did not use LVM - /boot and / are one one
drive set, /home on another and /var on another.  It was important
on some of them that I could yank any single drive and mount
it elsewhere without worrying about the rest of the set. Many
of the original drives have been replaced one at a time, often
hot-swapped and re-synced after a failure.  Some have survived
motherboard failures with the whole set swapped into a spare
chassis for immediate recovery and a couple of boxes were cloned
by swapping every other drive to a new chassis and re-syncing with
new partners. Everything except the ability to grab data from a
single drive should work the same under LVM.   With 12 drives you
probably don't want to deal with that many separate partitions.

I think you are going to lose a little potential speed with
everything on one channel but raid1 only hurts on writes
and doesn't add much CPU hit. And unlike raid5, there is
no hit at all when running with a broken drive in the
set.  I said 'potential' speed because the only time you
use the full bus speed is with several drives transferring
at the same time.  You can arrange this with large files
written over raid0 stripes, but in practice with small
files being randomly accessed tying the heads together
with stripes or raid5 may hurt more than it helps.

-- 
  Les Mikesell
    lesmikesell at gmail.com