CentOS - Feb 2009 - 4 X 500 gb drives - best software raid config for a backup server?

Hi there,

I am currently setting up a server that will house my backups (simply  
using rsync).

This system has 4 X 500 gb drives and I am looking to raid for max  
drive space and data safety. Performance is not so much a concern.

My experience with software raids in nil, so some of these may seem  
like pretty dumb questions.

I was thinking a raid 1 is probably sufficient.

Would it be best to raid 1 two drives each and LVM them together?

My next question would be about how to do this as I have never done a  
linux software raid.

D

For controller, what is the interface on your drives??  SCSI, SAS??

John Plemons



dnk wrote:
> Hi there,
>
> I am currently setting up a server that will house my backups (simply  
> using rsync).
>
> This system has 4 X 500 gb drives and I am looking to raid for max  
> drive space and data safety. Performance is not so much a concern.
>
> My experience with software raids in nil, so some of these may seem  
> like pretty dumb questions.
>
> I was thinking a raid 1 is probably sufficient.
>
> Would it be best to raid 1 two drives each and LVM them together?
>
> My next question would be about how to do this as I have never done a  
> linux software raid.
>
> D
>
>
>
>
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>
>
>

On Tue, Feb 17, 2009 at 01:52:52PM -0800, dnk wrote:
> Hi there,
> 
> I am currently setting up a server that will house my backups (simply  
> using rsync).
> 
> This system has 4 X 500 gb drives and I am looking to raid for max  
> drive space and data safety. Performance is not so much a concern.
> 
> My experience with software raids in nil, so some of these may seem  
> like pretty dumb questions.
> 
> I was thinking a raid 1 is probably sufficient.
> 
> Would it be best to raid 1 two drives each and LVM them together?
This probably gives you both the best reliability and the best
performance (RAID 10).  You can potentially lose two drives (although
they''d need to be on opposite mirrored subsets) and be OK.

If you want to maximize space, RAID5 would work as well.  You''d get
nearly 1500GB out of this setup but could only survive one disk
failure.

Your other option would be RAID6 which would net you the same amount of
space as RAID10 with four drives and also allow for two drive failures
(any of the drives in this case).  If you anticipate adding additional
drives later, this would allow you to add one drive at a time instead
of requiring a pair.

RAID6 has a write performance penalty, but I imagine it would still
run well enough to handle backups.

I think I might lean towards the RAID6 option in this case.... same
space as RAID10, but allows you to lose *any* two drives and still
maintain functionality at the expense of some write performance.
> 
> My next question would be about how to do this as I have never done a  
> linux software raid.
> 
> D
> 
Ray

dnk wrote:
> Hi there,
>
> I am currently setting up a server that will house my backups (simply  
> using rsync).
>
> This system has 4 X 500 gb drives and I am looking to raid for max  
> drive space and data safety. Performance is not so much a concern.
>
> My experience with software raids in nil, so some of these may seem  
> like pretty dumb questions.
>
> I was thinking a raid 1 is probably sufficient.
>
> Would it be best to raid 1 two drives each and LVM them together?
>
> My next question would be about how to do this as I have never done a  
> linux software raid.
>
> D
>   
Linux software raid works fine and I use this recipe
http://linux-raid.osdl.org/index.php/Partitioning_RAID_/_LVM_on_RAID
If you can afford a proper hardware raid controller for raid 6, that 
would be of better performance than linux software raid, other than that 
I have found performance of linux based md using raid 1 to work
fine.
>
>
>
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>   
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rkampen.vcf
Type: text/x-vcard
Size: 121 bytes
Desc: not available
Url :
http://lists.centos.org/pipermail/centos/attachments/20090217/f66c8b3f/attachment.vcf

On 17-Feb-09, at 2:01 PM, John Plemons wrote:
> For controller, what is the interface on your drives??  SCSI, SAS??
>
> John Plemons
>
Dell 2950, SAS 6 Host Bus Controller.

d

On 17-Feb-09, at 2:08 PM, Rob Kampen wrote:
> Linux software raid works fine and I use this recipe
> http://linux-raid.osdl.org/index.php/Partitioning_RAID_/_LVM_on_RAID
> If you can afford a proper hardware raid controller for raid 6, that  
> would be of better performance than linux software raid, other than  
> that I have found performance of linux based md using raid 1 to work  
> fine.
Rob,

So are you using 4 drives, and making two different raid 1 (2 drives  
each), then joining them via LVM?

d



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090217/a223a11a/attachment.html

Dnk,
I use two drives with linux raid 1 sets for the OS for all my CentOS 
machines, drives are cheaper than my rebuild time and hassle.
I actually use 3 partitions mirrored on each: /boot of 100M; swap of 2 
time RAM (disk is cheap);  70G as /, then the remainder is extended 
partition for lvm - in this case the extended partition is not mirrored 
as the contents are for mythTV recordings that I don''t mind loosing. I 
have a HD TV card that takes 5G per hour of recording so space matters 
more than redundancy - also use xfs on this lvm part as it seems to work 
better for huge files.
In your case I would set up both the raid 1 sets, put lvm volumes in 
place and allocate as needed for the mount points you want to have. 
Because you are using lvm, if you get the allocation wrong it can be 
remedied without too much hassle.
Hope this helps.
Rob

dnk wrote:
>
> On 17-Feb-09, at 2:08 PM, Rob Kampen wrote:
>
>> Linux software raid works fine and I use this recipe
>> http://linux-raid.osdl.org/index.php/Partitioning_RAID_/_LVM_on_RAID
>> If you can afford a proper hardware raid controller for raid 6, that 
>> would be of better performance than linux software raid, other than 
>> that I have found performance of linux based md using raid 1 to work 
>> fine.
>
> Rob,
>
> So are you using 4 drives, and making two different raid 1 (2 drives 
> each), then joining them via LVM?
>
> d
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090217/e934bcdd/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rkampen.vcf
Type: text/x-vcard
Size: 125 bytes
Desc: not available
Url :
http://lists.centos.org/pipermail/centos/attachments/20090217/e934bcdd/attachment.vcf

> Would it be best to raid 1 two drives each and LVM them together?
>
> My next question would be about how to do this as I have never done a  
> linux software raid.
>   
I would do it this way if they are not system disks:

Eg:
sdc + sdd = md0 (raid 1)
sde + sdf = md1 (raid 1)
md0 + md1 = md2 (raid 0)


Use md2 as a physical volume

> > For controller, what is the interface on your drives??  SCSI, SAS??
> Dell 2950, SAS 6 Host Bus Controller.
Integrated SAS 6/i(base): 4 port SAS controller (does support RAID 0/1)
But I don''t know if that is descent hw raid or crap raid...

JD

On 18-Feb-09, at 2:01 AM, John Doe wrote:
>>> For controller, what is the interface on your drives??  SCSI, SAS??
>> Dell 2950, SAS 6 Host Bus Controller.
>
> Integrated SAS 6/i(base): 4 port SAS controller (does support RAID  
> 0/1)
> But I don''t know if that is descent hw raid or crap raid...
>
> JD
This was kind of the reason I was thinking software raid.

Has anyone had any raid experience with this card?

d

On Wed, Feb 18, 2009 at 08:13:23AM -0800, dnk wrote:
> 
> On 18-Feb-09, at 2:01 AM, John Doe wrote:
> 
> >>> For controller, what is the interface on your drives??  SCSI,
SAS??
> >> Dell 2950, SAS 6 Host Bus Controller.
> >
> > Integrated SAS 6/i(base): 4 port SAS controller (does support RAID  
> > 0/1)
> > But I don''t know if that is descent hw raid or crap raid...
> >
> > JD
> 
> This was kind of the reason I was thinking software raid.
> 
> Has anyone had any raid experience with this card?
> 
> d
So this isn''t the PERC then?  The PERC should be real hardware RAID...

Ray

On 18-Feb-09, at 8:17 AM, Ray Van Dolson wrote:
> So this isn''t the PERC then?  The PERC should be real hardware
RAID...
>
> Ray
It is the SAS 6/i.

d

on 2-17-2009 1:52 PM dnk spake the following:
> Hi there,
> 
> I am currently setting up a server that will house my backups (simply  
> using rsync).
> 
> This system has 4 X 500 gb drives and I am looking to raid for max  
> drive space and data safety. Performance is not so much a concern.
Max size would be raid 5, max safety would be raid 10 or lvm over raid
1''s
> 
> My experience with software raids in nil, so some of these may seem  
> like pretty dumb questions.
> 
> I was thinking a raid 1 is probably sufficient.
> 
> Would it be best to raid 1 two drives each and LVM them together?
> 
> My next question would be about how to do this as I have never done a  
> linux software raid.
http://tldp.org/HOWTO/Software-RAID-HOWTO.html

http://www.linuxdevcenter.com/pub/a/linux/2002/12/05/RAID.html

> 
> D

-- 
MailScanner is like deodorant...
You hope everybody uses it, and
you notice quickly if they don''t!!!!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url :
http://lists.centos.org/pipermail/centos/attachments/20090218/81f323a5/attachment.bin

On 18-Feb-09, at 9:14 AM, Scott Silva wrote:
> http://tldp.org/HOWTO/Software-RAID-HOWTO.html
>
> http://www.linuxdevcenter.com/pub/a/linux/2002/12/05/RAID.html
If i am to understand the tutorials right, does one create the raid/ 
lvm after install? Or do you boot off the disk, use these tools, then  
install on existing structures. I was trying to do it through the disk  
druid during install.

d



-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090218/166316b1/attachment.html

on 2-18-2009 10:13 AM dnk spake the following:
> 
> On 18-Feb-09, at 9:14 AM, Scott Silva wrote:
> 
>> http://tldp.org/HOWTO/Software-RAID-HOWTO.html
>>
>> http://www.linuxdevcenter.com/pub/a/linux/2002/12/05/RAID.html
> 
> If i am to understand the tutorials right, does one create the raid/lvm
> after install? Or do you boot off the disk, use these tools, then
> install on existing structures. I was trying to do it through the disk
> druid during install.
> 
> d
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
You can make LVM over raid 1''s in Disk Druid, but I don''t
think it will do
raid 10. And you cannot boot from software raid 5 (yet).


-- 
MailScanner is like deodorant...
You hope everybody uses it, and
you notice quickly if they don''t!!!!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url :
http://lists.centos.org/pipermail/centos/attachments/20090218/37c3f8c9/attachment.bin

Scott Silva wrote:
> You can make LVM over raid 1''s in Disk Druid, but I don''t
think it will do
> raid 10. And you cannot boot from software raid 5 (yet).
>   
a LVM over several raid 1''s is effectively raid10 as LVM will stripe
the
volumes across the devices.   It would be nice if  LVM could do 
mirorring too (like LVM on AIX does) and was tighter integrated with the 
file system tools (again, like LVM on AIX...  grow a LV and it grows the 
JFS thats sitting on it, transparently and online).

on 2-18-2009 11:12 AM John R Pierce spake the following:
> Scott Silva wrote:
>> You can make LVM over raid 1''s in Disk Druid, but I
don''t think it will do
>> raid 10. And you cannot boot from software raid 5 (yet).
>>   
> 
> a LVM over several raid 1''s is effectively raid10 as LVM will
stripe the
> volumes across the devices.   It would be nice if  LVM could do 
> mirorring too (like LVM on AIX does) and was tighter integrated with the 
> file system tools (again, like LVM on AIX...  grow a LV and it grows the 
> JFS thats sitting on it, transparently and online).
Effectively and actually are two different things. Some people seem to be very
shy of LVM, even though it seems to be fairly mature technology. I
didn''t
think the LVM will stripe over the raid 1''s by itself, I thought you
had to
set stripes according to the underlying raid structure to get optimum speed.


-- 
MailScanner is like deodorant...
You hope everybody uses it, and
you notice quickly if they don''t!!!!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 258 bytes
Desc: OpenPGP digital signature
Url :
http://lists.centos.org/pipermail/centos/attachments/20090218/4d4be79d/attachment.bin

dnk wrote:
> Hi there,
> 
> I am currently setting up a server that will house my backups (simply  
> using rsync).
> 
> This system has 4 X 500 gb drives and I am looking to raid for max  
> drive space and data safety. Performance is not so much a concern.
> 
> My experience with software raids in nil, so some of these may seem  
> like pretty dumb questions.
> 
> I was thinking a raid 1 is probably sufficient.
> 
> Would it be best to raid 1 two drives each and LVM them together?
The configuration you want is a hybrid of RAID1 and RAID5. The RAID1 is 
because GRUB doesn''t grok RAID5, but is OK with just your /boot 
partition in RAID1.

Make a RAID1 for your /boot partition as follows:

/dev/md0 - /dev/sda1, /dev/sdb1, /dev/sdc1 (S), /dev/sdd1 (S) (100 Mbytes)


Make a RAID5 as follows for a LVM partition using the rest of your 
available space as follows (just under 1500 Mbytes):

/dev/md1 - /dev/sda2, /dev/sdb2, /dev/sdc2, /dev/sdd2

Create a LVM partition on /dev/md1 and carve out your / and swap 
partitions from it.
> My next question would be about how to do this as I have never done a  
> linux software raid.
-- 
Benjamin Franz

Benjamin Franz wrote:
> Make a RAID5 as follows for a LVM partition using the rest of your 
> available space as follows (just under 1500 Mbytes):
Sorry. Typo. Just under 1500 Gbytes.

-- 
Benjamin Franz

On Wed, Feb 18, 2009 at 11:12:13AM -0800, John R Pierce
wrote:
> Scott Silva wrote:
> > You can make LVM over raid 1''s in Disk Druid, but I
don''t think it will do
> > raid 10. And you cannot boot from software raid 5 (yet).
> >   
> 
> a LVM over several raid 1''s is effectively raid10 as LVM will
stripe the
> volumes across the devices.   It would be nice if  LVM could do 
> mirorring too (like LVM on AIX does) and was tighter integrated with the 
> file system tools (again, like LVM on AIX...  grow a LV and it grows the 
> JFS thats sitting on it, transparently and online).
> 
Can''t Linux LVM do mirroring?  I swear I read that it could in the man
page.  Never have tried it however and you certainly can''t set it up
from disk druid in anaconda.

Ray

On Wed, 2009-02-18 at 08:13 -0800, dnk wrote:
> On 18-Feb-09, at 2:01 AM, John Doe wrote:
> 
> >>> For controller, what is the interface on your drives??  SCSI,
SAS??
> >> Dell 2950, SAS 6 Host Bus Controller.
> >
> > Integrated SAS 6/i(base): 4 port SAS controller (does support RAID  
> > 0/1)
> > But I don''t know if that is descent hw raid or crap raid...
> >
> > JD
> 
> This was kind of the reason I was thinking software raid.
> 
> Has anyone had any raid experience with this card?
Yep - it''s real HW raid, though with a 2950, I would have gone for the
PERC instead.  The integrated SAS is usually a Fusion-MPT (LSI/Symbios)
card.  Do a ''lspci'' to be sure.  I''ve got a few of
these configured and
I can check the raid status with the mpt-status command (from the
mpt-status rpm).  Note that doing a ''fdisk -l'' only yields one
disk when
RAID is setup.  So yes - it''s real HW raid.  Just not much in the way
of
cache, which is why I prefer using PERCs instead...

	-I

Ray Van Dolson wrote:
> Can''t Linux LVM do mirroring?  I swear I read that it could in the
man
> page.  Never have tried it however and you certainly can''t set it
up
> from disk druid in anaconda.
>   

dunno.  the word ''mirror occurs exactly once in the man page for lvm(8)

        lvconvert -- Convert a logical volume from linear to mirror or 
snapshot.


the man page on lvconvert has more info, and this 
http://www.centos.org/docs/5/html/5.2/Cluster_Logical_Volume_Manager/mirrored_volumes.html
certainly implies CentOS5 supports it.

glad to know thats finally there.  however, it doesn''t seem to support 
little things like hot spares or automatic rebuilding.   in fact, the 
rebuild process looks involved and convoluted...   
http://www.centos.org/docs/5/html/5.2/Cluster_Logical_Volume_Manager/mirrorrecover.html

On Wed, Feb 18, 2009 at 11:51:55AM -0800, John R Pierce
wrote:
> Ray Van Dolson wrote:
> > Can''t Linux LVM do mirroring?  I swear I read that it could
in the man
> > page.  Never have tried it however and you certainly can''t
set it up
> > from disk druid in anaconda.
> >   
> 
> 
> dunno.  the word ''mirror occurs exactly once in the man page for
lvm(8)
> 
>         lvconvert -- Convert a logical volume from linear to mirror or 
> snapshot.
> 
> 
> the man page on lvconvert has more info, and this 
>
http://www.centos.org/docs/5/html/5.2/Cluster_Logical_Volume_Manager/mirrored_volumes.html
> certainly implies CentOS5 supports it.
> 
> glad to know thats finally there.  however, it doesn''t seem to
support
> little things like hot spares or automatic rebuilding.   in fact, the 
> rebuild process looks involved and convoluted...   
>
http://www.centos.org/docs/5/html/5.2/Cluster_Logical_Volume_Manager/mirrorrecover.html
> 
The other side of the coin (as I think you mentioned) is that many are
not comfortable having LVM handle the mirroring.  Are its mirroring
abilities as mature or fast as md?  It''s certainly not documented as
well at the very least. :)

For now, I prefer doing my mirroring with md and the striping with LVM.
Would be interesting to hear anyone else''s experiences with LVM
mirroring however.

Ray

John R Pierce wrote:
> Ray Van Dolson wrote:
>> Can''t Linux LVM do mirroring?  I swear I read that it could in
the man
>> page.  Never have tried it however and you certainly can''t set
it up
>> from disk druid in anaconda.
>>   
> 
> 
> dunno.  the word ''mirror occurs exactly once in the man page for
lvm(8)
> 
>         lvconvert -- Convert a logical volume from linear to mirror or 
> snapshot.
> 
> 

It''s in the lvcreate manpage . . .


LVM mirroring seems rather quirky to me - I''d rather use md raid1 sets 
as phy vols.


-- 
tkb

On Feb 18, 2009, at 2:29 PM, Ray Van Dolson <rayvd at bludgeon.org> wrote:
> On Wed, Feb 18, 2009 at 11:12:13AM -0800, John R Pierce wrote:
>> Scott Silva wrote:
>>> You can make LVM over raid 1''s in Disk Druid, but I
don''t think it
>>> will do
>>> raid 10. And you cannot boot from software raid 5 (yet).
>>>
>>
>> a LVM over several raid 1''s is effectively raid10 as LVM will
>> stripe the
>> volumes across the devices.   It would be nice if  LVM could do
>> mirorring too (like LVM on AIX does) and was tighter integrated  
>> with the
>> file system tools (again, like LVM on AIX...  grow a LV and it  
>> grows the
>> JFS thats sitting on it, transparently and online).
>>
>
> Can''t Linux LVM do mirroring?  I swear I read that it could in the
man
> page.  Never have tried it however and you certainly can''t set it
up
> from disk druid in anaconda.
Lvm can do mirroring, but of requires a third drive to do mirror  
logging which  kind of defeats the whole raid1 concept, I guess if you  
have a VG of a plethora of drives this is no biggie.

And besides grub still after all these years does not support booting  
LVM volumes.

Jeez, you think with the Solaris folk able to boot off of ZFS now from  
grub we''d have  the ability to boot off of LVM by now!

-Ross

> a LVM over several raid 1''s is effectively raid10 as LVM will
stripe the
> volumes across the devices.   It would be nice if  LVM could do 
> mirorring too (like LVM on AIX does) and was tighter integrated with the 
> file system tools (again, like LVM on AIX...  grow a LV and it grows the 
> JFS thats sitting on it, transparently and online).
>
>   
You heretic! Why are you asking for ZFS?


(btw the raid10 module is relatively new and is different from doing 
raid1+raid0 and is not supported by anaconda unlike raid1+0)

On Wed, Feb 18, 2009 at 06:20:59PM -0500, Ross Walker
wrote:
> On Feb 18, 2009, at 2:29 PM, Ray Van Dolson <rayvd at bludgeon.org>
wrote:
> 
> > On Wed, Feb 18, 2009 at 11:12:13AM -0800, John R Pierce wrote:
> >> Scott Silva wrote:
> >>> You can make LVM over raid 1''s in Disk Druid, but I
don''t think it
> >>> will do
> >>> raid 10. And you cannot boot from software raid 5 (yet).
> >>>
> >>
> >> a LVM over several raid 1''s is effectively raid10 as LVM
will
> >> stripe the
> >> volumes across the devices.   It would be nice if  LVM could do
> >> mirorring too (like LVM on AIX does) and was tighter integrated  
> >> with the
> >> file system tools (again, like LVM on AIX...  grow a LV and it  
> >> grows the
> >> JFS thats sitting on it, transparently and online).
> >>
> >
> > Can''t Linux LVM do mirroring?  I swear I read that it could
in the man
> > page.  Never have tried it however and you certainly can''t
set it up
> > from disk druid in anaconda.
> 
> Lvm can do mirroring, but of requires a third drive to do mirror  
> logging which  kind of defeats the whole raid1 concept, I guess if you  
> have a VG of a plethora of drives this is no biggie.
> 
> And besides grub still after all these years does not support booting  
> LVM volumes.
> 
> Jeez, you think with the Solaris folk able to boot off of ZFS now from  
> grub we''d have  the ability to boot off of LVM by now!
Hehe, I guess the fact that we can achieve fairly similar results using
md + LVM has made it a lower priority. :-/

Ray

> Subject: Re: [CentOS] 4 X 500 gb drives - best software raid 
> config for abackup server?
> > Hi there,
> > I am currently setting up a server that will house my 
> backups (simply 
> > using rsync).
> > 
> > This system has 4 X 500 gb drives and I am looking to raid for max 
> > drive space and data safety. Performance is not so much a concern.
> > 
> > My experience with software raids in nil, so some of these may seem 
> > like pretty dumb questions.
> > 
> > I was thinking a raid 1 is probably sufficient.
> > 
> > Would it be best to raid 1 two drives each and LVM them together?
> > 
> > My next question would be about how to do this as I have 
> never done a 
> > linux software raid.
You are in luck. I never did one before when I started too. I did a step by
step procedure that covers adding and removing drives... For centos 5.x,
with 4 drives.
http://www.bobhoffman.com/wordpress/?page_id=44

This will take 2 drives into a raid 1, with LVM and all that.
IT adds a third drive as a hot spare incase one fails.
It adds a fourth as a standalone backup (good for a rync of the raid)

Just remember to wait for the drives to sync up before doing too much.

This is a step by step for newbs like me and is perfectly tailored for what
you need.
Let me know if you have any issues.

On Thu, Feb 19, 2009 at 4:22 AM, Ray Van Dolson <rayvd at bludgeon.org>
wrote:
> The other side of the coin (as I think you mentioned) is that many are
> not comfortable having LVM handle the mirroring.  Are its mirroring
> abilities as mature or fast as md?  It''s certainly not documented
as
> well at the very least. :)
>
I remember googling for this before setting up a server some weeks ago and
somebody did a benchmark. The general conclusion was stick to md for RAID 1,
it has better performance. IIRC, one of the reason was while md1 will read
from both disk, LVM mirror apparently only reads from the "master"
unless it
fails.

Furthermore, given the nightmare of a time I''m having trying to restore
a
LVM PV sitting across 3 pairs of md RAID 1, I''ll strongly recommend
against
tempting fate by using LVM for mirroring as well.

Thankfully for the underlying md mirror, I can at least activate the LVM vg
and offload data in rescue mode even if it won''t work off a normal
boot.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090220/5d8db555/attachment.html

Noob Centos Admin schrieb:
> 
> 
> On Thu, Feb 19, 2009 at 4:22 AM, Ray Van Dolson <rayvd at bludgeon.org 
> <mailto:rayvd at bludgeon.org>> wrote:
> 
>     The other side of the coin (as I think you mentioned) is that many are
>     not comfortable having LVM handle the mirroring.  Are its mirroring
>     abilities as mature or fast as md?  It''s certainly not
documented as
>     well at the very least. :)
> 
> 
> I remember googling for this before setting up a server some weeks ago 
> and somebody did a benchmark. The general conclusion was stick to md for 
> RAID 1, it has better performance. IIRC, one of the reason was while md1 
> will read from both disk, LVM mirror apparently only reads from the 
> "master" unless it fails.
"md1 will read from both disk" is not true in general.
RAID1 md reads from one disk only; it uses the other one in case the 
first one fails. No performance gain from multiple copies.

You can easily see this for yourself by setting up a RAID1 from e.g. 
sda1 and sdb1 - /proc/mdstat is:

Personalities : [raid1]
md1 : active raid1 sdb1[1] sda1[0]
       104320 blocks [2/2] [UU]

and then comparing the output of hdparm -tT :

/dev/sda1:
  Timing cached reads:   29368 MB in  2.00 seconds = 14711.93 MB/sec
  Timing buffered disk reads:  100 MB in  0.92 seconds = 108.79 MB/sec

/dev/md1:
  Timing cached reads:   28000 MB in  2.00 seconds = 14023.66 MB/sec
  Timing buffered disk reads:  100 MB in  0.95 seconds = 105.81 MB/sec

/dev/sdb1:
  Timing cached reads:   23780 MB in  2.00 seconds = 11907.30 MB/sec
  Timing buffered disk reads:  100 MB in  0.98 seconds = 102.51 MB/sec


To get performance gain in RAID1 mode you need hardware RAID1.

HTH,

Kay

> "md1 will read from both disk" is not true in general.
> RAID1 md reads from one disk only; it uses the other one in case the 
> first one fails. No performance gain from multiple copies.
>   
I beg to differ. I have disks in a raid1 md array and iostat -x 1 will 
show reads coming off both disks. Unless you do not have the multipath 
module loaded, md will read off both disks. Now whether md will read 
equally off both disks, that certainly will not be true in
general.
> You can easily see this for yourself by setting up a RAID1 from e.g. 
> sda1 and sdb1 - /proc/mdstat is:
>
> Personalities : [raid1]
> md1 : active raid1 sdb1[1] sda1[0]
>        104320 blocks [2/2] [UU]
>
> and then comparing the output of hdparm -tT :
>   
ROTFL.

How about using the proper tool (iostat) and generating some disk load 
instead?
> To get performance gain in RAID1 mode you need hardware RAID1.
>   
Bollocks. The only area in which hardware raid has a significant 
performance advantage over software raid is raid5/6 given sufficient 
cache memory and processing power.

On Fri, 2009-02-20 at 22:52 +0800, Chan Chung Hang Christopher
wrote:
> Bollocks. The only area in which hardware raid has a significant 
> performance advantage over software raid is raid5/6 given sufficient 
> cache memory and processing power.
I''d have to say no on the processing power for RAID 5. 
Moore''s law has
grown CPU capabilities over the last 15 or so years.  HW RAID
controllers haven''t gotten that much faster because they
haven''t needed
to.  It''s faster to do it in software, though it''s preferable
to offload
it to HW RAID so that any apps aren''t affected directly.

I would agree on that cache memory is an advantage, especially when
considering battery-backed cache memory.

But those aren''t the only significant areas.  HW RAID allows for
hot-swap and pain-free (meaning zero commands needed) disk replacement.

	-I

Ian Forde wrote:
> On Fri, 2009-02-20 at 22:52 +0800, Chan Chung Hang Christopher wrote:
>   
>> Bollocks. The only area in which hardware raid has a significant 
>> performance advantage over software raid is raid5/6 given sufficient 
>> cache memory and processing power.
>>     
>
> I''d have to say no on the processing power for RAID 5. 
Moore''s law has
> grown CPU capabilities over the last 15 or so years.  HW RAID
> controllers haven''t gotten that much faster because they
haven''t needed
> to.  It''s faster to do it in software, though it''s
preferable to offload
> it to HW RAID so that any apps aren''t affected directly.
>   
You will have to prove that. I have previously posted posts with links 
to benchmarks that show that hardware raid with sufficient processing 
power beat the pants of software raid when it comes to raid5/6 
implementations. Hardware raid cards no longer come with crappy i960
cpus.
> I would agree on that cache memory is an advantage, especially when
> considering battery-backed cache memory.
>   
There is more to it. That cache memory also cuts down on bus traffic but 
the real kicker is that there is no bus contention between the board''s 
cpu and disk data whereas software raid needs to read of the disks for 
its calculations and therefore suffers latencies that hardware raid 
boards (which have direct connections to disks) do not. Of course, if 
the cache size is insufficient, then the hardware raid board will not 
perform much better if not worse than software raid.
> But those aren''t the only significant areas.  HW RAID allows for
> hot-swap and pain-free (meaning zero commands needed) disk replacement.
>   
Hmm...really? I guess it depends on the board. (okay, okay, thinking of 
antique 3ware 750x series may not be fair)

Chan Chung Hang Christopher schrieb:
>> "md1 will read from both disk" is not true in general.
>> RAID1 md reads from one disk only; it uses the other one in case the 
>> first one fails. No performance gain from multiple copies.
>>   
> I beg to differ. I have disks in a raid1 md array and iostat -x 1 will 
> show reads coming off both disks. Unless you do not have the multipath 
look more carefully - with the current 2.6.18-9.1.22 kernel the bulk of 
the data are read from one of the disks
> module loaded, md will read off both disks. Now whether md will read 
> equally off both disks, that certainly will not be true in general.
>> You can easily see this for yourself by setting up a RAID1 from e.g. 
>> sda1 and sdb1 - /proc/mdstat is:
>>
>> Personalities : [raid1]
>> md1 : active raid1 sdb1[1] sda1[0]
>>        104320 blocks [2/2] [UU]
>>
>> and then comparing the output of hdparm -tT :
>>   
> ROTFL.
> 
> How about using the proper tool (iostat) and generating some disk load 
> instead?
hdparm -tT tests one type of disk access, other tools test other 
aspects. I gave the hdparm numbers because everyone can reproduce them. 
For RAID0 with two disks you do see - using e.g. hdparm - the doubling 
of performance from two disks.
If you take the time to read (or do) RAID benchmarks you''ll discover 
that Linux software RAID1 is about as fast as a single disk (and RAID0 
with two disks is about twice the speed). It''s as simple as that.
> 
>> To get performance gain in RAID1 mode you need hardware RAID1.
>>   
> 
> Bollocks. The only area in which hardware raid has a significant 
> performance advantage over software raid is raid5/6 given sufficient 
> cache memory and processing power.
We were talking about RAID1; RAID5/6 is a different area. Linux software 
RAID1 is a safeguard against disk failure; it''s not designed for speed 
increase. There is a number of things that could be improved in Linux 
software RAID; read performance of RAID1 is one of them - this _is_ why 
some hardware RAID1 adapters indeed are faster than software.
Read http://kernelnewbies.org/KernelProjects/Raid1ReadBalancing - since 
the 2.6.25 kernel a simple alternating read is implemented, but that 
does not take the access pattern into account.

So Linux software RAID1 is just mirroring - and it''s good at that.

HTH,

Kay

Kay Diederichs wrote:
> hdparm -tT tests one type of disk access, other tools test other 
> aspects. I gave the hdparm numbers because everyone can reproduce them. 
> For RAID0 with two disks you do see - using e.g. hdparm - the doubling 
> of performance from two disks.
> If you take the time to read (or do) RAID benchmarks you''ll
discover
> that Linux software RAID1 is about as fast as a single disk (and RAID0 
> with two disks is about twice the speed). It''s as simple as that.
>   

maybe with a simple single threaded application.  if there are 
concurrent read requests pending it will dispatch them to both drives.

On Sat, Feb 21, 2009 at 6:04 PM, John R Pierce <pierce at hogranch.com>
wrote:
> Kay Diederichs wrote:
> > hdparm -tT tests one type of disk access, other tools test other
> > aspects. I gave the hdparm numbers because everyone can reproduce
them.
> > For RAID0 with two disks you do see - using e.g. hdparm - the doubling
> > of performance from two disks.
> > If you take the time to read (or do) RAID benchmarks you''ll
discover
> > that Linux software RAID1 is about as fast as a single disk (and RAID0
> > with two disks is about twice the speed). It''s as simple as
that.
> >
>
>
> maybe with a simple single threaded application.  if there are
> concurrent read requests pending it will dispatch them to both drives.

I''m waiting for a 10 hour backup to be completed before doing recovery
on a
server (ok recovery is a nice way to put it, truth is I gave up any hope of
making the screwed LVM setup work and going to wipe/reinstall after the
backup), I''ll probably be able to try some tests.

However, I don''t know enough to do this properly. So some questions:

Would running two CP command to copy 2 different set of files to two
different targets suffice as a basic two thread test?

Is there a way to monitor actual disk transfers from command line without
having to do manual timing?
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090221/cabce334/attachment.html

> Would running two CP command to copy 2 different set of files to two
> different targets suffice as a basic two thread test?
>   
So long as you generate disk access through a file system and not
hdparm.
> Is there a way to monitor actual disk transfers from command line without
> having to do manual timing?
Like I said: iostat

iostat -m 1
iostat -x 1
iostat -d 1
iostat -xm 1

Choose whatever information/data you fancy.


In fact, let me just post some iostat output to settle this once and for 
all. I shall risk the annoyance of all by attaching the output of a 
''iostat -m sda sdb md1 1'' command.

This was generated by a single Linux client connected via cifs do a cp 
-r off the cifs remote share to local disk. The output was generated on 
the remote share box.

 From the output, I guess one loses grounds for claiming that md raid1 
only reads off one disk even if there are periods of time when it really 
does only read off one disk.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: md_does_multipath.bz2
Type: application/x-bzip
Size: 2558 bytes
Desc: not available
Url :
http://lists.centos.org/pipermail/centos/attachments/20090221/22102ba0/attachment.bin

Kay Diederichs wrote:
> Chan Chung Hang Christopher schrieb:
>   
>>> "md1 will read from both disk" is not true in general.
>>> RAID1 md reads from one disk only; it uses the other one in case
the
>>> first one fails. No performance gain from multiple copies.
>>>   
>>>       
>> I beg to differ. I have disks in a raid1 md array and iostat -x 1 will 
>> show reads coming off both disks. Unless you do not have the multipath 
>>     
>
> look more carefully - with the current 2.6.18-9.1.22 kernel the bulk of 
> the data are read from one of the disks
>
>   
Hmm...right now I do not have a Centos 5 box handy. Come on you chums 
who have blasted me before about multipath. Prove him wrong with data 
please. I can only pull evidence off a Hardy box.
>> module loaded, md will read off both disks. Now whether md will read 
>> equally off both disks, that certainly will not be true in general.
>>     
>>> You can easily see this for yourself by setting up a RAID1 from
e.g.
>>> sda1 and sdb1 - /proc/mdstat is:
>>>
>>> Personalities : [raid1]
>>> md1 : active raid1 sdb1[1] sda1[0]
>>>        104320 blocks [2/2] [UU]
>>>
>>> and then comparing the output of hdparm -tT :
>>>   
>>>       
>> ROTFL.
>>
>> How about using the proper tool (iostat) and generating some disk load 
>> instead?
>>     
>
> hdparm -tT tests one type of disk access, other tools test other 
> aspects. I gave the hdparm numbers because everyone can reproduce them. 
> For RAID0 with two disks you do see - using e.g. hdparm - the doubling 
> of performance from two disks.
> If you take the time to read (or do) RAID benchmarks you''ll
discover
> that Linux software RAID1 is about as fast as a single disk (and RAID0 
> with two disks is about twice the speed). It''s as simple as that.
>
>   
I beg to differ again since I did get combined throughput from a md 
raid1 device. I would have saved them iostat output to disk if I had 
known they would have some use. Anyway, I have got some numbers in my 
other post but on an Ubuntu box.
>>> To get performance gain in RAID1 mode you need hardware RAID1.
>>>   
>>>       
>> Bollocks. The only area in which hardware raid has a significant 
>> performance advantage over software raid is raid5/6 given sufficient 
>> cache memory and processing power.
>>     
>
> We were talking about RAID1; RAID5/6 is a different area. Linux software 
> RAID1 is a safeguard against disk failure; it''s not designed for
speed
> increase. There is a number of things that could be improved in Linux 
> software RAID; read performance of RAID1 is one of them - this _is_ why 
> some hardware RAID1 adapters indeed are faster than software.
> Read http://kernelnewbies.org/KernelProjects/Raid1ReadBalancing - since 
> the 2.6.25 kernel a simple alternating read is implemented, but that 
> does not take the access pattern into account.
>   
I have not read that yet but that is odd since I have been blasted by 
others before for doubting md raid1 doing multiple disk reads.

BTW, the Hardy box''s kernel is 2.6.24-22-generic. I guess I need to try
to generate some from an Intrepid box and see if I get better
numbers.
> So Linux software RAID1 is just mirroring - and it''s good at that.
>
>   
It has gotten good...no more having to sync from the beginning to end I 
believe...just like some hardware raid cards.

On Sat, Feb 21, 2009 at 11:42 PM, Chan Chung Hang Christopher <
christopher.chan at bradbury.edu.hk> wrote:
>
>  Would running two CP command to copy 2 different set of files to two
>> different targets suffice as a basic two thread test?
>>
>>
> So long as you generate disk access through a file system and not hdparm.
>
>> Is there a way to monitor actual disk transfers from command line
without
>> having to do manual timing?
>>
> Like I said: iostat
>
> Thanks for the information. I checked iostat on one of my older servers
running off CentOS 5.0 (2.6.18-53.1.21.el5xen) which was also running md
raid 1 and it also confirmed that the md raid 1 was getting reads from both
member devices.

Although looking at it now, I think I really screwed up that installation,
being my first, I had md running on top of LVM PV *slap forehead*
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090222/73112584/attachment.html

Chan Chung Hang Christopher wrote:
>    
>> We were talking about RAID1; RAID5/6 is a different area. Linux
software
>> RAID1 is a safeguard against disk failure; it''s not designed
for speed
>> increase. There is a number of things that could be improved in Linux 
>> software RAID; read performance of RAID1 is one of them - this _is_ why
>> some hardware RAID1 adapters indeed are faster than software.
>> Read http://kernelnewbies.org/KernelProjects/Raid1ReadBalancing - since
>> the 2.6.25 kernel a simple alternating read is implemented, but that 
>> does not take the access pattern into account.
>>   
> I have not read that yet but that is odd since I have been blasted by 
> others before for doubting md raid1 doing multiple disk reads.
I know current centos alternates because I had a box with bad memory 
corrupt a raid1 filesystem and after fixing it and fsck''ing the disk, 
errors would slowly re-appear as they were hit on the alternate disk 
that previous fsck runs had not seen.

To really speed things up, you would want to try to avoid seeks.  A 
simple alternating pattern may let the heads be in different places on 
small reads, but for a large file you will end up doing the same head 
motions on both drives (wasting the same time) as the reads alternate.

-- 
   Les Mikesell
    lesmikesell at gmail.com

On Sat, 2009-02-21 at 08:40 +0800, Chan Chung Hang Christopher
wrote:
> Ian Forde wrote:
> > I''d have to say no on the processing power for RAID 5. 
Moore''s law has
> > grown CPU capabilities over the last 15 or so years.  HW RAID
> > controllers haven''t gotten that much faster because they
haven''t needed
> > to.  It''s faster to do it in software, though it''s
preferable to offload
> > it to HW RAID so that any apps aren''t affected directly.
> >   
> You will have to prove that. I have previously posted posts with links 
> to benchmarks that show that hardware raid with sufficient processing 
> power beat the pants of software raid when it comes to raid5/6 
> implementations. Hardware raid cards no longer come with crappy i960 cpus.
Just by doing some quick googling, I came across:

http://blogs.zdnet.com/storage/?p=126
http://storagemojo.com/2007/04/24/mo-better-zfs-performance-stats/
http://milek.blogspot.com/2007/04/hw-raid-vs-zfs-software-raid-part-iii.html

Now, bear in mind that I''m no ZFS fanboy, but I''m saying that
it''s not
so cut and dry anymore. The equation changes, of course, when we''re
talking about a purposed fileserver versus an application server that
needs RAID.  (The app server can suffer because its losing access to CPU
resources.)  But the point of contention is still there.  Both are
viable solutions, when considering that SW RAID was never a serious
contender for performance over the years, look at where it is now.  This
tells me that it''s trending up towards equaling or bettering HW RAID
performance.  And that''s not talking about price points.  When throwing
that in...

But again - I still like HW RAID.  I think we''re in agreement on this.
> > I would agree on that cache memory is an advantage, especially when
> > considering battery-backed cache memory.  
> There is more to it. That cache memory also cuts down on bus traffic but 
> the real kicker is that there is no bus contention between the
board''s
> cpu and disk data whereas software raid needs to read of the disks for 
> its calculations and therefore suffers latencies that hardware raid 
> boards (which have direct connections to disks) do not. Of course, if 
> the cache size is insufficient, then the hardware raid board will not 
> perform much better if not worse than software raid.
Indeed.
> > But those aren''t the only significant areas.  HW RAID allows
for
> > hot-swap and pain-free (meaning zero commands needed) disk
replacement.
> >   
> 
> Hmm...really? I guess it depends on the board. (okay, okay, thinking of 
> antique 3ware 750x series may not be fair)
I was thinking about when I was running a farm of 500 HP DL-x80 series
boxes and disk replacement became a 9x5 job that we farmed out.  Just
give a list of servers and locations (first drive or second drive) and
the person could pull old drives out, put new drives in, and resync was
automatic.  Same thing is true for Dell PERC hardware.  I note that
that''s not necessarily true with ALL HW RAID controllers, as they have
to support hot-swap, and the chassis has to have hot-swap slots. But
still, I''ve only seen one SW RAID implementation that does auto-sync.
That''s the Infrant ReadyNAS (http://www.readynas.com).  I wonder how
they did it?  Might not be a bad idea to see how they''re able to use
mdadm to detect and autosync drives.  I don''t *ever* want to go through
something like:

http://kev.coolcavemen.com/2008/07/heroic-journey-to-raid-5-data-recovery/

Not when a little planning can help me skip it... ;)

	-I

A good place to start comparing benchmark numbers for different RAID 
levels is
http://linux-raid.osdl.org/index.php/Performance
in particular the links given in section "Other benchmarks from
2007-2008"

HTH,

Kay

Ian Forde wrote:
> Might not be a bad idea to see how they''re able to use
> mdadm to detect and autosync drives.  I don''t *ever* want to go
through
> something like:
> 
> http://kev.coolcavemen.com/2008/07/heroic-journey-to-raid-5-data-recovery/
> 
> Not when a little planning can help me skip it... ;)
If you are really concerned about data recovery and can chunk up your 
filesystem mount points so things fit on a single disk (usually not too 
hard with 1 or 1.5 TB drives available now) just use software raid1 
since you can simply mount any single disk from it and access the files. 
  It becomes much more difficult with other raid levels or multi-disk lvm.

-- 
   Les Mikesell
    lesmikesell at gmail.com

On Sat, 2009-02-21 at 17:24 -0600, Les Mikesell wrote:
> Ian Forde wrote:
> > Might not be a bad idea to see how they''re able to use
> > mdadm to detect and autosync drives.  I don''t *ever* want to
go through
> > something like:
> > 
> >
http://kev.coolcavemen.com/2008/07/heroic-journey-to-raid-5-data-recovery/
> > 
> > Not when a little planning can help me skip it... ;)
> 
> If you are really concerned about data recovery and can chunk up your 
> filesystem mount points so things fit on a single disk (usually not too 
> hard with 1 or 1.5 TB drives available now) just use software raid1 
> since you can simply mount any single disk from it and access the files. 
>   It becomes much more difficult with other raid levels or multi-disk lvm.
My point is that at home, I''d rather do network mounts to a fileserver
utilizing HW RAID.  At work, I''d rather use HW RAID with hot-swap
disks.
This way, there''s are no hoops to go through.  Time is a more important
resource to me... SW RAID is a path that I went down well over a decade
ago in Solaris (DiskSuite and Veritas VM), followed by Linux mdadm.  If
you''ve ever had to do a Veritas encapsulated boot disk recovery,
you''ll
know why I''d rather never go down that road *ever again*... ;)

	-I

	-I

Ian Forde wrote:
> On Sat, 2009-02-21 at 17:24 -0600, Les Mikesell wrote:
>> Ian Forde wrote:
>>> Might not be a bad idea to see how they''re able to use
>>> mdadm to detect and autosync drives.  I don''t *ever* want
to go through
>>> something like:
>>>
>>>
http://kev.coolcavemen.com/2008/07/heroic-journey-to-raid-5-data-recovery/
>>>
>>> Not when a little planning can help me skip it... ;)
>> If you are really concerned about data recovery and can chunk up your 
>> filesystem mount points so things fit on a single disk (usually not too
>> hard with 1 or 1.5 TB drives available now) just use software raid1 
>> since you can simply mount any single disk from it and access the
files.
>>   It becomes much more difficult with other raid levels or multi-disk
lvm.
> 
> My point is that at home, I''d rather do network mounts to a
fileserver
> utilizing HW RAID.  At work, I''d rather use HW RAID with hot-swap
disks.
> This way, there''s are no hoops to go through.  Time is a more
important
> resource to me... SW RAID is a path that I went down well over a decade
> ago in Solaris (DiskSuite and Veritas VM), followed by Linux mdadm.  If
> you''ve ever had to do a Veritas encapsulated boot disk recovery,
you''ll
> know why I''d rather never go down that road *ever again*... ;)
Yes, but raid1 in software has none of those problems, since as far as 
the boot loader is concerned, you are booting from a single drive.  And 
there is a trade-off in complexity, since sw raid works the same on 
Linux across different hardware and you need to round up different 
vendors instructions and utilities for hardware raid - and have a backup 
controller around for recovery.

-- 
   Les Mikesell
    lesmikesell at gmail.com

On Sat, 2009-02-21 at 18:09 -0600, Les Mikesell wrote:
> Yes, but raid1 in software has none of those problems, since as far as 
> the boot loader is concerned, you are booting from a single drive.  And 
> there is a trade-off in complexity, since sw raid works the same on 
> Linux across different hardware and you need to round up different 
> vendors instructions and utilities for hardware raid - and have a backup 
> controller around for recovery.
RAID in software, whether RAID1 or RAID5/6, always has manual steps
involved in recovery.  If one is using standardized hardware, such as HP
DL-x80 hardware or Dell x950 boxes, HW RAID obviates the need for a
"recovery procedure".  It''s just easier.  You can still boot
from a
single drive, since that''s what the bootloader sees.  There are no
vendor instructions or utilities needed for recovery.  Nor is there a
backup controller needed.  The *only* time I''d use software RAID on
Linux is if I didn''t have a standard hardware base that supported
hotswap and commandless recovery, which in any enterprise within which I
were to be employed, I''d insist upon (and deploy)...

	-I

>> You will have to prove that. I have previously posted posts with links 
>> to benchmarks that show that hardware raid with sufficient processing 
>> power beat the pants of software raid when it comes to raid5/6 
>> implementations. Hardware raid cards no longer come with crappy i960
cpus.
>>     
>
> Just by doing some quick googling, I came across:
>
> http://blogs.zdnet.com/storage/?p=126
> http://storagemojo.com/2007/04/24/mo-better-zfs-performance-stats/
>
http://milek.blogspot.com/2007/04/hw-raid-vs-zfs-software-raid-part-iii.html
>   
I have an issue with that benchmark. I don''t know what that EMC is made
of nevermind the fancy FC.


Regarding my posts...they were on the ubuntu-user list...not 
here...so...here are the links:

Thread post: 
https://lists.ubuntu.com/archives/edubuntu-users/2008-December/004887.html

Don''t use underpowered hardward raid boards for raid5/6
http://www.chemistry.wustl.edu/~gelb/castle_raid.html

Use hardware raid boards with enough cpu/cache for raid5/6
http://www.linux.com/feature/140734

Kay Diederichs wrote:
> A good place to start comparing benchmark numbers for different RAID 
> levels is
> http://linux-raid.osdl.org/index.php/Performance
> in particular the links given in section "Other benchmarks from
2007-2008"
>   
I like this bit of info from

http://linux-raid.osdl.org/index.php/Performance

Right under "Fresh benchmarking tools"

"To check out speed and performance of your RAID systems, do NOT use 
hdparm. It won''t do real benchmarking of the arrays."

Ian Forde wrote:
> On Sat, 2009-02-21 at 18:09 -0600, Les Mikesell wrote:
>> Yes, but raid1 in software has none of those problems, since as far as 
>> the boot loader is concerned, you are booting from a single drive.  And
>> there is a trade-off in complexity, since sw raid works the same on 
>> Linux across different hardware and you need to round up different 
>> vendors instructions and utilities for hardware raid - and have a
backup
>> controller around for recovery.
> 
> RAID in software, whether RAID1 or RAID5/6, always has manual steps
> involved in recovery.
Don''t forget that ''recovery'' sometimes means taking
the still-working
drives and moving them to a new chassis.
> If one is using standardized hardware, such as HP
> DL-x80 hardware or Dell x950 boxes, HW RAID obviates the need for a
> "recovery procedure".
As long as you have an exactly-matching chassis/motherboard/controller 
to move to.
> It''s just easier.  You can still boot from a
> single drive, since that''s what the bootloader sees.  There are no
> vendor instructions or utilities needed for recovery.  Nor is there a
> backup controller needed.
Everything breaks eventually.  If yours hasn''t yet, good luck with
that.
> The *only* time I''d use software RAID on
> Linux is if I didn''t have a standard hardware base that supported
> hotswap and commandless recovery, which in any enterprise within which I
> were to be employed, I''d insist upon (and deploy)...
You can have hot spares in software raid if you can''t be bothered to 
type ''mdadm --add ....'' once every few years.

-- 
   Les Mikesell
    lesmikesell at gmail.com

On Sun, Feb 22, 2009 at 7:05 PM, Ian Forde <ian at duckland.org> wrote:
> RAID in software, whether RAID1 or RAID5/6, always has manual steps
> involved in recovery.  If one is using standardized hardware, such as HP
> DL-x80 hardware or Dell x950 boxes, HW RAID obviates the need for a
> "recovery procedure".  It''s just easier.  You can still
boot from a
> single drive, since that''s what the bootloader sees.  There are no
> vendor instructions or utilities needed for recovery.  Nor is there a
> backup controller needed.
>
If I have to do hardware raid, I''ll definitely spec in a backup
controller.
Learnt this the hard way when my raid 5 controller died years after I first
got it and I could no longer find a replacement.

For high budget projects, having the extra raid controller as insurance
isn''t a big deal. But for most budget setup and cost conscious clients,
soft
raid obviates that hardware dependency.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.centos.org/pipermail/centos/attachments/20090223/43c27fae/attachment.html

>
> If I have to do hardware raid, I''ll definitely spec in a backup 
> controller. Learnt this the hard way when my raid 5 controller died 
> years after I first got it and I could no longer find a replacement.
>
> For high budget projects, having the extra raid controller as 
> insurance isn''t a big deal. But for most budget setup and cost 
> conscious clients, soft raid obviates that hardware dependency.
You forgot flaky hardware. Boards that lose their configuration from 
time to time and what not...I heard stories about Mylex for example...