I have an Intel S875PWP1 motherboard with a Pentium4 CPU@2.40GHz. PCI
bus is 33Mhz, 32-bit. I recently purchased an LSI 320-2/128MB on eBay
(though the card really looks like a PERC4/DS) and just ran some
bonnie++ tests on a RAID 1 array between two U320 drives for the first
channel and on a RAID 0 array between one U320 drive for the second
channel. The 320-2 has the latest LSI firmware, 1L47.
#1. (RAID 1, two U320 drives, channel 1)
Version 1.93c ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
maetel.il.thew 300M 270 99 16917 5 15734 5 514 99 +++++ +++ 4237 48
Latency 33357us 130ms 86843us 33195us 1063us 120ms
Version 1.93c ------Sequential Create------ --------Random Create--------
maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 9214 26 +++++ +++ +++++ +++ 6687 19 +++++ +++ +++++ +++
Latency 424ms 129us 2629us 475ms 115us 82us
#2. (RAID 1, two U320 drives, one on channel 1, one on channel 2)
Version 1.93c ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
maetel.il.thew 300M 272 99 16787 4 15949 5 515 99 +++++ +++ 4575 50
Latency 31339us 139ms 1925ms 29314us 848us 1623ms
Version 1.93c ------Sequential Create------ --------Random Create--------
maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 7497 22 +++++ +++ +++++ +++ 5850 17 +++++ +++ +++++ +++
Latency 516ms 123us 3381us 577ms 14425us 50us
#3. (RAID 0, one U320 drive, channel 2)
Version 1.93c ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
maetel.il.thew 300M 270 99 17347 5 16347 5 530 99 +++++ +++ 4510 46
Latency 34033us 131ms 100ms 34354us 1061us 1575ms
Version 1.93c ------Sequential Create------ --------Random Create--------
maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 8401 24 +++++ +++ +++++ +++ 6489 19 +++++ +++ +++++ +++
Latency 574ms 127us 7480us 640ms 16607us 46us
#4. I also testing an Adaptec 39160 card with one U320 drive:
Version 1.93c ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
maetel.il.thew 300M 259 99 62293 18 36054 12 522 99 +++++ +++ 5747 58
Latency 33370us 26379us 43979us 34708us 1064us 1264ms
Version 1.93c ------Sequential Create------ --------Random Create--------
maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 16761 49 +++++ +++ +++++ +++ 13569 40 +++++ +++ +++++ +++
Latency 142ms 141us 49us 203ms 123us 43us
I don't understand how #1 gives 16MB/s (RAID 1) while #3 gives only
17MB/s. I'd expect #3 to be about x2 the performance of #1.
The arrays are configured with "write-thru" write policy,
"adaptive"
read policy, and "cachedio" cache policy.
Anything I can do to improve the speed. Even though the bus is 33Mhz,
I should definitely be doing better than 16MB/s.
I'm running bonnie++ as:
$ cd [some tmp dir on UFS2 file system]
$ bonnie++
The system runs FreeBSD 6.1-STABLE:
$ uname -a
FreeBSD maetel.il.thewrittenword.com 6.1-STABLE FreeBSD 6.1-STABLE #0: Mon May
8 22:25:41 CDT 2006
$ dmesg | grep amr0
amr0: <LSILogic MegaRAID 1.53> mem 0xf47f0000-0xf47fffff irq 21 at
device 0.0 on pci3
amr0: delete logical drives supported by controller
amr0: <LSILogic MegaRAID SCSI 320-2> Firmware 1L47, BIOS G121, 128MB RAM
amr0: delete logical drives supported by controller
--
albert chin (china@thewrittenword.com)
On Thu, Sep 28, 2006 at 05:15:05PM -0500, Albert Chin wrote:> I don't understand how #1 gives 16MB/s (RAID 1) while #3 gives only > 17MB/s. I'd expect #3 to be about x2 the performance of #1. > > The arrays are configured with "write-thru" write policy, "adaptive" > read policy, and "cachedio" cache policy. > > Anything I can do to improve the speed. Even though the bus is 33Mhz, > I should definitely be doing better than 16MB/s. > > {snip} > > I'm running bonnie++ as: > $ cd [some tmp dir on UFS2 file system] > $ bonnie++Pure shot in the dark (I have no experience with LSI controllers, only Adaptec and Promise): have you tried adjusting some of the tunefs parameters for the UFS2 filesystem bonnie++ is using? Options -e and -f may help with this. Also, at least in the Adaptec world: if the controller does its own caching (that is, it has actual RAM on-board or DRAM of some kind), it is sometimes *better* to disable SCSI disk caching on each individual SCSI drive and let the controller do it. Again, can't speak for LSI controllers... -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Albert Chin wrote:> I have an Intel S875PWP1 motherboard with a Pentium4 CPU@2.40GHz. PCI > bus is 33Mhz, 32-bit. I recently purchased an LSI 320-2/128MB on eBay > (though the card really looks like a PERC4/DS) and just ran some > bonnie++ tests on a RAID 1 array between two U320 drives for the first > channel and on a RAID 0 array between one U320 drive for the second > channel. The 320-2 has the latest LSI firmware, 1L47. >[...]> > I don't understand how #1 gives 16MB/s (RAID 1) while #3 gives only > 17MB/s. I'd expect #3 to be about x2 the performance of #1.> > The arrays are configured with "write-thru" write policy, "adaptive" > read policy, and "cachedio" cache policy.In LSI terms, "write-thru" means, "write cache is turned off". With that set, performance becomes very consistently describable as "poor", as your tests demonstrate. Linux can get around this a little bit in benchmarks by using larger I/O sizes than FreeBSD (FreeBSD is limited to 128K max, while Linux is not). These are very speedy cards once you put them into their intended configuration. With some decent SCSI disks and a PCI-X bus, it performs quite well. However, I'm actually kinda surprised that the card works at all in your 5V slot. Scott
On Thu, Sep 28, 2006 at 05:15:05PM -0500, Albert Chin wrote:> I have an Intel S875PWP1 motherboard with a Pentium4 CPU@2.40GHz. PCI > bus is 33Mhz, 32-bit. I recently purchased an LSI 320-2/128MB on eBay > (though the card really looks like a PERC4/DS) and just ran some > bonnie++ tests on a RAID 1 array between two U320 drives for the first > channel and on a RAID 0 array between one U320 drive for the second > channel. The 320-2 has the latest LSI firmware, 1L47.I reran some of the tests with the same 320-2 but on an Intel SE7520BD2 with 32-bit and 64-bit (100Mhz) slots: #1. RAID 1, two U320 drives, channel 1, 32-bit, 33Mhz slot Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP maetel.il.thew 300M 186 99 16707 5 16063 6 654 99 537320 93 4129 50 Latency 45215us 199ms 89764us 34740us 1215us 1808ms Version 1.93c ------Sequential Create------ --------Random Create-------- maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 7441 23 +++++ +++ +++++ +++ 5799 18 +++++ +++ +++++ +++ Latency 479ms 122us 2508us 606ms 13549us 101us #2. RAID 1, two U320 drives, channel 1, 64-bit, 100Mhz slot Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP maetel.il.thew 300M 186 99 18006 6 15964 5 634 99 571275 99 4450 57 Latency 44992us 139ms 130ms 35143us 1238us 120ms Version 1.93c ------Sequential Create------ --------Random Create-------- maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 7581 24 +++++ +++ +++++ +++ 5750 18 +++++ +++ +++++ +++ Latency 511ms 255us 2615us 622ms 12691us 53us Odd that I don't get x2 the performance when the bus bandwidth doubles in speed. With the write policy set to "write-back", performance increases to: #3. RAID 1, two U320 drives, channel 1, 32-bit, 33Mhz slot Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP maetel.il.thew 300M 182 99 36466 12 28602 11 624 97 570155 99 +++++ +++ Latency 102ms 918ms 1739ms 35305us 1210us 198ms Version 1.93c ------Sequential Create------ --------Random Create-------- maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 18283 57 +++++ +++ +++++ +++ 16092 50 +++++ +++ +++++ +++ Latency 65371us 107us 101us 119ms 258us 104us #4. RAID 1, two U320 drives, channel 1, 64-bit, 100Mhz slot Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP maetel.il.thew 300M 186 99 43215 14 35049 12 560 88 569643 99 +++++ +++ Latency 44935us 2071ms 1174ms 35558us 1213us 279ms Version 1.93c ------Sequential Create------ --------Random Create-------- maetel.il.thewritte -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 19689 62 +++++ +++ +++++ +++ 15578 51 +++++ +++ +++++ +++ Latency 67923us 1227us 49362us 117ms 198us 93us -- albert chin (china@thewrittenword.com)
Albert Chin wrote:> The arrays are configured with "write-thru" write policy, "adaptive" > read policy, and "cachedio" cache policy.cachedio will slow you down on the old LSI cards. Only enable it on the PCI-X and above adapters. /Martin