[CentOS] Write performance with 3ware 9550

Bowie Bailey
Thu, 12 Oct 2006 20:22:41 -0000

I have two identical servers.  The only difference is that the first
one has Maxtor 250G drives and the second one has Seagate 320G drives.

OS: CentOS-4.4 (fully patched)
CPU: dual Opteron 280
Memory: 16GB
Raid card: 3ware 9550Sx-8LP
Raid volume: 4-disk Raid 5 with NCQ and Write Cache enabled

On the first server I have decent performance.  Nothing spectacular,
but good enough.  The second one has about 1/3 the write speed.  I
can't find any difference between the systems.  Both of them have the
same stripe size, both have ext3 filesystems, both have write caching
and NCQ turned on.  I have already increased the read ahead setting to
16384 on both servers.

I ran the tests like this:

# sync; bonnie++ -d /iotest -s 50g -n 0 -b -f
(I have removed some extra information from the reports for brevity)

And here are the results for the two servers:

                    ------Output------- --Input--
                    --Block-- -Rewrite- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP  /sec %CP
First           50G 62893  25 46763  12 160672  19 120.6   1
Second          50G 18835   7 44025  12 194719  24 122.8   1

As you can see, the write performance of the second server is
terrible.  Anyone have any suggestions of what I can look for?  I keep
thinking there must be something I tweaked on the first server that I
forgot about for the second one, but so far I haven't been able to
find it.

Any suggestions appreciated!

--
Bowie