[CentOS] Write performance with 3ware 9550

Bowie Bailey
Fri, 13 Oct 2006 13:05:09 -0000

Non-interleaved memory sounds interesting.  Where do I change that
setting?

I don't want to change the StorSave setting.  I'm not really looking
for maximum performance here, just reasonable.  If I can get the same
60 M/s speed that I got from the first server, I'll be happy.

The drives are all running at 3.0GB/Sec according to the drive
information in 3DM.

I have read the entire "Calling All FS Fanatics" thread as it passed
through the list.  It was quite interesting, but the main things I got
from it are:

    - Increase the read ahead settings
    - XFS, JFS, and ReiserFS are faster, but less fault-tolerant, than
      ext3

I have already changed the read ahead on both the physical volume and
the logical volume.  This gives me increased read performance, but has
no effect on writes.  I am sticking with ext3 since I am much more
concerned about stability than performance.

Bowie

Kirk Bocek wrote:
> I saw a definite improvement by turning off NCQ and setting StorSave
> to 'Balanced.' Are these 1.5GB/Sec or 3.0GB/Sec SATA drives? During
> my testing I changed from non-interleaved memory and 1.5GB to
> interleaved and 3.0GB. Made a big difference in bonnie++ results.
> Unfortunately, I can't say which was more important.
>
> If you have the patience, read through my recent (but lengthy) thread
> on the 3Ware 9550 titled "Calling All FS Fanatics." There's a
lot of
> good info from many helpful people. I've only gotten full performance
> using JFS or XFS.
>
> Kirk Bocek
>
> Bowie Bailey wrote:
> > I have two identical servers.  The only difference is that the first
> > one has Maxtor 250G drives and the second one has Seagate 320G
> > drives.
> >
> > OS: CentOS-4.4 (fully patched)
> > CPU: dual Opteron 280
> > Memory: 16GB
> > Raid card: 3ware 9550Sx-8LP
> > Raid volume: 4-disk Raid 5 with NCQ and Write Cache enabled
> >
> > On the first server I have decent performance.  Nothing spectacular,
> > but good enough.  The second one has about 1/3 the write speed.  I
> > can't find any difference between the systems.  Both of them have
> > the same stripe size, both have ext3 filesystems, both have write
> > caching and NCQ turned on.  I have already increased the read ahead
> > setting to 16384 on both servers.
> >
> > I ran the tests like this:
> >
> > # sync; bonnie++ -d /iotest -s 50g -n 0 -b -f
> > (I have removed some extra information from the reports for brevity)
> >
> > And here are the results for the two servers:
> >
> >                     ------Output------- --Input--
> >                     --Block-- -Rewrite- --Block-- --Seeks--
> > Machine        Size K/sec %CP K/sec %CP K/sec %CP  /sec %CP
> > First           50G 62893  25 46763  12 160672  19 120.6   1
> > Second          50G 18835   7 44025  12 194719  24 122.8   1
> >
> > As you can see, the write performance of the second server is
> > terrible.  Anyone have any suggestions of what I can look for?  I
> > keep thinking there must be something I tweaked on the first server
> > that I forgot about for the second one, but so far I haven't been
> > able to find it.
> >
> > Any suggestions appreciated!