freebsd stable - Jun 2016 - mfi driver performance too bad on LSI MegaRAID SAS 9260-8i

Hi,

I am working on storage service based on FreeBSD.  I look forward to a good
result because many professional storage company use FreeBSD as its OS.  But I
am disappointed with the Bad performance.  I tested the the performance of LSI
MegaRAID 9260-8i and had the following bad result:

  1.  Test environment:
       (1) OS:   FreeBSD 10.0 release
       (2) Memory:  16G
       (3) RAID adapter:   LSI MegaRAID 9260-8i
       (4) Disks:  9 SAS hard drives (10000 rpm),  performance is expected for
each hard drive
       (5) Test tools:   fio with  io-depth=1, thread num is 32 and block size
is 64k or 1M
       (6)  RAID configuration:  RAID 5,   stripe size is 1M

 2.  Test result:
      (1)  write performance too bad:  20Mbytes/s throughput and 200 random
write IOPS
      (2)  read performance is expected:  700Mbytes/s throughput and 1500 random
read IOPS


I tested the same hardware configuration with CentOS linux and Linux's write
performance is 5 times better than FreeBSD.


Anyone encountered the same performance problem?  Does the mfi driver have
performance issue or I should give up on FreeBSD?





Jason

On Fri, Jun 17, 2016, at 02:17, Jason Zhang wrote:
> Hi,
> 
> I am working on storage service based on FreeBSD.  I look forward to a
> good result because many professional storage company use FreeBSD as its
> OS.  But I am disappointed with the Bad performance.  I tested the the
> performance of LSI MegaRAID 9260-8i and had the following bad result:
> 
>   1.  Test environment:
>        (1) OS:   FreeBSD 10.0 release
10.0-RELEASE is no longer supported. Can you reproduce this on
10.3-RELEASE?


-- 
  Mark Felder
  feld at feld.me

On Fri, Jun 17, 2016, at 02:17, Jason Zhang wrote:
> Hi,
> 
> I am working on storage service based on FreeBSD.  I look forward to a
> good result because many professional storage company use FreeBSD as its
> OS.  But I am disappointed with the Bad performance.  I tested the the
> performance of LSI MegaRAID 9260-8i and had the following bad result:
> 
>   1.  Test environment:
>        (1) OS:   FreeBSD 10.0 release
10.0-RELEASE is no longer supported. Can you test this on 10.3-RELEASE?

Have you confirmed that both servers are using identical RAID controller
settings? It's possible the CentOS install has enabled write caching but
it's disabled on your FreeBSD server. Are you using UFS or ZFS on
FreeBSD? Do you have atime enabled? I believe CentOS is going to have
"relatime" or "nodiratime" by default to mitigate the write
penalty on
each read access.

We need more data :-)


-- 
  Mark Felder
  ports-secteam member
  feld at FreeBSD.org