samba - Apr 2012 - SMB2 write performace slower than SMB1 in 10Gb network

Hi forks:

I've been testing SMB2 with samba 3.6.4 performance these days,
and I find a weird benchmark that SMB2 write performance is
slower than SMB1 in 10Gb ethernet network.

Server
-----------------------
Linux: Redhat Enterprise 6.1 x64
Kernel: 2.6.31 x86_64
Samba: 3.6.4 (almost using the default configuration)
Network: Chelsio T4 T420-SO-CR 10GbE network adapter
RAID:
Adaptec 51645 RAID Controller (Writeback RAID0 with 16 * 1TB
SATA II disks)
Filesystem: xfs (barrier off)

Clinet
-----------------------
Windows 2008 Server R2 64bit
Network: Chelsio T4 T420-SO-CR 10GbE network adapter
Test tool: Iometer
Iometer configuration:
Normal I/O test policy, 1MB sequential read/write

Every Iometer test run by 3 minutes, Iometer test file size
 is 180GB.

Server and client are connected directly with fabric links,
without any 10GbE switches.

I use Iometer to test normal file read/write performance,
at first, SMB2 is not enabled,test 1MB sequential read/write performance
with SMB1, I got 610 MB/s write performance which
is really good, while read performance is just 280 MB/s.

Then I change "max protocol" to "SMB2" in smb.conf to enable
SMB2, other configs in smb.conf are not changed, restart samba,
reconnect in Windows 2008, use Iometer to run test again, I got
 470 MB/s write performance and 505 MB/s read performance.

It's quite good that read performance bursts with SMB2, but
write performace droped by 140 MB/s, I run this write test
serveral times again, write performance indeed droped a lot,
so is it normal that SMB2 write performance is slower than
SMB1?

Thanks ^_^

Regards
Uranus Zhou

On Sat, Apr 28, 2012 at 01:46:35PM +0800, Zhiming Zhou
wrote:
> I use Iometer to test normal file read/write performance,
> at first, SMB2 is not enabled,test 1MB sequential read/write performance
> with SMB1, I got 610 MB/s write performance which
> is really good, while read performance is just 280 MB/s.
> 
> Then I change "max protocol" to "SMB2" in smb.conf to
enable
> SMB2, other configs in smb.conf are not changed, restart samba,
> reconnect in Windows 2008, use Iometer to run test again, I got
>  470 MB/s write performance and 505 MB/s read performance.
> 
> It's quite good that read performance bursts with SMB2, but
> write performace droped by 140 MB/s, I run this write test
> serveral times again, write performance indeed droped a lot,
> so is it normal that SMB2 write performance is slower than
> SMB1?
No, this is not normal. Maybe you want to enable async I/O.
Depending on the platform, plain aio might work with

aio write size = 1

It might be a good idea to also enable

vfs objects = aio_fork

With latest Samba Jeremy Allison has also added the
aio_pthread module, but that is not yet in released code.
You can get it from the latest v3-6-test branch in our git
repository.

With best regards,

Volker Lendecke

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 G?ttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG G?ttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt at sernet.de