samba - Oct 2010 - samba 3.3 - poor performance (compared to NFS)

I have a system that I'm vetting as a NAS server.  It has a 2.0TB XFS
filesystem mounted on /storage and I'm doing benchmarks using nfs3, nfs4,
and samba.  I'm testing via iozone by mounting the filesystem from my
"nas client" box and then running iozone on the mounted filesystem. 
NFS seems pretty fast - ie, several orders of magnitude faster than samba, and
I'm wondering why, so I'm beseeching the help of the List. :)

 

server: sama 3.3.8

client: Linux CentOS 5.5 cifs mount, "mount -t cifs -o
rsize=32768,wsize=32768 //server/storage /storage"

Client is on the same LAN as the server, albeit different VLANs.  Traffic is
routed through intel gigabit NICs and Cisco Nexus 5000/7000 series switches. 
NAS server has a 4x 1gbe 802.3ad port channel set up with the Cisco 7000 switch,
although I've run these tests both with and without the port channel with
very similar results (as I'd expect, since the client is only a single 1gbe
interface to begin with).

 

(the 32768 numbers are the same as used in the NFS3/NFS4 tests).

Again, the problem is *markedly* slower performance on CIFS than with NFS, and I
cannot discern why, so I'm assuming it's some kind of samba tuning
issue.  I do plan to re-test with samba4, but any recommendations as to a
specific version of samba that I could use which would provide maximum
performance/stability would also be much appreciated.

 

/etc/smb/smb.conf on the server is below:

 

[global]

               workgroup = myworkgroup

               server string = Samba %v

               netbios name = myhostname.mydomain

               hosts allow = 10.

               log file = /var/log/samba/%m.log

               max log size = 50

               security = user

               passdb backend = tdbsam

               os level = 33

               wins support = yes

               wins proxy = yes

               dns proxy = yes

               load printers = no

               map archive = no

               map hidden = no

               map read only = no

               map system = no

               store dos attributes = yes

               socket options = IPTOS_LOWDELAY TCP_NODELAY

               pam password change = yes

 

[storage]

        comment = storage volume

        browseable = yes

        writable = yes

        path = /storage

 

===================
Scott Stone <scott_stone at trendmicro.com>

Lead Developer, DCS-RD

Trend Micro, Inc. http://www.trendmicro.com

 


TREND MICRO EMAIL NOTICE
The information contained in this email and any attachments is confidential and
may be subject to copyright or other intellectual property protection. If you
are not the intended recipient, you are not authorized to use or disclose this
information, and we request that you notify us by reply mail or telephone and
delete the original message from your mail system.

On Mon, Oct 04, 2010 at 02:51:17PM -0700, scott_stone at trendmicro.com
wrote:
> I have a system that I'm vetting as a NAS server.  It has a 2.0TB XFS
filesystem mounted on /storage and I'm doing benchmarks using nfs3, nfs4,
and samba.  I'm testing via iozone by mounting the filesystem from my
"nas client" box and then running iozone on the mounted filesystem. 
NFS seems pretty fast - ie, several orders of magnitude faster than samba, and
I'm wondering why, so I'm beseeching the help of the List. :)
> 
>  
> 
> server: sama 3.3.8
> 
> client: Linux CentOS 5.5 cifs mount, "mount -t cifs -o
rsize=32768,wsize=32768 //server/storage /storage"
> 
> Client is on the same LAN as the server, albeit different VLANs.  Traffic
is routed through intel gigabit NICs and Cisco Nexus 5000/7000 series switches. 
NAS server has a 4x 1gbe 802.3ad port channel set up with the Cisco 7000 switch,
although I've run these tests both with and without the port channel with
very similar results (as I'd expect, since the client is only a single 1gbe
interface to begin with).
> 
>  
> 
> (the 32768 numbers are the same as used in the NFS3/NFS4 tests).
> 
> Again, the problem is *markedly* slower performance on CIFS than with NFS,
and I cannot discern why, so I'm assuming it's some kind of samba tuning
issue.  I do plan to re-test with samba4, but any recommendations as to a
specific version of samba that I could use which would provide maximum
performance/stability would also be much appreciated.
You might want to try a more recent cifsfs build than the one on CentOS 5.5.

It's almost certainly a client issue here, I know Steve and Jeff have been
putting work into improving the CIFSFS client performance (Steve and Jeff
please comment :-).

Jeremy.