samba - Mar 2014 - suggestions for a "fast" fileserver - 1G / 10G - focus on smb.conf/samba

Hi,

thanks for all the feedback and topics.

May be I can/should draw a better picture of my settings and question.

We "still" use samba 3.6.9-167 from Red Hat with an openldap backend.

I tried from an older posting somewhere:

socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536

But there was no noticeable impact. The log level = 2.

Thats all I changed compared to the Red Hat default settings.

We have some reliable, fast hardware SATA ISCSI raid boxes with min. 12
disks, most are 16 disks, configured in raid  10, 5, 6 depending on the
use case and date stored. (e.g. lots of smaller r/w, some are big files
min. some Gig.)

We did also a lot of tests with different filesystems settings, so
currently we use ext4 and xfs (xfs e.g. for filesystems about 40 TB)

And: We do backups, have BBUs and "all luxury" we can get and need.

So my original question was focused:

Why do I get r/w speed around 80-90MB/s with ftp or scp and only
+-40MB/s on samba?

I don't want to believe, that smb has such a biiiigggg overhead :)

It is the same server, raid, filesystem, network konnection. In that
Case 1Gbit from Client (OS X 10.9) -> Gbit Switch -> Server -> Storage.

The Storage is connected to the server by a separate GBit VLAN.

As someone mentioned, I looked at the cpu usage and with smb it is >60/70%.

In my tests mentioned, I'm the only user on the server and Storage, no
heavy background tasks.

So what can speed up smb?

	Any feedback and suggestions are welcome. Regards . G?tz
-- 
G?tz Reinicke
IT-Koordinator

Tel. +49 7141 969 82 420
Fax  +49 7141 969 55 420
E-Mail goetz.reinicke at filmakademie.de

Filmakademie Baden-W?rttemberg GmbH
Akademiehof 10
71638 Ludwigsburg
www.filmakademie.de

Eintragung Amtsgericht Stuttgart HRB 205016

Vorsitzender des Aufsichtsrats:
J?rgen Walter MdL
Staatssekret?r im Ministerium f?r Wissenschaft,
Forschung und Kunst Baden-W?rttemberg

Gesch?ftsf?hrer:
Prof. Thomas Schadt

>
> Why do I get r/w speed around 80-90MB/s with ftp or scp and only
> +-40MB/s on samba?
>
Did you set? :

max protocol = SMB2

in the [global] section and restart/reload samba? It looks like it wasn't
until 3.6.16 that max protocol = SMB2 was actually removed as experimental
and enabled by default, and you're still at 3.6.9 :

https://wiki.samba.org/index.php/Samba_3.6_Features_added/changed#Samba_3.6.16

Like I said before, I got the same 40MB/s when connected at SMBv1 .

Le Tue, 25 Mar 2014 08:14:06 +0100 vous ?criviez:
> Hi,
> 
> thanks for all the feedback and topics.
> 
> May be I can/should draw a better picture of my settings and question.
> 
> We "still" use samba 3.6.9-167 from Red Hat with an openldap
backend.
That shouldn't be a problem, really.
> I tried from an older posting somewhere:
> 
> socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536
> SO_SNDBUF=65536
Normally these are not necessary. In particular I'm pretty sure
enforcing buffer size can do more harm than good.
> We have some reliable, fast hardware SATA ISCSI raid boxes with min.
> 12 disks, most are 16 disks, configured in raid  10, 5, 6 depending
> on the use case and date stored. (e.g. lots of smaller r/w, some are
> big files min. some Gig.)
><...>
> 
> So what can speed up smb?
What is the disk subsystem performance like from the samba
server point of view? Do you use the nobarrier/barrier=0 option on the
shared filesystem?  

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac at intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

On Tue, Mar 25, 2014 at 08:14:06AM +0100, G?tz Reinicke - IT Koordinator
wrote:
> Hi,
> 
> thanks for all the feedback and topics.
> 
> May be I can/should draw a better picture of my settings and question.
> 
> We "still" use samba 3.6.9-167 from Red Hat with an openldap
backend.
> 
> I tried from an older posting somewhere:
> 
> socket options = TCP_NODELAY IPTOS_LOWDELAY SO_RCVBUF=65536 SO_SNDBUF=65536
> 
> But there was no noticeable impact.
socket options changes are voodoo bullshit on
modern kernels. They come from a time long long
ago in a galaxy far away... Modern kernels do a
much better job of tuning themselves. Please don't
mess with them in smb.conf :-(.

One other tuning note I just found out (from experimenting).

Use a high speed congestion control algorithm.

I just tried htcp and reno(the default)...and both had lower
throughput by ~200MB/s than my normal default
of 'highspeed'. 

It seems that it is irrelevant of actual congestion, since
there's a dedicated line between the two machines
tested.

Just another 'heads-up'