samba - Mar 2009 - slow throughput on 1gbit lan

Hello,

I'm trying to squeeze out maximum possible transfer between two Samba 3.3.1
servers. Both servers and switch supports jumbo frames, network link and
disks does not appear to be bottleneck, between these two servers FTP
transfer rates are about 115MB/s (1gbit link is saturated at this point) -
and I'd like to see similar transfer from Samba (now I get half of it).

10.0.0.5 # wget ftp://10.0.0.2/1G
100%[=================================================================================================================>]
1,048,576,000  118M/s   in 8.5s
2009-02-27 09:16:00 (118 MB/s) - `1G' saved [1048576000]


Also testing samba locally, it gives me transfer via loopback of over
300MB/s

10.0.0.2 # smbclient '\\10.0.0.2\test\' xxx -U xxx -c 'GET 1GB'
WARNING: The "write cache size" option is deprecated
Domain=[EPW] OS=[Unix] Server=[Samba 3.3.1]
getting file \1GB of size 1048576000 as 1GB (331177.2 kb/s) (average
331177.2 kb/s)


Now, when I try to get this file from another server (10.0.0.5), maximum
transfer drops to 55MB/s....

10.0.0.5 # smbclient '\\10.0.0.2\test\' xxx -U xxx -c 'GET 1GB'
Domain=[EPW] OS=[Unix] Server=[Samba 3.3.1]
getting file \1GB of size 1048576000 as 1GB (55664.3 kb/s) (average 55664.3
kb/s)


So far, I was turning these knobs:

socket options = TCP_NODELAY SO_RCVBUF=262144 SO_SNDBUF=262144
max xmit = 131072
block size = 4096
aio read size = 1
aio write size = 1
use mmap = No
use sendfile = Yes
blocking locks = No

For me only SO_RCVBUF and SO_SNDBUF seems to have any influence, but setting
it to more than 65534 doesn't give any more performance. I suppose TCP
kernel settings are fine, because FTP transfer is able to saturate link...
Do you have any ideas how to improve performance ?

Regards,
Tomek

On Mon, Mar 02, 2009 at 05:33:49PM +0100, Tomasz Lutelmowski
wrote:
> Hello,
> 
> I'm trying to squeeze out maximum possible transfer between two Samba
3.3.1
> servers. Both servers and switch supports jumbo frames, network link and
> disks does not appear to be bottleneck, between these two servers FTP
> transfer rates are about 115MB/s (1gbit link is saturated at this point) -
> and I'd like to see similar transfer from Samba (now I get half of it).
> 
> 10.0.0.5 # wget ftp://10.0.0.2/1G
>
100%[=================================================================================================================>]
> 1,048,576,000  118M/s   in 8.5s
> 2009-02-27 09:16:00 (118 MB/s) - `1G' saved [1048576000]
> 
> 
> Also testing samba locally, it gives me transfer via loopback of over
> 300MB/s
> 
> 10.0.0.2 # smbclient '\\10.0.0.2\test\' xxx -U xxx -c 'GET
1GB'
> WARNING: The "write cache size" option is deprecated
> Domain=[EPW] OS=[Unix] Server=[Samba 3.3.1]
> getting file \1GB of size 1048576000 as 1GB (331177.2 kb/s) (average
> 331177.2 kb/s)
> 
> 
> Now, when I try to get this file from another server (10.0.0.5), maximum
> transfer drops to 55MB/s....
> 
> 10.0.0.5 # smbclient '\\10.0.0.2\test\' xxx -U xxx -c 'GET
1GB'
> Domain=[EPW] OS=[Unix] Server=[Samba 3.3.1]
> getting file \1GB of size 1048576000 as 1GB (55664.3 kb/s) (average 55664.3
> kb/s)
> 
> 
> So far, I was turning these knobs:
> 
> socket options = TCP_NODELAY SO_RCVBUF=262144 SO_SNDBUF=262144
> max xmit = 131072
> block size = 4096
> aio read size = 1
> aio write size = 1
> use mmap = No
> use sendfile = Yes
> blocking locks = No
What do you get without any of these? And, are you 100% sure
you've got a 3.3.1 smbclient?

Volker
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://lists.samba.org/archive/samba/attachments/20090302/bae6b6ee/attachment.bin