samba - Aug 2023 - Increase data length for SMB2 write and read requests for Windows 10 clients

Hello,

We are currently running a Linux Samba gateway with a CephFS 
kernel-client to allow Windows 10 clients to transfer data to and from 
our Ceph storage cluster. We have been using this in production for a 
while now quite successfully but we are finding that our Windows clients 
are having their rates limited to ~1 Gbps despite having 10 Gbit 
interfaces (iperf tests show a 4.18 Gbps network speed). We can achieve 
far higher transfer rates when using Linux clients by setting the 
"wsize" and "rsize" mount.cifs options (we find that 8MB
works well) but
we've been unable to find an equivalent option for our Windows clients 
as there doesn't seem to be an option to specify how much the data will 
be sent in 1 SMB request.

When inspecting the logs (as well as with Wireshark), we can see that 
the Linux clients have their read and write data lengths specified by 
the "wsize" and "rsize" options:

    [2022/12/09 11:47:26.550170,? 3]
    ../../source3/smbd/smb2_write.c:220(smb2_write_complete_internal)
     ? smb2: fnum 3199665708, file
    iso_test_folder/copy_back/output_8gb.dat, length=4194304
    offset=2948595712 wrote=4194304

    # where "mount.cifs -o wsize=4MB" by default for dialects at least
    SMB2.1. Hence the "length=" and "wrote=" values are
4*1024^2 bytes


However for a Windows client, the data length of the SMB packets are 
limited to 1 MB however:

    [2022/12/09 11:35:49.331003,? 3]
    ../../source3/smbd/smb2_read.c:430(smb2_read_complete)
     ? smbd_smb2_read: fnum 1997996337, file
    iso_test_folder/output_8gb.dat, length=1048576 offset=7298088960
    read=1048576

    # "length=" and "read=" are 1MB in size


We are running the same version of Samba (4.16.4) when testing with the 
clients with a 4.18 kernel (for our gateway) but we've been unable to 
find a way to have Windows to send larger SMB packets. Has anyone else 
looked into this and have they found any success? Is there an option in 
the smb.conf that we are missing?

Below are the options we've added to our smb.conf to try and optimize 
data transfer (where any further advice would also be welcome):

[global]
 ??? bind interfaces only = yes
 ?? ?interfaces = "ens18;capability=RSS,speed=10000000000"
 ?? ?workgroup = ceph
 ?? ?server role = standalone server
 ?? ?server min protocol = SMB2_10
 ?? ?client min protocol = SMB2_10
 ??? winbind max domain connections = 10
 ?? ?winbind cache time = 5
 ?? ?allocation roundup size = 524288
 ?? ?socket options = IPTOS_THROUGHPUT TCP_NODELAY SO_RCVBUF=319488 
SO_SNDBUF=319488
 ?? ?block size = 4096
 ?? ?write raw = Yes
 ?? ?read raw = Yes
 ?? ?max xmit = 65535
 ?? ?min receivefile size = 128000
 ?? ?aio max threads = 200
 ?? ?aio read size = 2048
 ?? ?aio write size = 2048


We would be very eager to hear any thoughts or comments that the wider 
Samba community have to offer!


Kindest regards,

Ivan Clayson

-- 
Ivan Clayson
-----------------
Scientific Computing Officer
MRC Laboratory of Molecular Biology
Francis Crick Ave, Cambridge
CB2 0QH

On Wed, Aug 16, 2023 at 04:02:47PM +0100, Ivan Clayson via samba
wrote:
>Hello,
>
>We are currently running a Linux Samba gateway with a CephFS 
>kernel-client to allow Windows 10 clients to transfer data to and from 
>our Ceph storage cluster. We have been using this in production for a 
>while now quite successfully but we are finding that our Windows 
>clients are having their rates limited to ~1 Gbps despite having 10 
>Gbit interfaces (iperf tests show a 4.18 Gbps network speed). We can 
>achieve far higher transfer rates when using Linux clients by setting 
>the "wsize" and "rsize" mount.cifs options (we find that
8MB works
>well) but we've been unable to find an equivalent option for our 
>Windows clients as there doesn't seem to be an option to specify how 
>much the data will be sent in 1 SMB request.
>
>When inspecting the logs (as well as with Wireshark), we can see that 
>the Linux clients have their read and write data lengths specified by 
>the "wsize" and "rsize" options:
I don't know of any Windows client tuning parameter to set to
increase their r/w sizes (or even to change them in any way).

Any Windows experts on this list who might know a "secret"
registry key ?

On Wed, Aug 16, 2023 at 04:02:47PM +0100, Ivan Clayson via samba
wrote:
>
>We are running the same version of Samba (4.16.4) when testing with 
>the clients with a 4.18 kernel (for our gateway) but we've been unable 
>to find a way to have Windows to send larger SMB packets. Has anyone 
>else looked into this and have they found any success? Is there an 
>option in the smb.conf that we are missing?
FYI, to my knowledge there's nothing in any server setting
that will cause Windows clients to change r/w sizes.