Hello,
I, like many before me, am having an issue with SMB multichannel. I am running
Samba v4.10.16 (package is provided by Amazon) on an Amazon Linux 2 EC2
instance. I am aware that SMB multichannel is an experimental feature until
4.15, but the documentation states it is supported so I would like to evaluate
it.
We have had reports of performance degradation when certain workloads (heavy
write workloads) are using this Samba server. Upon further investigation, I
believe we are not getting the benefits of SMB multichannel. According to the
documentation the feature can be enabled by setting "server multi channel
support = true" and also an option to force it on. There are also many
articles or posts where it is suggested to set interfaces along with their IP,
speed, and capabilities (such as RSS). All of these things are set in my
configuration, but SMB multichannel does not engage.
On the client side, I am running another EC2 instance but it is a Windows Server
2019 instance. SMB3 is supported and multichannel support is enabled by default,
along with BandwidthThrottling being disabled. When I connect to an SMB share
hosted on another Windows EC2 instance, the results of the
Get-SmbMultichannelConnection -IncludeNotSelected command returns that the
connection is a multichannel connection. However, when connected to an SMB share
hosted on my Samba EC2 instance, I get no results for it at all. The results of
Get-SmbConnection does return a result showing a connection is established to my
Samba server.
I have done a LOT of digging including trying a newer Samba version (v4.16.5),
which is after 4.15 where multichannel is no longer experimental, and I got the
same results. I am aware that SMB multichannel from a hardware perspective
relies on multiple physical CPU cores and either multiple NICs or a single
RSS-capable NIC on both sides. I have tried EC2 instances (both client and
server) that have had up to 16 physical cores each (32 vCPUs with
hyperthreading), 32 GB of memory, and NICs capable of 25 Gbps. AWS has concepts
of CPU and Network burst credits but for short bursts it should not be a problem
here.
I have tried multiple storage types including EBS, EFS, and FSx (ZFS). For EBS,
with 16k provisioned IOPS and a 1 GB/s throughput max, I was only getting
300-400 MB/s average and 500-600 MB/s max throughput at the beginning of
transfers through samba. No results returned in powershell for multichannel
connections. I got very similar performance from the other solutions even when I
have configured the resources with a specific performance configuration trying
to test peak performance.
My goal here is to ultimately confirm that SMB multichannel is working, but I
have a feeling that the results of the PowerShell commands on the client side
prove it is not. Thank you for taking the time to read this and if you provide
feedback I greatly appreciate it!
Here is my smb.conf file:
[global]
log file = /var/log/samba/%m
log level = 1
unix password sync = yes
netbios name = sambatoebs
map to guest = bad user
guest account = samba-client
hosts allow = 172.27.252.0/255.255.255.0
force user = samba-client
guest ok = yes
force create mode = 777
force directory mode = 777
server multi channel support = yes
force:server multi channel support = yes
interfaces = "172.27.252.40;capability=RSS,speed=25000000000"
aio write size = 1
aio read size = 1
[filestore]
path = /mnt/filestore
read only = no
writable = yes
browsable = yes
public = yes
[filestore_readonly]
path = /mnt/filestore
read only = yes
writable = no
browsable = yes
public = yes
[tmp]
path = /mnt/filestore/tmp
read only = no
writable = yes
browsable = yes
public = yes
[https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_03.png]
Carter Sheehan
Cloud Engineer
o: +1 (781) 224-7753<tel:+1%20(781)%20224-7753>
e: csheehan at vestmark.com<mailto:csheehan at vestmark.com>
Learn more about our next-generation UMA capabilities - Watch the
Video<https://vimeo.com/543224747>
Check us out on Social Media:
[https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_07.png]
<https://www.linkedin.com/company/vestmark/>
[https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_09.png]
<https://www.youtube.com/channel/UCjiTzgNkrybI3RiI4KUrKGQ/featured>
[https://smart.vestmark.com/v2/imagebucket/vestmark.com/Vestmark_01_large_11.png]
<https://twitter.com/Vestmark>
This e-mail and any attachments hereto, are intended for use by the addressee(s)
only and may contain information that is confidential information of Vestmark,
Inc. If you are not the intended recipient of this e-mail, or if you have
otherwise received this e-mail in error, please immediately notify me by
telephone or by e-mail, and please permanently delete the original, any print
outs and any copies of the foregoing. Any dissemination, distribution or copying
of this e-mail is strictly prohibited.