Greetings!! I am seeking your help to resolve a problem I am facing with the network throughput of an SMB share. We have two SMB shares: one containing files which are bigger in size and the other containing a large number of small files. Both share sizes are in terabytes. While mapping the SMB share with large files on a Windows host and copying files and directories, we observe a good network throughput of more than 300MB/s. However, when we map the other SMB share with a large number of small files and attempt to copy files and directories, we experience a significantly reduced network throughput of around 10MB/s. Both SMB server and client are using SMB3 protocol. The "EnableMultichannel" parameter has been configured to 'True' on the Windows SMB client, but it is unclear how many channels will be created or if multiple channels have been established between the SMB client and server. A Wireshark dump on the SMB client side shows only eight connections with single client and single server IP addresses. Although the SMB client has only two NIC cards, the dump indicates only one IP address for the client in the source address of the TCP connection. There is little knowledge on optimizing multichannel and how many NICs are required on both the SMB client and server to achieve optimal network throughput. Additionally, the number of SMB/TCP connections needed between the SMB client and server to transfer terabytes of data at a high throughput rate remains unknown. We have configured the following parameters on our Windows SMB client host. PS C:\Users > Get-SmbClientConfiguration ConnectionCountPerRssNetworkInterface : 8 DirectoryCacheEntriesMax : 4096 DirectoryCacheEntrySizeMax : 65536 DirectoryCacheLifetime : 60 DormantFileLimit : 1023 EnableBandwidthThrottling : False EnableByteRangeLockingOnReadOnlyFiles : True EnableInsecureGuestLogons : True EnableLargeMtu : True EnableLoadBalanceScaleOut : True EnableMultiChannel : True EnableSecuritySignature : True ExtendedSessionTimeout : 1000 FileInfoCacheEntriesMax : 32768 FileInfoCacheLifetime : 60 FileNotFoundCacheEntriesMax : 32768 FileNotFoundCacheLifetime : 60 KeepConn : 600 MaxCmds : 32768 MaximumConnectionCountPerServer : 64 OpLocksDisabled : False RequireSecuritySignature : False SessionTimeout : 60 UseOpportunisticLocking : True WindowSizeThreshold : 1 PS C:\Users > Any advice on the optimal SMB tuning parameters and their respective values for both the SMB client and server sides would be highly appreciated. This is necessary to address the issue at hand and to achieve optimal network throughput on an SMB share that contains a large number of small files. Please provide recommendations on any other settings that may be relevant in this context. Your assistance in this matter would be highly valued. Thanks & regards, Swapnil M.
On 06/04/2023 22:37, Swapneel More via samba wrote:> Greetings!! > > I am seeking your help to resolve a problem I am facing with the network > throughput of an SMB share. > > We have two SMB shares: one containing files which are bigger in size and > the other containing a large number of small files. Both share sizes are in > terabytes. > > While mapping the SMB share with large files on a Windows host and copying > files and directories, we observe a good network throughput of more than > 300MB/s. However, when we map the other SMB share with a large number of > small files and attempt to copy files and directories, we experience a > significantly reduced network throughput of around 10MB/s. > > Both SMB server and client are using SMB3 protocol. > > The "EnableMultichannel" parameter has been configured to 'True' on the > Windows SMB client, but it is unclear how many channels will be created or > if multiple channels have been established between the SMB client and > server. A Wireshark dump on the SMB client side shows only eight > connections with single client and single server IP addresses. Although the > SMB client has only two NIC cards, the dump indicates only one IP address > for the client in the source address of the TCP connection. There is little > knowledge on optimizing multichannel and how many NICs are required on both > the SMB client and server to achieve optimal network throughput. > Additionally, the number of SMB/TCP connections needed between the SMB > client and server to transfer terabytes of data at a high throughput rate > remains unknown. > > We have configured the following parameters on our Windows SMB client host. > > PS C:\Users > Get-SmbClientConfiguration > > ConnectionCountPerRssNetworkInterface : 8 > DirectoryCacheEntriesMax : 4096 > DirectoryCacheEntrySizeMax : 65536 > DirectoryCacheLifetime : 60 > DormantFileLimit : 1023 > EnableBandwidthThrottling : False > EnableByteRangeLockingOnReadOnlyFiles : True > EnableInsecureGuestLogons : True > EnableLargeMtu : True > EnableLoadBalanceScaleOut : True > EnableMultiChannel : True > EnableSecuritySignature : True > ExtendedSessionTimeout : 1000 > FileInfoCacheEntriesMax : 32768 > FileInfoCacheLifetime : 60 > FileNotFoundCacheEntriesMax : 32768 > FileNotFoundCacheLifetime : 60 > KeepConn : 600 > MaxCmds : 32768 > MaximumConnectionCountPerServer : 64 > OpLocksDisabled : False > RequireSecuritySignature : False > SessionTimeout : 60 > UseOpportunisticLocking : True > WindowSizeThreshold : 1 > > PS C:\Users > > > Any advice on the optimal SMB tuning parameters and their respective values > for both the SMB client and server sides would be highly appreciated. This > is necessary to address the issue at hand and to achieve optimal network > throughput on an SMB share that contains a large number of small files. > Please provide recommendations on any other settings that may be relevant > in this context. > > Your assistance in this matter would be highly valued. > > Thanks & regards, > Swapnil M.Whilst you have told us how you have setup your Windows machine, you have told us very little about the SMB client, just that it uses SMBv3. For all we know, they could be both Windows machines, though this is doubtful, or you wouldn't be asking here. Can you please tell us what OS you are running on the client. What version of Windows the server is running Please provide the output of 'testparm -s' run on the client. Rowland
On 06.04.2023 23:37, Swapneel More via samba wrote:> Greetings!! > > I am seeking your help to resolve a problem I am facing with the network > throughput of an SMB share. > > We have two SMB shares: one containing files which are bigger in size and > the other containing a large number of small files. Both share sizes are in > terabytes. > > While mapping the SMB share with large files on a Windows host and copying > files and directories, we observe a good network throughput of more than > 300MB/s. However, when we map the other SMB share with a large number of > small files and attempt to copy files and directories, we experience a > significantly reduced network throughput of around 10MB/s. > > Both SMB server and client are using SMB3 protocol. > > The "EnableMultichannel" parameter has been configured to 'True' on the > Windows SMB client, but it is unclear how many channels will be created or > if multiple channels have been established between the SMB client and > server. A Wireshark dump on the SMB client side shows only eight > connections with single client and single server IP addresses. Although the > SMB client has only two NIC cards, the dump indicates only one IP address > for the client in the source address of the TCP connection. There is little > knowledge on optimizing multichannel and how many NICs are required on both > the SMB client and server to achieve optimal network throughput. > Additionally, the number of SMB/TCP connections needed between the SMB > client and server to transfer terabytes of data at a high throughput rate > remains unknown. > > We have configured the following parameters on our Windows SMB client host. > > PS C:\Users > Get-SmbClientConfiguration > > ConnectionCountPerRssNetworkInterface : 8 > DirectoryCacheEntriesMax : 4096 > DirectoryCacheEntrySizeMax : 65536 > DirectoryCacheLifetime : 60 > DormantFileLimit : 1023 > EnableBandwidthThrottling : False > EnableByteRangeLockingOnReadOnlyFiles : True > EnableInsecureGuestLogons : True > EnableLargeMtu : True > EnableLoadBalanceScaleOut : True > EnableMultiChannel : True > EnableSecuritySignature : True > ExtendedSessionTimeout : 1000 > FileInfoCacheEntriesMax : 32768 > FileInfoCacheLifetime : 60 > FileNotFoundCacheEntriesMax : 32768 > FileNotFoundCacheLifetime : 60 > KeepConn : 600 > MaxCmds : 32768 > MaximumConnectionCountPerServer : 64 > OpLocksDisabled : False > RequireSecuritySignature : False > SessionTimeout : 60 > UseOpportunisticLocking : True > WindowSizeThreshold : 1 > > PS C:\Users > > > Any advice on the optimal SMB tuning parameters and their respective values > for both the SMB client and server sides would be highly appreciated. This > is necessary to address the issue at hand and to achieve optimal network > throughput on an SMB share that contains a large number of small files. > Please provide recommendations on any other settings that may be relevant > in this context. > > Your assistance in this matter would be highly valued. > > Thanks & regards, > Swapnil M.Hi Swapnil, I guess this is the classic file copy performance problem. The OS overhead for storing a small file is the same as for a large file, particularly if there are classic hard disks involved. The hard disk mechanics sets physical limits for what is achievable. I had a look at the Hitachi Vantara HNAS specifications, and Samba isn't mentioned even once in that documentation. So your best bet is Hitachi support, or the Hitachi community forums. Best regards, Peter