Hello, I'm testing the performance of Samba share when a windows client copy a directory that contains many small files (8-18KB). From the windows client during the transfer the speed goes up and down to 0KB/sec, so it's not stable. I have run the same test on a Windows server share (on the same hardware as the Samba share) and the transfer speed is stable, about 404KB/sec which is three time quicker then the overage speed in the Samba share. So is there any tuning that I can add to the smb.conf to speed up the performance when copying a directory that contains many small files (8-18KB)? -- Regards Andrea Cucciarre'
On Mon, 15 Jun 2020, Andrea Cucciarre' via samba wrote:> I have run the same test on a Windows server share (on the same hardware as > the Samba share) and the transfer speed is stable, about 404KB/sec which is > three time quicker then the overage speed in the Samba share.Are you doing it on the same hardware, same drive, same everything? Even if you are, keep in mind that one of the primary differences would be the speed of your underlying filesystem that Samba is using. Optimize that filesystem for small files. The usual solution is to go with a filesystem that has strong metadata cache control. Good candidates for this are XFS and ZFS (also Lustre on the high-end for clusters). Those all offer some level of metadata cache control. Also, if you're using Linux, consider using dm-cache which supercharges your read caching.> So is there any tuning that I can add to the smb.conf to speed up the > performance when copying a directory that contains many small files > (8-18KB)?I can't help with the Samba tuning part, hopefully others have decent suggestions there. However, another thing to consider is using a streaming archiver to transfer the files. Ie.. tar into a pipe over netcat works very quickly. That only works if you are doing a full send every time. Otherwise, see my first suggestion. -Swift PARSEC Group is proud to be recognized through both the Colorado Department of Higher Education and the U.S. Department of Veterans Affairs as an authorized training organization and is now accepting GI Bill funding. Follow @PARSECgroup on Twitter Find PARSEC Group on Facebook http://www.facebook.com/ParsecGroup and LinkedIn http://www.linkedin.com/company/3041461?trk=tyah **The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message is privileged and confidential. If you have received this communication in error, please notify us via e-mail and delete the original message. The opinions, conclusions and other information in this message do not necessarily represent the views and/or opinions of PARSEC Group.**
Hello Swift, the filesystem is just ZFS on flash storage, so filesystem and storage shouldn't be an issue. That's why I suspect the bottleneck is Samba. Regards Andrea Cucciarre' On 6/15/2020 5:57 PM, Swift Griggs via samba wrote:> On Mon, 15 Jun 2020, Andrea Cucciarre' via samba wrote: >> I have run the same test on a Windows server share (on the same >> hardware as the Samba share) and the transfer speed is stable, about >> 404KB/sec which is three time quicker then the overage speed in the >> Samba share. > > Are you doing it on the same hardware, same drive, same everything? > Even if you are, keep in mind that one of the primary differences > would be the speed of your underlying filesystem that Samba is using. > Optimize that filesystem for small files. The usual solution is to go > with a filesystem that has strong metadata cache control. > > Good candidates for this are XFS and ZFS (also Lustre on the high-end > for clusters). Those all offer some level of metadata cache control. > Also, if you're using Linux, consider using dm-cache which > supercharges your read caching. > >> So is there any tuning that I can add to the smb.conf to speed up the >> performance when copying a directory that contains many small files >> (8-18KB)? > > I can't help with the Samba tuning part, hopefully others have decent > suggestions there. However, another thing to consider is using a > streaming archiver to transfer the files. Ie.. tar into a pipe over > netcat works very quickly. That only works if you are doing a full > send every time. Otherwise, see my first suggestion. > > -Swift > > > PARSEC Group is proud to be recognized through both the Colorado > Department of Higher Education and the U.S. Department of Veterans > Affairs as an authorized training organization and is now accepting GI > Bill funding. > > Follow @PARSECgroup on Twitter > > Find PARSEC Group on Facebook http://www.facebook.com/ParsecGroup > and LinkedIn http://www.linkedin.com/company/3041461?trk=tyah > > **The information contained in this e-mail message is intended only > for the personal and confidential use of the recipient(s) named above. > This message is privileged and confidential. If you have received this > communication in error, please notify us via e-mail and delete the > original message. The opinions, conclusions and other information in > this message do not necessarily represent the views and/or opinions of > PARSEC Group.** > >