samba - Oct 2020 - Performance Question: Lots of Small Files vs One Large File

In an architecture where we have Samba running on RHEL exposing shares to
Windows, when we have 1 large 1GB file, the write performance to storage is
very fast, even over distances of 5000 miles. However, even writes to local
Samba servers, with 100 10MB files being copied onto a shared drive,
Windows Explorer is MUCH slower. I don't know if it's really Samba, but
more than likely Windows.

Does anyone on channel have experience with copying multiple small files
onto shares? Are there any ways to make copying files onto a share faster?

Thanks

-- 

BOB BUCK
SENIOR PLATFORM SOFTWARE ENGINEER

SKIDMORE, OWINGS & MERRILL
7 WORLD TRADE CENTER
250 GREENWICH STREET
NEW YORK, NY 10007
T  (212) 298-9624
ROBERT.BUCK at SOM.COM

We had a similar issue that I fought for a while, and the issue was with 
the NIC Card settings on the PC's, Turn off energy efficient ethernet, 
Green Ethernet, set wake on magic packet? = disabled, turn off any other 
power saving modes on the NIC card.

You can also try adjusting MTU settings, but leave it at the default of 
1500 on both ends of the connection to start.

Take some base benchmarks for transferring a 10MB file, a 100 MB file 
and a 1 GB file without changing anything, then make those changes on 
one PC, then test again with the same files, and reply back with the 
numbers.

Chris

On 10/6/2020 12:49 PM, Robert Buck via samba wrote:
> In an architecture where we have Samba running on RHEL exposing shares to
> Windows, when we have 1 large 1GB file, the write performance to storage is
> very fast, even over distances of 5000 miles. However, even writes to local
> Samba servers, with 100 10MB files being copied onto a shared drive,
> Windows Explorer is MUCH slower. I don't know if it's really Samba,
but
> more than likely Windows.
>
> Does anyone on channel have experience with copying multiple small files
> onto shares? Are there any ways to make copying files onto a share faster?
>
> Thanks
>
-- 
Christopher Wensink
IS Administrator
Five Star Plastics, Inc
1339 Continental Drive
Eau Claire, WI 54701
Office:  715-831-1682
Mobile:  715-563-3112
Fax:  715-831-6075
cwensink at five-star-plastics.com
www.five-star-plastics.com

Am 10/6/20 um 7:49 PM schrieb Robert Buck via samba:
> In an architecture where we have Samba running on RHEL exposing shares to
> Windows, when we have 1 large 1GB file, the write performance to storage is
> very fast, even over distances of 5000 miles. However, even writes to local
> Samba servers, with 100 10MB files being copied onto a shared drive,
> Windows Explorer is MUCH slower. I don't know if it's really Samba,
but
> more than likely Windows.
> 
> Does anyone on channel have experience with copying multiple small files
> onto shares? Are there any ways to make copying files onto a share faster?
to save you a lot rocking back and forth, yes, small files workloads
will be slower. The smaller, the slower... :)

-slow

-- 
Ralph Boehme, Samba Team                https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/
GPG-Fingerprint   FAE2C6088A24252051C559E4AA1E9B7126399E46

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/samba/attachments/20201006/ad99cac7/signature.sig>

Is this a protocol issue? A decade ago I saw writes to small files less
than 16k were awful, because the cost of opening and other file ops dwarfed
writing actual content. So very small files should be slow, but these files
are 10mb each.

Or is this a Windows issue? If so, what's causing the problem?

Just trying to understand.

On Tue, Oct 6, 2020 at 2:39 PM Ralph Boehme <slow at samba.org> wrote:
> Am 10/6/20 um 7:49 PM schrieb Robert Buck via samba:
> > In an architecture where we have Samba running on RHEL exposing shares
to
> > Windows, when we have 1 large 1GB file, the write performance to
storage
> is
> > very fast, even over distances of 5000 miles. However, even writes to
> local
> > Samba servers, with 100 10MB files being copied onto a shared drive,
> > Windows Explorer is MUCH slower. I don't know if it's really
Samba, but
> > more than likely Windows.
> >
> > Does anyone on channel have experience with copying multiple small
files
> > onto shares? Are there any ways to make copying files onto a share
> faster?
>
> to save you a lot rocking back and forth, yes, small files workloads
> will be slower. The smaller, the slower... :)
>
> -slow
>
> --
> Ralph Boehme, Samba Team                https://samba.org/
> Samba Developer, SerNet GmbH   https://sernet.de/en/samba/
> GPG-Fingerprint   FAE2C6088A24252051C559E4AA1E9B7126399E46
>
> --
BOB BUCK
SENIOR PLATFORM SOFTWARE ENGINEER

SKIDMORE, OWINGS & MERRILL
7 WORLD TRADE CENTER
250 GREENWICH STREET
NEW YORK, NY 10007
T  (212) 298-9624
ROBERT.BUCK at SOM.COM