samba - Feb 2023 - long delays with file enumeration & listing in large data storage environment

Hey samba-people,

just our anecdotal findings and advice:

Directory listings from our MacOS clients took a big hit when we 
switched from AFP shares and SMB shares from our old Mac OS Server to 
the new debian fileserver with samba shares.

Especially opening big directories with AFP is blazingly fast, while in 
samba it takes forever. E. g. we have a directory with 37k 
subdirectories and 245k files in one level. Opening this dir on a Mac 
via a samba share take approx. 9 minutes. Our advice was to reduce the 
amount of files in a single directory and create cronjobs that 
automatically moves files into subdirectories.

Any improvement on samba server would be greatly appreciated!

Thanks, Matthias K?hne.

Am 08.02.23 um 07:04 schrieb Ralph Boehme via samba:
> On 2/8/23 02:42, Jeremy Allison via samba wrote:
>> I would suggest you try with Samba 4.17.latest to see if you
>> find an improvement.
>
> while this is generally a good advice :) I don't think this will help 
> in this case. Directory enumeration didn't really see any changes and 
> the OP mentions that enumeration was fast before joining to the 
> domain. This is quite unexpected and this is the point where I would 
> start investigating.
>
> -slow
>
>
-- 
Senior Webentwickler
Datenschutzbeauftragter

Ellerhold Aktiengesellschaft
Friedrich-List-Str. 4
01445 Radebeul

Telefon: +49 (0) 351 83933-61
Web: www.ellerhold.de
Facebook: www.facebook.com/ellerhold.gruppe
Instagram: www.instagram.com/ellerhold.gruppe
Twitter: https://twitter.com/EllerholdGruppe

Amtsgericht Dresden / HRB 23769
Vorstand: Stephan Ellerhold, Maximilian Ellerhold
Vorsitzender des Aufsichtsrates: Frank Ellerhold



---Diese E-Mail und Ihre Anlagen enthalten vertrauliche Mitteilungen. Sollten
Sie nicht der beabsichtigte Adressat sein, so bitten wir Sie um Mitteilung und
um sofortiges l?schen dieser E-Mail und der Anlagen.

Unsere Hinweise zum Datenschutz finden Sie hier:
http://www.ellerhold.de/datenschutz/

This e-mail and its attachments are privileged and confidential. If you are not
the intended recipient, please notify us and immediately delete this e-mail and
its attachments.

You can find our privacy policy here: http://www.ellerhold.de/datenschutz/

On 2/8/23 09:29, Matthias K?hne | Ellerhold Aktiengesellschaft via samba 
wrote:
> Directory listings from our MacOS clients took a big hit when we
> switched from AFP shares and SMB shares from our old Mac OS Server to
> the new debian fileserver with samba shares.
> 
> Especially opening big directories with AFP is blazingly fast, while in
> samba it takes forever. E. g. we have a directory with 37k
> subdirectories and 245k files in one level. Opening this dir on a Mac
> via a samba share take approx. 9 minutes. Our advice was to reduce the
> amount of files in a single directory and create cronjobs that
> automatically moves files into subdirectories.
Apple designed a few SMB2 protocol extensions that improve directory 
listing performance and Samba implements those in vfs_fruit. 
Additionally latest macos since a few releases has additinoal 
improvements that should improve listing regardless of these extensions.


-slow

-- 
Ralph Boehme, Samba Team                 https://samba.org/
SerNet Samba Team Lead      https://sernet.de/en/team-samba

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/samba/attachments/20230208/8b652d06/OpenPGP_signature.sig>