samba - Nov 2023 - understanding stat cache

Hello Ralph,

Thanks for that hint about case sensitivity's performance penalty.

For clarifaction: The user is doing mainly reads, so does the "create"
you mention also cover opening/reading files? If only _creation_ of files is
suffering from that we probably have some other/further performance issue.

We have gpfs, which does not offer a case-insensitive mode, neither does the
vfs_gpfs module.

Uli


-----Urspr?ngliche Nachricht-----
Von: Ralph Boehme <slow at samba.org> 
Gesendet: Montag, 13. November 2023 17:22
An: Ulrich Sibiller <ulrich.sibiller at atos.net>; samba at
lists.samba.org
Betreff: Re: [Samba] understanding stat cache

Hello Ulrich,

On 11/10/23 13:47, Ulrich Sibiller via samba wrote:
> We have a user that switched from Linux to Windows with his 
> engineering software. Previously he was using NFS to access data and 
> there were no performance complaints.
> 
> Now, with Windows, the same procedures take minutes instead of 
> seconds.
the classic workload where Samba performance sucks is when applications create
many files in directories. In Samba this involves scanning the parent directory
for every file as part of the case insensitive implementation on-top of a case
sensitive filesystem.

There are a few things that can be done here, eg man smb.conf

        default case = upper/lower
            controls what the default case is for new
            filenames (ie. files that don't currently exist
            in the filesystem). Default lower. IMPORTANT
            NOTE: As part of the optimizations for
            directories containing large numbers of files,
            the following special case applies. If the
            options case sensitive = yes, preserve case = No,
            and short preserve case = No are set, then the
            case of all incoming client filenames, not just
            new filenames, will be modified. See additional
            notes below.

Ideally you'd have a filesystem that is case insensitive (ext4 added support
recently) or a filesystem that is supported by Samba via a VFS hook to lookup
files.
> So I am wondering why the first two lookups always fail and/or 
> obviously are never stored to the stat cache?  Is there a depth limit 
> in the stat cache?
the name stat cache is a bit misleading, only caches filenames, not stat info.
It only helps if clients repeatedly access existing files with a wrong case.

-slow

-- 
SerNet Samba Team Lead          https://samba.plus/
Samba Team Member                https://samba.org/
SAMBA+ packages                 https://samba.plus/
Worldwide Samba Support, Consulting and Development

Hi,

On 11/15/23 09:33, Ulrich Sibiller wrote:
> Thanks for that hint about case sensitivity's performance penalty.
> 
> For clarifaction: The user is doing mainly reads, so does the
> "create" you mention also cover opening/reading files?
no.

Is it a metadata heave small file workload? That will likely be somewhat 
slower compared to Windows, but not in the order of magnitudes.

Otoh there were a few regression in the gpfs module in recent months, so 
it might make some sense digging deeper why your workload performs so 
bad in this setup.
> If only
> _creation_ of files is suffering from that we probably have some
> other/further performance issue.
> 
> We have gpfs, which does not offer a case-insensitive mode, neither
> does the vfs_gpfs module.
the gpfs VFS module actually does support looking up files in a 
case-sensitive manner by using gpfs_get_realfilename_path() from libgpfs.

-slow

-- 
SerNet Samba Team Lead          https://samba.plus/
Samba Team Member                https://samba.org/
SAMBA+ packages                 https://samba.plus/
Worldwide Samba Support, Consulting and Development

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/samba/attachments/20231115/10b801b5/OpenPGP_signature.sig>