samba - Jan 2018 - stat(2) cache, samba3 versus samba4

Short version:

Is there a way to enforce use of this cache:
  ./source3/smbd/statcache.c
I noticed it is absent from the ./source4 directory

I limited 
  server max protocol = SMB2
but that didn't do it.

Long story:
-----------

My samba server (4.2.14) hosts a couple of game install trees for a 
Windows 7 computer.

Some of the games cause storms of stat(2) system calls, here is a
partial strace:

newfstatat(35, "i50145t.bmp", {st_mode=S_IFREG|0664, st_size=35344,
...}, 0) = 0
newfstatat(35, "i01014o.BMP", {st_mode=S_IFREG|0664, st_size=21536,
...}, 0) = 0
newfstatat(35, "i50027g.bmp", {st_mode=S_IFREG|0664, st_size=28416,
...}, 0) = 0
newfstatat(35, "i02017g.BMP", {st_mode=S_IFREG|0664, st_size=11584,
...}, 0) = 0
newfstatat(35, "i99509o.bmp", {st_mode=S_IFREG|0664, st_size=73080,
...}, 0) = 0

Depending on the game they either slow down during startup due to
stat(2)ing all their files (toaw4), or the game becomes unusable
because checks continue during gameplay (some older versions of DSC
flight sims).  There are a lot more calls than needed to actually load
the game.  Whatever the reason for all these checks for changed files
is, they expect them to be "cheap" because they would come out the FS
buffer cache of the local OS if it wasn't a network drive.

Is there a way to cause the cache to be used for a specific client, or
a specific share?

To clarify, I know it will still be slow since instead of locally
cached system calls into the Windows kernel it will still have a
network turnaround.  However, right now I am doing actual system calls
on the server side, and exchanging those for pure userland replies
will help.

I also know iSCSI would help but it's a pain.

Thank you
    Martin
-- 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Martin Cracauer <cracauer at cons.org>   http://www.cons.org/cracauer/

On Fri, Jan 05, 2018 at 09:02:06AM -0500, Martin Cracauer via samba
wrote:
> Short version:
> 
> Is there a way to enforce use of this cache:
>   ./source3/smbd/statcache.c
> I noticed it is absent from the ./source4 directory
> 
> I limited 
>   server max protocol = SMB2
> but that didn't do it.
> 
> Long story:
> -----------
> 
> My samba server (4.2.14) hosts a couple of game install trees for a 
> Windows 7 computer.
> 
> Some of the games cause storms of stat(2) system calls, here is a
> partial strace:
> 
> newfstatat(35, "i50145t.bmp", {st_mode=S_IFREG|0664,
st_size=35344, ...}, 0) = 0
> newfstatat(35, "i01014o.BMP", {st_mode=S_IFREG|0664,
st_size=21536, ...}, 0) = 0
> newfstatat(35, "i50027g.bmp", {st_mode=S_IFREG|0664,
st_size=28416, ...}, 0) = 0
> newfstatat(35, "i02017g.BMP", {st_mode=S_IFREG|0664,
st_size=11584, ...}, 0) = 0
> newfstatat(35, "i99509o.bmp", {st_mode=S_IFREG|0664,
st_size=73080, ...}, 0) = 0
> 
> Depending on the game they either slow down during startup due to
> stat(2)ing all their files (toaw4), or the game becomes unusable
> because checks continue during gameplay (some older versions of DSC
> flight sims).  There are a lot more calls than needed to actually load
> the game.  Whatever the reason for all these checks for changed files
> is, they expect them to be "cheap" because they would come out
the FS
> buffer cache of the local OS if it wasn't a network drive.
> 
> Is there a way to cause the cache to be used for a specific client, or
> a specific share?
> 
> To clarify, I know it will still be slow since instead of locally
> cached system calls into the Windows kernel it will still have a
> network turnaround.  However, right now I am doing actual system calls
> on the server side, and exchanging those for pure userland replies
> will help.
The stat cache is always used in smbd, which I presume if what
is serving your files (not the s4:ntfs "discontinued" file server).

It's use is limited (I wrote it :-), once a file has been looked
up from the Windows client it's used to re-canonicalize the name
from case-insensitive Windows to case-sensitive UNIX (e.g. if
Windows looks up FILE.BMP but the file stored on disk is file.BMP
the stat cache stores that mapping and will be used to map an
incoming name of FILE.BMP -> file.BMP to avoid smbd having to
do a directory search.

This, while useful, it probably not what you were expecting here.

Jeremy.