samba - Jan 2023 - oplocks, kernel oplocks, kernel share modes, .. - how it all works?

20.01.2023 14:49, Ralph Boehme via samba wrote:
> I don't have that much time to explain this in glory detail, but here
are a few pointers.
Thank you very much for this write-up/summary. It's exactly what was needed!
> There are three different apsects:
> 
> - locking in the Windows sharemode flavor
> - file change notify
> - client side caching via oplocks/leases
> 
> File change notify is fully supported and will detect local changes via
inotify on Linux. Iirc there are some caveats like recursive change notify but
> that doesn't seem to be a production problem as Windows clients
typically don't make use of this feature.
This is the most important (I think) part, - the change notify.  Yes I
understand
there are issues with recursive inotify even without windows or extra protocols
involved.

Surprisingly for me, I haven't even looked at inotify in samba sources
before asking
this (I can't know the "support status" for it just by looking at
the sources, though).

And indeed, there's quite some code in samba/smbd to hook inotify.  And it
is actually
what I suspected long before this topic, - eg windows explorer immediately
refreshes
the directory listing once something changes on the samba server side, - it
should be
some FAM, and linux has inotify, there's nothing else like this (yes I know
dnotify too).
> Client side caching (oplocks/leases): somewhat limitted support by enabled
"kernel oplocks". Kernel oplocks don't work with SMB2 Leases, so
you're
> restricted to SMB1 oplocks. Cf man smb.conf "kernel oplocks".
And SMB1 oplocks does not work when SMB2+ protocol is negotiated/used, right?
This one I'll take a look at the source.

And here there's some interesting stuff happening. Again, I'll try to
set up some
testbed for tracing a bit later.

The thing is that smbd immediately detects the change of a linux file, and
immediately notifies windows about this change, and windows asks to re-read
the directory.

But the lease which is granted to windows on that file.. stays. And it is even
funnier - I open this file in a text viewer (read-only), win asks shared lease
for it, next I modify the file on linux, close it on windows and re-open it on
windows. And it shows me the old cached contents of this file, while in the
directory listing which windows displayed already even before second open,
this file has zero size.

It is either that samba does not try to break this lease even if it detects the
file change (ie, change notify isn't hooked into the file lease code), or
windows
ignores this. From the plain strace of smbd, it looks like there's no
"join" of
file change monitoring and leases or it doesn't work.

Is this supposed to work with current samba?
> Locking (sharemodes): we used to leverage Linux support for mandatory
flock() locks (LOCK_MAND) to "store" and let the kernel enforce access
> restrictions for local access, but unfortunately the kernel support got
removed because it was buggy:
> 
>
<https://github.com/torvalds/linux/commit/f7e33bdbd6d1bdf9c3df8bba5abcf3399f957ac3>
> 
> We still have VFS hooks for this but iirc the only filesystem that supports
this via the VFS interface is GPFS.
Oh.  Heh. Fun stuff it was :)
> Hope this helps somewhat.
It *definitely* helps, thank you very much Ralf!

I'll try to dig further to understand what's actually going on there.
It looks like this really deserves some attention, because the problem
is quite common but largely unknown.

And it deserves a separate wiki page too, at least :)

/mjt

Howdy!

On 1/20/23 14:50, Michael Tokarev via samba wrote:
>> Client side caching (oplocks/leases): somewhat limitted support by 
>> enabled "kernel oplocks". Kernel oplocks don't work with
SMB2 Leases,
>> so you're restricted to SMB1 oplocks. Cf man smb.conf "kernel
oplocks".
> 
> And SMB1 oplocks does not work when SMB2+ protocol is negotiated/used, 
> right?
No, they work just fine even with SMB2 and newer. Leases just offer a 
richer set of caching operations and are thus more efficient for certain 
workloads. Enabling "kernel oplocks" makes it worse as they don't
work
with shared/read oplocks, only for exclusive/read+write ones.
> But the lease which is granted to windows on that file.. stays. And it 
> is even
> funnier - I open this file in a text viewer (read-only), win asks shared 
> lease
> for it, next I modify the file on linux, close it on windows and re-open 
> it on
> windows. And it shows me the old cached contents of this file, while in the
> directory listing which windows displayed already even before second open,
> this file has zero size.
As said, change notify is something else entirely then caching via 
oplocks or leases. Change notify is a pure notification system while 
oplocks requires blocking oplock contenders from accessing a file until 
the client holding the oplock downgrades his oplock or times out.
> It is either that samba does not try to break this lease even if it 
> detects the
> file change (ie, change notify isn't hooked into the file lease code), 
Enable "kernel oplocks", then it should work.
> And it deserves a separate wiki page too, at least :)
That would be awesome! Contributions welcome! :)

-slow

-- 
Ralph Boehme, Samba Team                 https://samba.org/
SerNet Samba Team Lead      https://sernet.de/en/team-samba

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.samba.org/pipermail/samba/attachments/20230120/3deaeeeb/OpenPGP_signature.sig>