Hello Michael,
I understand that it's not possible to get debug logs/traces from the
server.
However is it possible to collect network traces from one of a client, once
the client starts reporting slow-ness?
This way we would know what sort of lease requests are being sent as a
starter to look for issues.
--Vinit.
On Tue, Sep 23, 2025 at 4:57?PM Michael Tokarev via samba <
samba at lists.samba.org> wrote:
> Hi!
>
> We've a busy server (a member of windows AD) which is read-only,
> serving a read-only application (executable files, dlls, etc)
> to a big number of windows clients.
>
> The whole share is read-only, this is important (I think).
>
> The clients requests the same files, - like the main executable,
> main set of DLLs etc. All of them are always open by a number of
> clients.
>
> The server is busy, - 2000..3000 concurrent connections all the time.
>
> And sometimes, out of the sudden, clients can't open the files anymore,
> and all operations becomes very slow. At the same time, there are
> numerous, multiple messages on the server, like this:
>
> [2025/09/08 15:14:11.011301, 1, pid=2619092]
> source3/smbd/smb2_oplock.c:356(lease_timeout_handler)
> lease break timed out for file ekis/xml_upload.exe -- replying anyway
> [2025/09/08 15:30:28.212243, 1, pid=2619092]
> source3/smbd/smb2_oplock.c:356(lease_timeout_handler)
> lease break timed out for file ekis/prtcontract.exe -- replying anyway
> [2025/09/08 15:40:24.427341, 1, pid=2619092]
> source3/smbd/smb2_oplock.c:356(lease_timeout_handler)
> lease break timed out for file ekis/xml_upload.exe -- replying anyway
> [2025/09/08 15:44:12.739445, 1, pid=2619092]
> source3/smbd/smb2_oplock.c:356(lease_timeout_handler)
> lease break timed out for file ekis/xml_upload.exe -- replying anyway
> [2025/09/08 15:55:56.191710, 1, pid=2619092]
> source3/smbd/smb2_oplock.c:356(lease_timeout_handler)
> lease break timed out for file ekis/xml_upload.exe -- replying anyway
>
> This happened maybe 4 times during several months with samba 4.21.x.
> In all cases I was able to solve this issue by just restarting smbd -
> after a restart, things becomes normal again for weeks.
>
> However, if I upgrade to samba 4.22, this starts happening almost
> immediately, and multiple users starts complaining about the file
> server being in non-working state.
>
> 4.23 shows the same bad behavior like 4.22.
>
> This is a big production environment, so it's very difficult to
> experiment. The server is also quite busy, with multiple new sessions
> per second, so turning on any tracing or debugging is almost pointless,
> since it's hardly possible to get hold of events in single session.
>
> At one time, when we used samba 4.21, this happened again, and just
> restart of smbd didn't fix the issue. So after number of attempts,
> I disabled smb2 oplocks entirely (which required restarting of all
> clients, since without restart they complaining "the server can not
> complete the requested operation" - my guess it's because of the
> change being incompatible with existing sessions. After the change
> in configuration and restarting of all clients (it was really painful
> because the number of clients is huge), the problem happened again,
> now with oplocks:
>
> [2025/09/16 16:37:35.369774, 0, pid=3784914]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file ekis/d2kwutil.plx -- replying anyway
> [2025/09/16 16:37:36.003843, 0, pid=3784925]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file ekis/requpload.exe -- replying anyway
> [2025/09/16 16:37:36.079860, 0, pid=3784922]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file ekis/oc_rolls_action.fmx -- replying anyway
> [2025/09/16 16:37:37.165932, 0, pid=3784909]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file ekis/strakh.plx -- replying anyway
> [2025/09/16 16:37:46.907498, 0, pid=3784934]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file conf/tnsnames.ora -- replying anyway
> [2025/09/16 16:37:49.321069, 0, pid=3784928]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file ekis/attachfile.exe -- replying anyway
> [2025/09/16 16:37:59.104781, 0, pid=3784914]
> source3/smbd/smb2_oplock.c:809(oplock_timeout_handler)
> Oplock break failed for file ekis/consforremoving.exe -- replying anyway
>
> so I finally turned on fake oplocks, which finally helped.
>
> Now, obviously, I can't return into the previous configuration
> without restarting all clients, so I can't test any changes.
>
> I tried to replicate this environment multiple times, but in all our
> local tests it just works, without any of this mess.
>
>
> The questions.
>
> 1. Maybe there's some guess about what's going on here, already
exists?
>
> 2. With read-only everything, shuldn't there be no exclusive (op)locks
> at all, - the ones which has to be broken - to begin with?
> With unix flock/fcntl stuff, you can't request an exclusive lock
> without opening file for writing, and here, writing is never granted.
>
> 3. Any suggestion how can I debug this one, without breaking all
> clients which are already connected? I can revert the smb2 leases
> disabling change overnight - but I need a way to make the whole thing
> working again if it shows the same locked-out situation, and if this
> will require disabling smb2 leases again, I'll have another very
> painful rebooting-everything experience, which I'd love to avoid..
>
> With all this -- big number of clients, busy server - what is the way
> to debug the issue anyway? Which debugging knobs can I turn on?
>
> Smb.conf is below.
>
> Thank you!
>
> /mjt
>
> [global]
> netbios name = ekis-files
> realm = RGS.RU
> workgroup = RGSMAIN
> server string = Samba %v (%h)
>
> server role = member server
>
> # default (misc, aux) range
> idmap config * : range = 3000-3999
> idmap config * : backend = tdb
>
> idmap config RGSMAIN : range = 1000000-1999999
> idmap config RGSMAIN : backend = rid
>
> # log file = /var/log/samba/log.%m
> max log size = 1000
> logging = file
> debug pid = yes
> log level = 1
>
> hostname lookups = yes
> name resolve order = host
>
> deadtime = 10 ; minutes
>
> acl allow execute always = true
> map archive = no
> nt acl support = no
> store dos attributes = no
> smb1 unix extensions = no
> durable handles = no
>
> smb2 leases = no
> fake oplocks = yes
>
> load printers = no
> disable netbios = yes
> smb ports = 445
>
> [files]
> comment = EKIS RDS
> path = /share/files
> msdfs root = yes
> guest ok = yes
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
>
--
One Ring to rule them all, One Ring to find them,
One Ring to bring them all and in the darkness bind them
In the Land of Mordor where the Shadows lie.