samba - Jun 2024 - missing msdfs referrals from samba directory listing: wrong order in smbd_dirptr_get_entry()?

Hi!

For quite some time I'm trying to find what's going on with MSDFS
referrals.
Samba version is 4.19.6.

We've global host msdfs = yes (the default anyway), and for a share in
question,
msdfs root = yes.  testparam confirms the settings.

There's a symlink created in the root dir of the share, which points to the
same
server but different path:

   dfstest => msdfs:thissrv/othershare/subdir

However, when opening the directory in question from client, this name is not
shown in the listing.  Samba skips this name from the listing when looping over
readdir entries:

[2024/06/06 19:02:40.521314, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:868(openat_pathref_fsp_nosymlink)
   openat_pathref_fsp_nosymlink: path_in=dfstest
[2024/06/06 19:02:40.521339,  5, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:77(fsp_new)
   fsp_new: allocated files structure (9 used)
[2024/06/06 19:02:40.521372, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:978(openat_pathref_fsp_nosymlink)
   openat_pathref_fsp_nosymlink: SMB_VFS_OPENAT(., dfstest, RESOLVE_NO_SYMLINKS)
returned 40 Too many levels of symbolic links =>
NT_STATUS_OBJECT_PATH_NOT_FOUND
[2024/06/06 19:02:40.521437, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/mangle_hash2.c:415(is_mangled)
   is_mangled dfstest ?
[2024/06/06 19:02:40.521466, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/mangle_hash2.c:354(is_mangled_component)
   is_mangled_component dfstest (len 7) ?
[2024/06/06 19:02:40.521493,  5, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:77(fsp_new)
   fsp_new: allocated files structure (10 used)
[2024/06/06 19:02:40.521517, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:115(fsp_bind_smb)
   fsp_bind_smb: INTERNAL_OPEN_ONLY, skipping smbXsrv_open
[2024/06/06 19:02:40.521542,  5, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:179(file_new)
   file_new: new file fnum [invalid value]
[2024/06/06 19:02:40.521569, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:2198(file_name_hash)
   file_name_hash: /ws/ws/. hash 0x3eac6da4
[2024/06/06 19:02:40.521608, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/open.c:930(fd_openat)
   fd_openat: name ., flags = 0400000 mode = 00, fd = 38
[2024/06/06 19:02:40.521669,  5, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:1985(file_free)
   file_free: freed files structure 0 (9 used)
[2024/06/06 19:02:40.521711, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:1172(openat_pathref_fsp_nosymlink)
   openat_pathref_fsp_nosymlink: SMB_VFS_OPENAT() failed: No such file or
directory
[2024/06/06 19:02:40.521781,  5, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/files.c:1985(file_free)
   file_free: freed files structure 0 (8 used)
[2024/06/06 19:02:40.521810,  3, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/dir.c:750(smbd_dirptr_get_entry)
   smbd_dirptr_get_entry: Could not open dfstest:
NT_STATUS_OBJECT_NAME_NOT_FOUND
[2024/06/06 19:02:40.521845, 10, pid=1173240, effective(1000, 1000), real(1000,
0)] source3/smbd/dir.c:568(smbd_dirptr_get_entry)
   smbd_dirptr_get_entry: dir [.] dirptr [0x564f0a10ed10] offset [9] => dname
[(finished)]
[2024/06/06 19:02:40.521878, 10, pid=1173240, effective(1000, 1000), real(1000,
0), class=smb2]
source3/smbd/smb2_query_directory.c:188(smbd_smb2_request_find_done)
   smbd_smb2_request_find_done: out_output_buffer.length = 942
[2024/06/06 19:02:40.521904, 10, pid=1173240, effective(1000, 1000), real(1000,
0), class=smb2] source3/smbd/smb2_server.c:3916(smbd_smb2_request_done_ex)
   smbd_smb2_request_done_ex: mid [12841] idx[5] status[NT_STATUS_OK] body[8]
dyn[yes:942] at source3/smbd/smb2_query_directory.c:193

This is consistent for smbclient and for windows10 and windows11 clients.

However, when specifying whole path in question, in form \\thisserver\dfstest\,
it works, - samba correctly returns referral and the client handles it:

$ smbclient //thissrv/share
Password for...:
smb: \> dir
   .                                   D        0  Thu Jun  6 19:10:50 2024
   ..                                  D        0  Thu Jun  6 19:10:50 2024
smb: \> cd dfstest
smb: \dfstest\> dir
   .                                   D        0  Wed Apr  2 21:56:01 2008
   ..                                  D        0  Mon Sep 18 11:25:54 2023
   doc                                 D        0  Mon Jun 20 13:37:17 2005
...other entries..
smb: \dfstest\> cd ..
smb: \> _


So it looks like only the file listing does not work.

But the fun thing is that it is omitted in the listing *sometimes* (most of the
time), - At one point I did see it in the listing, so I concluded it works fine
and forgot about it, until users started complaining they don't see their
dirs.

I'm certain nothing had changed since that time, - exactly the same samba
version
and the same config, but we had a couple of reboots in between.

Reading source3/smbd/dir.c:smbd_dirptr_get_entry() I can't see where it can
work,
ever.  Because it does openat_pathref_fsp_lcomp() first, which never works on a
symlink not pointing to a real file (which is how msdfs symlink looks like),
that
one return !NT_STATUS_OK() (NT_STATUS_OBJECT_NAME_NOT_FOUND in the above debug
log), and whole thing ends up in there.  is_msdfs_link() is checked later.

If I move is_msdfs_link() check right before openat_pathref_fsp_lcomp() call, it
seems to work fine.

Should such a move be done in the actual code?

Thanks,

/mjt--
GPG Key transition (from rsa2048 to rsa4096) since 2024-04-24.
New key: rsa4096/61AD3D98ECDF2C8E  9D8B E14E 3F2A 9DD7 9199  28F1 61AD 3D98 ECDF
2C8E
Old key: rsa2048/457CE0A0804465C5  6EE1 95D1 886E 8FFB 810D  4324 457C E0A0 8044
65C5
Transition statement: http://www.corpit.ru/mjt/gpg-transition-2024.txt

06.06.2024 19:33, Michael Tokarev via samba wrote:
> For quite some time I'm trying to find what's going on with MSDFS
referrals.
> Samba version is 4.19.6.
..
> However, when opening the directory in question from client, this name is
not
> shown in the listing.? Samba skips this name from the listing when looping
over
> readdir entries:
So, the same config just works in samba-4.20.  There has been a quite few
changes in
this area in 4.20, but the order of checks in smbd_dirptr_get_entry() didn't
actually
change (but the code is different), -- looks like the change happened in
smbd_dirptr_get_entry
itself.

msdfs is broken in 4.19.  It worked fine in 4.18.

Is no one using msdfs?  I wonder why I was the first to discover this?

Thanks,

/mjt
-- 
GPG Key transition (from rsa2048 to rsa4096) since 2024-04-24.
New key: rsa4096/61AD3D98ECDF2C8E  9D8B E14E 3F2A 9DD7 9199  28F1 61AD 3D98 ECDF
2C8E
Old key: rsa2048/457CE0A0804465C5  6EE1 95D1 886E 8FFB 810D  4324 457C E0A0 8044
65C5
Transition statement: http://www.corpit.ru/mjt/gpg-transition-2024.txt