CaptainTrips28
2023-Mar-17 18:43 UTC
[Samba] Empty folder deletion issue - Samba 4.15 thru 4.18
So the filesystem thing is a bit detailed. It's Windows client ----> RHEL
Samba server ----> Hadoop data cluster via NFS mount.
The backend filesystem on the datastorage is HDFS. It's technically a
proprietary implementation of HPE MapR (ezmerald data fabric) but it's just
a Hadoop setup at heart.
This datastorage cluster is mounted through a provided HPE/MapR loopback
NFS agent on our RHEL 7/8 samba servers (which they themselves are using
XFS). The NFSv3 mount point for each directory is shared out through the
Samba share path.
Yes... I know. NFS mount points being shared out through Samba is not
recommended. However, we don't have any other option as of current (FUSE
isn't supported yet with our encryption agent quite yet). It has however
worked relatively issue-free (aside from not being able to use ACL's and
some occasional ".nfs silly renames" lock file issues).
I've sanitized the smb.conf and narrowed it down to one share (there's
multiple, but they all use the same options)
[global]
workgroup = FOO
security = ads
passdb backend = tdbsam
kerberos method = secrets and keytab
dedicated keytab file = /etc/krb5.keytab
winbind refresh tickets = yes
#winbind offline logon = no
template homedir = /home/%U@%D
template shell = /bin/bash
realm = FOO.BAR.NET
password server = DCTESTSERVER.FOO.BAR.NET
idmap config FOO.BAR.NET : range = 1610000000 - 1620000000
idmap config FOO.BAR.NET : backend = ad
idmap config FOO.BAR.NET : schema_mode = rfc2307
idmap config * : range = 10000-999999
idmap config * : backend = tdb
winbind use default domain = no
winbind enum groups = yes
winbind enum users = yes
#current testing settings below for performance tuning
winbind max domain connections = 25
#winbind nested groups = no
winbind reconnect delay = 5
winbind request timeout = 120
name resolve order = host lmhosts bcast
[NAME-OF-SHARE-PATH]
path = /mapr/prod/NAME/OF/SHARE/PATH
writable = yes
browsable= yes
create mask = 0770
force create mode = 0770
directory mask = 0775
hide unreadable = yes
force group = "FOO\groupname"
valid users = +foo.bar.net\"groupname"
On Wed, Mar 15, 2023, 1:41 PM Jeremy Allison via samba <
samba at lists.samba.org> wrote:
> On Wed, Mar 15, 2023 at 04:53:16PM +0000, Rowland Penny via samba wrote:
> >
> >
> >On 15/03/2023 16:47, Jeremy Allison via samba wrote:
> >>On Wed, Mar 15, 2023 at 11:29:10AM -0400, CaptainTrips28 via samba
wrote:
> >>>Encountering a strange one... has been occuring since samba
4.15.x and
> >>>continues to occur even in 4.17.5 and 4.18
> >>>
> >>>Issue does not occur in earlier versions (4.13.x) and have
switched back
> >>>and forth between versions to confirm.
> >>>
> >>>Servers are winbind joined, PAM and krb5 auth, all seems to
work fine
> >>>(reading, writing, creating, moving) until trying to delete an
> >>>empty folder
> >>>on the share from Windows client. Folder briefly disappears,
but then
> >>>reappears once refreshed. I've confirmed no attributes set,
> >>>files/subfolders or hidden files are in the folder. Age of the
folder
> >>>doesn't matter, new empty folders, old empty folders...
issue occurs on
> >>>them regardless.
> >>>
> >>>Servers are all various RHEL releases, version 7.9, 8.4 and
8.6. Issue
> >>>occurs regardless if RHEL repo samba packages 4.15.x or Samba
> >>>source builds
> >>>(4.17.5 or 4.18). I did originally open a ticket with Redhat
but they
> >>>simply suggested it was a bug since it doesn't occur with
samba
> >>>4.13.x and
> >>>passed it onto their engineering team (haven't heard any
follow-up in
> >>>months since opening that ticket)
> >>>
> >>>All I can find is a rmdir_internal related error in the logs:
> >>>
> >>>"""""""
> >>>[2023/03/15 10:59:21.285508, 10, pid=8606,
effective(1617208032,
> >>>1617207642), real(1617208032, 0)]
> >>>../../source3/smbd/files.c:528(openat_pathref_fullname)
> >>>
> >>> openat_pathref_fullname: fsp [wrk/anly/testfolder/New folder]:
OK
> >>>
> >>>[2023/03/15 10:59:21.285573, 3, pid=8606,
effective(1617208032,
> >>>1617207642), real(1617208032, 0)]
> >>>../../source3/smbd/close.c:1134(rmdir_internals)
> >>>
> >>> rmdir_internals: couldn't remove directory
wrk/anly/testfolder/New
> >>>folder/New folder2 : Invalid argument
> >>>
> >>>[2023/03/15 10:59:21.285596, 5, pid=8606,
effective(1617208032,
> >>>1617207642), real(1617208032, 0)]
> >>>../../source3/smbd/files.c:1814(file_free)
> >>>
> >>> file_free: freed files structure 0 (3 used)
> >>>
> >>>[2023/03/15 10:59:21.285611, 5, pid=8606,
effective(1617208032,
> >>>1617207642), real(1617208032, 0)]
> >>>../../source3/smbd/close.c:1573(close_directory)
> >>>
> >>> close_directory: wrk/anly/testfolder/New folder/New folder2.
Delete on
> >>>close was set - deleting directory returned
NT_STATUS_INVALID_PARAMETER.
> >>>""""""
> >>
> >>That would normally mean the flags to unlinkat() were incorrect.
> >>
> >>Is this completely repeatable (i.e. any empty folder cannot
> >>be removed) or is it only sometimes or on some shares ?
> >>
> >
> >Very repeatable, create a directory on a share with a space in the
> >name, login with smbclient to the share and the only way to delete it
> >is to surround the directory name with double quotes, single quotes or
> >the escape character '\' will not work.
>
> Yeah but the original reporter uses a Windows client.
>
> What you're reporting here looks like a smbclient string
> quoting issue.
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
>
Jeremy Allison
2023-Mar-17 18:46 UTC
[Samba] Empty folder deletion issue - Samba 4.15 thru 4.18
On Fri, Mar 17, 2023 at 02:43:05PM -0400, CaptainTrips28 wrote:> So the filesystem thing is a bit detailed. It's Windows client ----> RHEL > Samba server ----> Hadoop data cluster via NFS mount. > The backend filesystem on the datastorage is HDFS. It's technically a > proprietary implementation of HPE MapR (ezmerald data fabric) but it's > just a Hadoop setup at heart. > This datastorage cluster is mounted through a provided HPE/MapR loopback > NFS agent on our RHEL 7/8 samba servers (which they themselves are using > XFS). The NFSv3 mount point for each directory is shared out through the > Samba share path. > Yes... I know. NFS mount points being shared out through Samba is not > recommended. However, we don't have any other option as of current (FUSE > isn't supported yet with our encryption agent quite yet). It has however > worked relatively issue-free (aside from not being able to use ACL's and > some occasional ".nfs silly renames" lock file issues). > I've sanitized the smb.conf and narrowed it down to one share (there's > multiple, but they all use the same options)OK, almost certainly it's the NFS mount causing the problem. Can you try and reproduce on a locally hosted share. I expect not. Given it's on an NFS mount, we can investigate but I'm guessing it'll be the telldir()/seekdir() error that seems to be endemic to NFS mounts.