samba - Apr 2022 - Odd "File exists" behavior when copy-pasting many files to an SMB exported Lustre FS

Hi all,

Resident wild-guy Christian here. In summary, I'm seeing an odd problem
from both Windows Explorer and Mac Finder when copy-pasting a large batch
of files to a Lustre-backed SMB share wherein the file manager appears to
enumerate over all the files to determine if they can be made in the
destination, makes 0-byte stubs of each file, and then proudly states that
the files already exist in the destination. This behavior is
less-than-desirable and I was hoping someone else here had run into it, or
someone better than I at diagnosing SMB weirdness could give me a hand
debugging. I'm inclined to think it's a Lustre idiosyncracy, since this
does not occur on a ZFS share.

Environment details:

Testparm output:
Server role: ROLE_STANDALONE
# Global parameters
[global]
disable spoolss = Yes
dns proxy = No
load printers = No
local master = No
log file = /var/log/samba/clients/%m.log
logging = syslog at 0 file
map to guest = Bad User
max log size = 100
max stat cache size = 65536
max xmit = 1048676
name resolve order = host wins bcast
printcap name = /dev/null
socket options = IPTOS_LOWDELAY TCP_NODELAY
username map cache time = 60
usershare path idmap config * : backend = tdb
include = /etc/samba/smb-shares.conf.local
printing = bsd


[smb-lustre]
access based share enum = Yes
case sensitive = Yes
create mask = 0666
delete veto files = Yes
directory mask = 0777
mangled names = no
path = /lfsmnt/mydfs
read only = No
strict sync = No
use sendfile = Yes
vfs objects = fileid catia fruit acl_xattr aio_pthread
fileid:algorithm = fsname
readdir_attr:aapl_rsize = no
readdir_attr:aapl_max_access = yes
readdir_attr:aapl_finder_info = yes
fruit:zero_file_id = yes
fruit:wipe_intentionally_left_blank_rfork = yes
fruit:veto_appledouble = no
fruit:resource = file
fruit:posix_rename = yes
fruit:nfs_aces = yes
fruit:model = MacSamba
fruit:metadata = netatalk
fruit:locking = none
fruit:encoding = private
fruit:delete_empty_adfiles = yes
fruit:copyfile = yes
fruit:aapl = yes

Lustre config is pretty simple, self-compiled Lustre 2.14, ZFS 2.0, kernel
4.19 on Debian 10, client mounts configured with defaults.

The more detailed summary;
I've got a folder called "lottafiles" that contains 817 files of
size 3KB.
In Mac Finder or Windows Explorer, when I drag that file to a new
destination on the SMB share, I get the pop-up dialog showing me the
progress of the operation. In windows, this shows it enumerating over each
file in the transfer with "calculating" in the transfer size bar. When
it's
done "calculating" I get an additional prompt about
"<number_of_files>
exist in the destination" that lets me overwrite/skip. Sure enough, the
destination folder has been created with 817 0-byte files, all with the
appropriate names.  The lowest number of 3K files I've see it happen with
thus far is 112.

Cheers,
Christian Kuntz

-- 
* 
<https://opendrives.ac-page.com/nabshow2022?utm_source=signature&utm_medium=email&utm_campaign=nab022&utm_content=bookameeting>
*
*Need a free NAB guest pass? Use code: LV56768 ($159 value)*

On Mon, Apr 04, 2022 at 03:26:03PM -0700, Christian Kuntz via samba
wrote:
>Hi all,
>
>Resident wild-guy Christian here. In summary, I'm seeing an odd problem
>from both Windows Explorer and Mac Finder when copy-pasting a large batch
>of files to a Lustre-backed SMB share wherein the file manager appears to
>enumerate over all the files to determine if they can be made in the
>destination, makes 0-byte stubs of each file, and then proudly states that
>the files already exist in the destination. This behavior is
>less-than-desirable and I was hoping someone else here had run into it, or
>someone better than I at diagnosing SMB weirdness could give me a hand
>debugging. I'm inclined to think it's a Lustre idiosyncracy, since
this
>does not occur on a ZFS share.
If it doesn't happen on a ZFS share, but does on a Lustre one then
it's certainly a Lustre oddity. Try looking at debug logs to figure
out what Lustre is doing that smbd doesn't like.