Michael Weiser
2022-Sep-22 14:38 UTC
[Samba] [lustre-discuss] Odd "File exists" behavior when copy-pasting many files to an SMB exported Lustre FS
Hello Daniel,> In the lustre.lov xattr, Lustre exposes layout information (ie. content > distribution across servers) to regular users. In some cases, it's also > possible to set the desired layout through this interface. LayoutAlright, that makes sense and indeed I can as unprivileged user write back at least the same value to lustre.lov without error: $ getfattr -n lustre.lov blarf # file: blarf lustre.lov=0s0AvRCwEAAABFJAEAAAAAAIQjBAACAAAAAAAQAAEAAADrXSYBAAAAAAAAAAAAAAAAAAAAAAgAAAA $ setfattr -n lustre.lov -v 0s0AvRCwEAAABFJAEAAAAAAIQjBAACAAAAAAAQAAEAAADrXSYBAAAAAAAAAAAAAAAAAAAAAAgAAAA= blarf <no error here>> information depends on the innards of the specific fs setup, and should > not be retained when moving files between different filesystems, hence > the hint in the docs.Then I guess it's time to give in and polish up the xattr-filtering VFS module I started work on so samba can replicate the xattr.conf workaround already documented.> So for a working setup, it boils down to either turning off 'ea support' > in Samba (on by default since 4.9), or turning on 'user_xattr' in the > filesystem.The first option would cement the notion in people's minds that somehow extended attributes cause problems and do not work reliably for yet another decade - I don't want that. The latter would still cause massive confusion, because lustre.lov would travel to Windows clients as LUSTRE.LOV and then back onto Lustre as user.lustre.lov and then back to Windows as USER.LUSTRE.LOV, polluting files everywhere with nonsensical information. So best to tackle the problem at the root and filter the attribute out on the way to the client, I guess. I'd try to give the VFS module some config option where one could specify a list of attributes to filter, possibly with wildcards so xattr.conf could be replicated exactly. Or is there a way to leverage libattr for exactly this functionality? It seems to be a hard dependency already even though only used if the system's fgetxattr is unavailable or not acceptable (macos). Is that determination correct?>From a quick glance at the code, attr_copy_action() could be leveraged to interrogate xattr.conf for what the actual libattr xattr copy functions attr_copy_fd()/attr_copy_file() would to with an attribute based on configuration in xattr.conf.Would this be desireable or an smb.conf option be preferred?> >> No. Lustre is returning "fictional" EA's that > >> cannot be set. Linux filesystems that don't have > >> EA's don't do that.> On a Lustre system, the lustre.lov xattr can be set alright without > receiving an error. But that's not what Samba does here. Instead, it > tries to copy any user-readable xattr, and prepends a 'user.' prefix to > any name unless it's already there. This only works if the filesystem > has been mounted with the 'user_xattr' option.So just to make a final push for a fix of that error code path: If I had a file with one of the XDG user attributes[1] such as user.xdg.origin.url on my local ext4 filesystem and copied that to Lustre with user_xattr enabled, all would be well. If user_xattr weren't enabled on the Lustre, I'd again run into this "file already exists" scenario and be confused. The alternative (with the error code path removing the file from the failed creation attempt) would be for the extended attribute to vanish silently during copy which at least in my mind would be preferable. [1] https://www.freedesktop.org/wiki/CommonExtendedAttributes/ -- Thanks, Michael ________________________________________ From: Daniel Kobras <kobras at puzzle-itc.de> Sent: 22 September 2022 15:53:23 To: Michael Weiser; Jeremy Allison Cc: lustre-discuss at lists.lustre.org; samba at lists.samba.org Subject: Re: [lustre-discuss] [Samba] Odd "File exists" behavior when copy-pasting many files to an SMB exported Lustre FS Caution: External email. Do not open attachments or click links, unless this email comes from a known sender and you know the content is safe. Hi Micha, hi Jeremy! Am 22.09.22 um 11:21 schrieb Michael Weiser:> [from the docs]: >> If a client(s) will be mounted on several file systems, add the following line to /etc/xattr.conf > file to avoid problems when files are moved between the file systems: lustre.* skip" > > What exactly were those problems hinted at in the documentation? > Is the visibility of the lustre.lov attribute for unprivileged users actually needed for anything? > Can exposing it to unprivileged users be switched off Lustre-side?In the lustre.lov xattr, Lustre exposes layout information (ie. content distribution across servers) to regular users. In some cases, it's also possible to set the desired layout through this interface. Layout information depends on the innards of the specific fs setup, and should not be retained when moving files between different filesystems, hence the hint in the docs.> Jeremy wrote: > >> Lutre really should not be exposing EA's to callers if >> it doesn't actually support EA's.Just to clarify as Micha's original post had it hidden in parenthesis: Lustre does support xattrs in general, and it does support the 'user.*' namespace. It's just that the latter needs to be enabled explicitly with the 'user_xattr' mount option. By default, access to 'user.*' xattrs is rejected with EOPNOTSUPP.>> No. Lustre is returning "fictional" EA's that >> cannot be set. Linux filesystems that don't have >> EA's don't do that.On a Lustre system, the lustre.lov xattr can be set alright without receiving an error. But that's not what Samba does here. Instead, it tries to copy any user-readable xattr, and prepends a 'user.' prefix to any name unless it's already there. This only works if the filesystem has been mounted with the 'user_xattr' option. So for a working setup, it boils down to either turning off 'ea support' in Samba (on by default since 4.9), or turning on 'user_xattr' in the filesystem. Kind regards, Daniel -- Daniel Kobras Principal Architect Puzzle ITC Deutschland +49 7071 14316 0 https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.puzzle-itc.de%2F&data=05%7C01%7Cmichael.weiser%40atos.net%7C2499367eca8543abdf5d08da9ca1d25a%7C33440fc6b7c7412cbb730e70b0198d5a%7C0%7C0%7C637994516076263555%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=jlZDne%2FdyPXTLjFd1O2HVgTFYk7e7RCFhWm31e78B6Y%3D&reserved=0 -- Puzzle ITC Deutschland GmbH Sitz der Gesellschaft: Eisenbahnstra?e 1, 72072 T?bingen Eingetragen am Amtsgericht Stuttgart HRB 765802 Gesch?ftsf?hrer: Lukas Kallies, Daniel Kobras, Mark Pr?hl