samba - Dec 2018 - vfs_fruit causes delay in listing directories for Windows clients

Listing directories with many files (10000+) from a Windows client is 
nociceably slower when vfs_fruit is enabled on the samba server compared 
to the same setup without vfs_fruit.

On my setup it's roughly 2.5 times slower. To me it looks like this is 
caused by the getxattr call which is only present with vfs_fruit 
activated and introduces an additional delay of ~ 0.00033 s per listed file.

strace -T reveals the following with vfs_fruit activated:

newfstatat(35, "03915", {st_mode=S_IFREG|0660, st_size=1265, ...}, 
AT_SYMLINK_NOFOLLOW) = 0 <0.000243>
fcntl(20, F_SETLKW, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=8368, 
l_len=1}) = 0 <0.000010>
fcntl(20, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=8368, 
l_len=1}) = 0 <0.000009>
stat("stroth/testfiles_10000-50000/10000/03915",
{st_mode=S_IFREG|0660,
st_size=1265, ...}) = 0 <0.000312>
getxattr("stroth/testfiles_10000-50000/10000/03915", 
"user.org.netatalk.Metadata", 0x55803d5c9a90, 402) = -1 ENODATA (No
data
available) <0.000348>
listxattr("testfiles_10000-50000/10000/03915", "", 1024) = 0
<0.000345>

a trace of the same directory listing with vfs_fruit disabled is faster 
and doesn't show the getxattr call:

newfstatat(36, "03239", {st_mode=S_IFREG|0660, st_size=303, ...}, 
AT_SYMLINK_NOFOLLOW) = 0 <0.000252>
fcntl(20, F_SETLKW, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=19600, 
l_len=1}) = 0 <0.000008>
fcntl(20, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=19600, 
l_len=1}) = 0 <0.000007>
stat("stroth/testfiles_10000-50000/10000/03239",
{st_mode=S_IFREG|0660,
st_size=303, ...}) = 0 <0.000419>
listxattr("testfiles_10000-50000/10000/03239", "", 1024) = 0
<0.000355>


My goal with activating vfs_fruit was to speed up directory listings for 
Mac clients, which works. Can the accompanying slowdown for Windows 
clients be avoided?

Thank you for any insights,
Stephan

Hai, 

Can you tell the following that helps. 

OS ?   
Samba Version?  ( pre-builded or source ) 
Kernel version? 

I think you need to wait for 4.9.4 if all patches ive seen on vfs_fruit are
getting in.
These might help. Go through bugzilla, you might find more about this here. 


Greetz, 

Louis

 
> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens 
> Stephan Roth via samba
> Verzonden: woensdag 12 december 2018 15:35
> Aan: samba at lists.samba.org
> Onderwerp: [Samba] vfs_fruit causes delay in listing 
> directories for Windows clients
> 
> Listing directories with many files (10000+) from a Windows client is 
> nociceably slower when vfs_fruit is enabled on the samba 
> server compared 
> to the same setup without vfs_fruit.
> 
> On my setup it's roughly 2.5 times slower. To me it looks 
> like this is 
> caused by the getxattr call which is only present with vfs_fruit 
> activated and introduces an additional delay of ~ 0.00033 s 
> per listed file.
> 
> strace -T reveals the following with vfs_fruit activated:
> 
> newfstatat(35, "03915", {st_mode=S_IFREG|0660, st_size=1265,
...},
> AT_SYMLINK_NOFOLLOW) = 0 <0.000243>
> fcntl(20, F_SETLKW, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=8368, 
> l_len=1}) = 0 <0.000010>
> fcntl(20, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=8368, 
> l_len=1}) = 0 <0.000009>
> stat("stroth/testfiles_10000-50000/10000/03915", 
> {st_mode=S_IFREG|0660, 
> st_size=1265, ...}) = 0 <0.000312>
> getxattr("stroth/testfiles_10000-50000/10000/03915", 
> "user.org.netatalk.Metadata", 0x55803d5c9a90, 402) = -1 
> ENODATA (No data 
> available) <0.000348>
> listxattr("testfiles_10000-50000/10000/03915", "",
1024) = 0
> <0.000345>
> 
> a trace of the same directory listing with vfs_fruit disabled 
> is faster 
> and doesn't show the getxattr call:
> 
> newfstatat(36, "03239", {st_mode=S_IFREG|0660, st_size=303, ...},
> AT_SYMLINK_NOFOLLOW) = 0 <0.000252>
> fcntl(20, F_SETLKW, {l_type=F_RDLCK, l_whence=SEEK_SET, 
> l_start=19600, 
> l_len=1}) = 0 <0.000008>
> fcntl(20, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, 
> l_start=19600, 
> l_len=1}) = 0 <0.000007>
> stat("stroth/testfiles_10000-50000/10000/03239", 
> {st_mode=S_IFREG|0660, 
> st_size=303, ...}) = 0 <0.000419>
> listxattr("testfiles_10000-50000/10000/03239", "",
1024) = 0
> <0.000355>
> 
> 
> My goal with activating vfs_fruit was to speed up directory 
> listings for 
> Mac clients, which works. Can the accompanying slowdown for Windows 
> clients be avoided?
> 
> Thank you for any insights,
> Stephan
> 
> 
> -- 
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>

On Wed, Dec 12, 2018 at 03:35:00PM +0100, Stephan Roth via samba
wrote:
>My goal with activating vfs_fruit was to speed up directory listings 
>for Mac clients, which works. Can the accompanying slowdown for 
>Windows clients be avoided?
yeah, I guess so, but somebody has to dig through the relevant codepaths for a 
few days to implement the optimisations.

-slow

-- 
Ralph Boehme, Samba Team                https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/
GPG-Fingerprint   FAE2C6088A24252051C559E4AA1E9B7126399E46

On Wed, Dec 12, 2018 at 04:37:43PM +0100, Ralph Böhme via samba
wrote:
>On Wed, Dec 12, 2018 at 03:35:00PM +0100, Stephan Roth via samba wrote:
>>My goal with activating vfs_fruit was to speed up directory listings 
>>for Mac clients, which works. Can the accompanying slowdown for 
>>Windows clients be avoided?
>
>yeah, I guess so, but somebody has to dig through the relevant 
>codepaths for a few days to implement the optimisations.
thinking about it, most codepaths already do avoid special processing if not in 
AAPL mode or when not operating on a stream.

There's one exception however: fetching the file creation date from Netatalk
metadata when in "fruit:metadata = netatalk" mode which is the
default.

"fruit:metadata = stream" would get rid of this. Check your setup
carefully to
decide which mode you need.

-slow

-- 
Ralph Boehme, Samba Team                https://samba.org/
Samba Developer, SerNet GmbH   https://sernet.de/en/samba/
GPG-Fingerprint   FAE2C6088A24252051C559E4AA1E9B7126399E46

Hello Louis,

Thanks for the ideas, I checked for vfs_fruit on Bugzilla before 
bothering the list and I couldn't find anything related to the problem I 
observed.

Anyway, so far it seems the code would have to be adapted for clients 
not negotiating AAPL (see Ralph's answer to my question).

Cheers,
Stephan

On 12.12.2018 15:58, L.P.H. van Belle via samba wrote:
> Hai,
> 
> Can you tell the following that helps.
> 
> OS ?
> Samba Version?  ( pre-builded or source )
> Kernel version?
> 
> I think you need to wait for 4.9.4 if all patches ive seen on vfs_fruit are
getting in.
> These might help. Go through bugzilla, you might find more about this here.
> 
> 
> Greetz,
> 
> Louis
> 
>   
> 
>> -----Oorspronkelijk bericht-----
>> Van: samba [mailto:samba-bounces at lists.samba.org] Namens
>> Stephan Roth via samba
>> Verzonden: woensdag 12 december 2018 15:35
>> Aan: samba at lists.samba.org
>> Onderwerp: [Samba] vfs_fruit causes delay in listing
>> directories for Windows clients
>>
>> Listing directories with many files (10000+) from a Windows client is
>> nociceably slower when vfs_fruit is enabled on the samba
>> server compared
>> to the same setup without vfs_fruit.
>>
>> On my setup it's roughly 2.5 times slower. To me it looks
>> like this is
>> caused by the getxattr call which is only present with vfs_fruit
>> activated and introduces an additional delay of ~ 0.00033 s
>> per listed file.
>>
>> strace -T reveals the following with vfs_fruit activated:
>>
>> newfstatat(35, "03915", {st_mode=S_IFREG|0660, st_size=1265,
...},
>> AT_SYMLINK_NOFOLLOW) = 0 <0.000243>
>> fcntl(20, F_SETLKW, {l_type=F_RDLCK, l_whence=SEEK_SET, l_start=8368,
>> l_len=1}) = 0 <0.000010>
>> fcntl(20, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=8368,
>> l_len=1}) = 0 <0.000009>
>> stat("stroth/testfiles_10000-50000/10000/03915",
>> {st_mode=S_IFREG|0660,
>> st_size=1265, ...}) = 0 <0.000312>
>> getxattr("stroth/testfiles_10000-50000/10000/03915",
>> "user.org.netatalk.Metadata", 0x55803d5c9a90, 402) = -1
>> ENODATA (No data
>> available) <0.000348>
>> listxattr("testfiles_10000-50000/10000/03915", "",
1024) = 0
>> <0.000345>
>>
>> a trace of the same directory listing with vfs_fruit disabled
>> is faster
>> and doesn't show the getxattr call:
>>
>> newfstatat(36, "03239", {st_mode=S_IFREG|0660, st_size=303,
...},
>> AT_SYMLINK_NOFOLLOW) = 0 <0.000252>
>> fcntl(20, F_SETLKW, {l_type=F_RDLCK, l_whence=SEEK_SET,
>> l_start=19600,
>> l_len=1}) = 0 <0.000008>
>> fcntl(20, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET,
>> l_start=19600,
>> l_len=1}) = 0 <0.000007>
>> stat("stroth/testfiles_10000-50000/10000/03239",
>> {st_mode=S_IFREG|0660,
>> st_size=303, ...}) = 0 <0.000419>
>> listxattr("testfiles_10000-50000/10000/03239", "",
1024) = 0
>> <0.000355>
>>
>>
>> My goal with activating vfs_fruit was to speed up directory
>> listings for
>> Mac clients, which works. Can the accompanying slowdown for Windows
>> clients be avoided?
>>
>> Thank you for any insights,
>> Stephan