samba - Aug 2017 - Friendly Reminder: Would you please comment on my findings?

Hi Alain,

many thanks for responding! :-)

Unfortunately, it's not that easy: In my case, the performance 
difference/regression between SMB1 and SMB2/3 is unrelated to any 
smb.conf settings, and even unrelated to Samba versions. It looks like 
Microsoft has introduced a performance bug with the way that their 
Windows SMB2/3 clients handle files with many entries, and this 
performance hit becomes excessively apparent once you try to copy a huge 
number of small files - please see the Wireshark SMBx packet analysis here:

https://lists.samba.org/archive/samba/2017-July/209749.html
https://lists.samba.org/archive/samba/2017-July/209750.html
https://lists.samba.org/archive/samba/2017-July/209751.html

for the details of how the Windows 10 client sends completely 
inefficient SMB2/3 commands in to to list the files in the current 
directory. I'm basically looking for confirmation that my assessment is 
correct, and especially for help when addressing this issue to Microsoft...

Thanks anyway & best regards
Andreas


Am 18.08.2017 um 14:07 schrieb Alain Deleglise via
samba:
> Hi Andreas, Maybe somtehing you already tried but, just in case : try 
> commenting "hide unreadable" or set it to "no".
Regards, Alain
> DELEGLISE Architecte Infrastructure Responsable Avant-Vente Lille 
> alter way <http://www.alterway.fr/> @alterway.fr 
> <https://twitter.com/alterway> EuraTechnologies, 166 bis avenue de 
> Bretagne, 59000 LILLE 2017-08-18 12:51 GMT+02:00 awl1 via samba 
> <samba at lists.samba.org>:
>> Hello one more time, unfortunately, one more time, another two weeks 
>> have passed without any reaction from neither Jeremy in person nor 
>> the Samba team/community. I am quite a bit astonished, disappointed 
>> and negatively surprised indeed: Did I do anything wrong when raising 
>> my issue with you? Everybody else asking questions or raising issues 
>> on this list seems to receive great advice, while I no longer get any 
>> reactions at all... Is my issue (a huge performance regression when 
>> copying a large number of small files both from and to a share with 
>> SMB2/3 as compared to SMB1 using a Windows 10 client) too 
>> technical/complicated (can hardly believe that), too hard to 
>> understand, or simply (for whatever else reasons) otherwise not 
>> welcome!? Note that - as requested by Jeremy on July 14th: "Look
into
>> it some more (I'm a bit busy with other stuff right now)." - I
think
>> have successfully tracked down the performance regression to at least 
>> one major root cause, which is a client-side issue in SMB2/SMB3 on 
>> the Windows platform - see my several posts on this list dated July 
>> 15th: https://lists.samba.org/archive/samba/2017-July/209749.html 
>> https://lists.samba.org/archive/samba/2017-July/209750.html 
>> https://lists.samba.org/archive/samba/2017-July/209751.html So 
>> basically at this point this analysis seems mostly done and I am 
>> "only" waiting for Jeremy and/or the Samba team to look into
my
>> assessment and provide your feedback: a) Do you think that my 
>> analysis as laid out in this full thread, particularly the posts 
>> referenced above, is valid? b) In case you think it is, how can we 
>> proceed in terms of making Microsoft aware of their performance 
>> regression? As I feel that it would be much more likely to be 
>> successful, I would like the Samba team to raise this issue with your 
>> Microsoft contacts, rather than me as a private end customer
(there's
>> no company backing me with regards to this) complaining about this 
>> (which will only have a very low chance of success)... Eagerly 
>> looking forward to some feedback and support from the Samba 
>> community... Many thanks for your understanding and support and best 
>> regards Andreas Am 04.08.2017 um 16:49 schrieb awl1:
>>> Hello again, Jeremy (and hello, fellow Samba experts - maybe
someone
>>> of you can comment on this, too!?), please accept my apologies in 
>>> case you are on vacation - otherwise in case you are still terribly
>>> busy, please at least reply very shortly stating when you will 
>>> finally have some time to look into this issue and my findings... 
>>> Sorry to still be such a pain in your neck(s), but another two
weeks
>>> passed, and I still did not receive any feedback on my findings
(and
>>> questions re making Microsoft aware of their issue in case my 
>>> analysis is correct). Looking forward to your assessment and 
>>> proposal on how to move forward... ;-) Thanks a million, best 
>>> regards & have a great weekend Andreas Am 21.07.2017 um 15:56 
>>> schrieb awl1:
>>>> Hello Jeremy, Am 14.07.2017 um 23:33 schrieb Jeremy Allison:
>>>>> Look into it some more (I'm a bit busy with other stuff
right now).
>>>> when will you be available again to look into my latest
findings
>>>> (see previous posts) and provide your feedback? If you can
confirm
>>>> that it is Microsoft to blame, what would you propose to raise
this
>>>> issue with them? Wouldn't this ideally be done by somebody
from the
>>>> Samba team rather than me? Many thanks & best regards
Andreas
>> -- To unsubscribe from this list go to the following URL and read the 
>> instructions: https://lists.samba.org/mailman/options/samba

I have read some of your previous e-mails.

I told you that because we suffered the same strange situation, upgrading
from samba 3.X to 4.X.

Finally, did you, in your many attempts, try to copy those large amounts of
small files with "hide unreadable = no" or not, just to be sure ?




Alain DELEGLISE
Architecte Infrastructure
Responsable Avant-Vente Lille
alter way <http://www.alterway.fr/> @alterway.fr
<https://twitter.com/alterway>

T +33 1 41 16 83 42 - M +33 6 83 68 90 86
EuraTechnologies, 166 bis avenue de Bretagne, 59000 LILLE

<https://ci3.googleusercontent.com/proxy/-3qH8WcFM0f0828T7THm5CoQdjB9b5ejfV0V49zEvYg8nGS-_YYdtVbzdat4nOsWd0e3SOXuv8umFSxVqQ=s0-d-e1-ft#http://www.alterway.fr/signatures/url/1>

2017-08-18 14:42 GMT+02:00 awl1 <awl1 at mnet-online.de>:
> Hi Alain,
>
> many thanks for responding! :-)
>
> Unfortunately, it's not that easy: In my case, the performance
> difference/regression between SMB1 and SMB2/3 is unrelated to any smb.conf
> settings, and even unrelated to Samba versions. It looks like Microsoft has
> introduced a performance bug with the way that their Windows SMB2/3 clients
> handle files with many entries, and this performance hit becomes
> excessively apparent once you try to copy a huge number of small files -
> please see the Wireshark SMBx packet analysis here:
>
> https://lists.samba.org/archive/samba/2017-July/209749.html
> https://lists.samba.org/archive/samba/2017-July/209750.html
> https://lists.samba.org/archive/samba/2017-July/209751.html
>
> for the details of how the Windows 10 client sends completely inefficient
> SMB2/3 commands in to to list the files in the current directory. I'm
> basically looking for confirmation that my assessment is correct, and
> especially for help when addressing this issue to Microsoft...
>
> Thanks anyway & best regards
> Andreas
>
>
> Am 18.08.2017 um 14:07 schrieb Alain Deleglise via samba:
>
>> Hi Andreas, Maybe somtehing you already tried but, just in case : try
>> commenting "hide unreadable" or set it to "no".
Regards, Alain DELEGLISE
>> Architecte Infrastructure Responsable Avant-Vente Lille alter way <
>> http://www.alterway.fr/> @alterway.fr
<https://twitter.com/alterway>
>> EuraTechnologies, 166 bis avenue de Bretagne, 59000 LILLE 2017-08-18
12:51
>> GMT+02:00 awl1 via samba <samba at lists.samba.org>:
>>
>>> Hello one more time, unfortunately, one more time, another two
weeks
>>> have passed without any reaction from neither Jeremy in person nor
the
>>> Samba team/community. I am quite a bit astonished, disappointed and
>>> negatively surprised indeed: Did I do anything wrong when raising
my issue
>>> with you? Everybody else asking questions or raising issues on this
list
>>> seems to receive great advice, while I no longer get any reactions
at
>>> all... Is my issue (a huge performance regression when copying a
large
>>> number of small files both from and to a share with SMB2/3 as
compared to
>>> SMB1 using a Windows 10 client) too technical/complicated (can
hardly
>>> believe that), too hard to understand, or simply (for whatever else
>>> reasons) otherwise not welcome!? Note that - as requested by Jeremy
on July
>>> 14th: "Look into it some more (I'm a bit busy with other
stuff right now)."
>>> - I think have successfully tracked down the performance regression
to at
>>> least one major root cause, which is a client-side issue in
SMB2/SMB3 on
>>> the Windows platform - see my several posts on this list dated July
15th:
>>> https://lists.samba.org/archive/samba/2017-July/209749.html
>>> https://lists.samba.org/archive/samba/2017-July/209750.html
>>> https://lists.samba.org/archive/samba/2017-July/209751.html So
>>> basically at this point this analysis seems mostly done and I am
"only"
>>> waiting for Jeremy and/or the Samba team to look into my assessment
and
>>> provide your feedback: a) Do you think that my analysis as laid out
in this
>>> full thread, particularly the posts referenced above, is valid? b)
In case
>>> you think it is, how can we proceed in terms of making Microsoft
aware of
>>> their performance regression? As I feel that it would be much more
likely
>>> to be successful, I would like the Samba team to raise this issue
with your
>>> Microsoft contacts, rather than me as a private end customer
(there's no
>>> company backing me with regards to this) complaining about this
(which will
>>> only have a very low chance of success)... Eagerly looking forward
to some
>>> feedback and support from the Samba community... Many thanks for
your
>>> understanding and support and best regards Andreas Am 04.08.2017 um
16:49
>>> schrieb awl1:
>>>
>>>> Hello again, Jeremy (and hello, fellow Samba experts - maybe
someone of
>>>> you can comment on this, too!?), please accept my apologies in
case you are
>>>> on vacation - otherwise in case you are still terribly busy,
please at
>>>> least reply very shortly stating when you will finally have
some time to
>>>> look into this issue and my findings... Sorry to still be such
a pain in
>>>> your neck(s), but another two weeks passed, and I still did not
receive any
>>>> feedback on my findings (and questions re making Microsoft
aware of their
>>>> issue in case my analysis is correct). Looking forward to your
assessment
>>>> and proposal on how to move forward... ;-) Thanks a million,
best regards &
>>>> have a great weekend Andreas Am 21.07.2017 um 15:56 schrieb
awl1:
>>>>
>>>>> Hello Jeremy, Am 14.07.2017 um 23:33 schrieb Jeremy
Allison:
>>>>>
>>>>>> Look into it some more (I'm a bit busy with other
stuff right now).
>>>>>>
>>>>> when will you be available again to look into my latest
findings (see
>>>>> previous posts) and provide your feedback? If you can
confirm that it is
>>>>> Microsoft to blame, what would you propose to raise this
issue with them?
>>>>> Wouldn't this ideally be done by somebody from the
Samba team rather than
>>>>> me? Many thanks & best regards Andreas
>>>>>
>>>> -- To unsubscribe from this list go to the following URL and
read the
>>> instructions: https://lists.samba.org/mailman/options/samba
>>>
>>
>

Hi again, Alain,

as Jeremy and Rowland advised to strip down my smb.conf as much as 
possible, I haven't yet tried with "hide unreadable = no".

Do you indeed mean to say that setting "hide unreadable = no" on the 
Samba *server* might indeed cause a difference in the way the Windows 
*client* sends its SMB2_FIND_ID_BOTH_DIRECTORY_INFO commands? This would 
mean that client and server would have to previously explicitly 
negotiate about the fact whether the server should hide unreadable files 
("hide unreadable"), and make the Windows client decide to send a 
different client-side SMB2_FIND_* command with less performance penalty 
in case the server does not need to hide any files!?

Nevertheless, I will try to set "hide unreadable = no" and report back
later tonight if I see any changes...

Many thanks so far & best regards
Andreas


Am 18.08.2017 um 14:50 schrieb Alain Deleglise:
> I have read some of your previous e-mails.
>
> I told you that because we suffered the same strange situation, 
> upgrading from samba 3.X to 4.X.
>
> Finally, did you, in your many attempts, try to copy those large 
> amounts of small files with "hide unreadable = no" or not, just
to be
> sure ?
>
>
>
>
> Alain DELEGLISE
> ArchitecteInfrastructure
> Responsable Avant-Vente Lille
> alter way <http://www.alterway.fr/>@alterway.fr 
> <https://twitter.com/alterway>
>
> T +33 1 41 16 83 42 - M +33 6 83 68 90 86
> EuraTechnologies, 166 bis avenue de Bretagne, 59000 LILLE
>
>
<https://ci3.googleusercontent.com/proxy/-3qH8WcFM0f0828T7THm5CoQdjB9b5ejfV0V49zEvYg8nGS-_YYdtVbzdat4nOsWd0e3SOXuv8umFSxVqQ=s0-d-e1-ft#http://www.alterway.fr/signatures/url/1>
>
> 2017-08-18 14:42 GMT+02:00 awl1 <awl1 at mnet-online.de 
> <mailto:awl1 at mnet-online.de>>:
>
>     Hi Alain,
>
>     many thanks for responding! :-)
>
>     Unfortunately, it's not that easy: In my case, the performance
>     difference/regression between SMB1 and SMB2/3 is unrelated to any
>     smb.conf settings, and even unrelated to Samba versions. It looks
>     like Microsoft has introduced a performance bug with the way that
>     their Windows SMB2/3 clients handle files with many entries, and
>     this performance hit becomes excessively apparent once you try to
>     copy a huge number of small files - please see the Wireshark SMBx
>     packet analysis here:
>
>     https://lists.samba.org/archive/samba/2017-July/209749.html
>     <https://lists.samba.org/archive/samba/2017-July/209749.html>
>     https://lists.samba.org/archive/samba/2017-July/209750.html
>     <https://lists.samba.org/archive/samba/2017-July/209750.html>
>     https://lists.samba.org/archive/samba/2017-July/209751.html
>     <https://lists.samba.org/archive/samba/2017-July/209751.html>
>
>     for the details of how the Windows 10 client sends completely
>     inefficient SMB2/3 commands in to to list the files in the current
>     directory. I'm basically looking for confirmation that my
>     assessment is correct, and especially for help when addressing
>     this issue to Microsoft...
>
>     Thanks anyway & best regards
>     Andreas
>
>
>     Am 18.08.2017 um 14:07 schrieb Alain Deleglise via samba:
>
>         Hi Andreas, Maybe somtehing you already tried but, just in
>         case : try commenting "hide unreadable" or set it to
"no".
>         Regards, Alain DELEGLISE Architecte Infrastructure Responsable
>         Avant-Vente Lille alter way <http://www.alterway.fr/>
>         @alterway.fr <http://alterway.fr>
>         <https://twitter.com/alterway> EuraTechnologies, 166 bis
>         avenue de Bretagne, 59000 LILLE 2017-08-18 12:51 GMT+02:00
>         awl1 via samba <samba at lists.samba.org
>         <mailto:samba at lists.samba.org>>:
>
>             Hello one more time, unfortunately, one more time, another
>             two weeks have passed without any reaction from neither
>             Jeremy in person nor the Samba team/community. I am quite
>             a bit astonished, disappointed and negatively surprised
>             indeed: Did I do anything wrong when raising my issue with
>             you? Everybody else asking questions or raising issues on
>             this list seems to receive great advice, while I no longer
>             get any reactions at all... Is my issue (a huge
>             performance regression when copying a large number of
>             small files both from and to a share with SMB2/3 as
>             compared to SMB1 using a Windows 10 client) too
>             technical/complicated (can hardly believe that), too hard
>             to understand, or simply (for whatever else reasons)
>             otherwise not welcome!? Note that - as requested by Jeremy
>             on July 14th: "Look into it some more (I'm a bit busy
with
>             other stuff right now)." - I think have successfully
>             tracked down the performance regression to at least one
>             major root cause, which is a client-side issue in
>             SMB2/SMB3 on the Windows platform - see my several posts
>             on this list dated July 15th:
>             https://lists.samba.org/archive/samba/2017-July/209749.html
>            
<https://lists.samba.org/archive/samba/2017-July/209749.html>
>             https://lists.samba.org/archive/samba/2017-July/209750.html
>            
<https://lists.samba.org/archive/samba/2017-July/209750.html>
>             https://lists.samba.org/archive/samba/2017-July/209751.html
>            
<https://lists.samba.org/archive/samba/2017-July/209751.html>
>             So basically at this point this analysis seems mostly done
>             and I am "only" waiting for Jeremy and/or the Samba
team
>             to look into my assessment and provide your feedback: a)
>             Do you think that my analysis as laid out in this full
>             thread, particularly the posts referenced above, is valid?
>             b) In case you think it is, how can we proceed in terms of
>             making Microsoft aware of their performance regression? As
>             I feel that it would be much more likely to be successful,
>             I would like the Samba team to raise this issue with your
>             Microsoft contacts, rather than me as a private end
>             customer (there's no company backing me with regards to
>             this) complaining about this (which will only have a very
>             low chance of success)... Eagerly looking forward to some
>             feedback and support from the Samba community... Many
>             thanks for your understanding and support and best regards
>             Andreas Am 04.08.2017 um 16:49 schrieb awl1:
>
>                 Hello again, Jeremy (and hello, fellow Samba experts -
>                 maybe someone of you can comment on this, too!?),
>                 please accept my apologies in case you are on vacation
>                 - otherwise in case you are still terribly busy,
>                 please at least reply very shortly stating when you
>                 will finally have some time to look into this issue
>                 and my findings... Sorry to still be such a pain in
>                 your neck(s), but another two weeks passed, and I
>                 still did not receive any feedback on my findings (and
>                 questions re making Microsoft aware of their issue in
>                 case my analysis is correct). Looking forward to your
>                 assessment and proposal on how to move forward... ;-)
>                 Thanks a million, best regards & have a great weekend
>                 Andreas Am 21.07.2017 um 15:56 schrieb awl1:
>
>                     Hello Jeremy, Am 14.07.2017 um 23:33 schrieb
>                     Jeremy Allison:
>
>                         Look into it some more (I'm a bit busy with
>                         other stuff right now).
>
>                     when will you be available again to look into my
>                     latest findings (see previous posts) and provide
>                     your feedback? If you can confirm that it is
>                     Microsoft to blame, what would you propose to
>                     raise this issue with them? Wouldn't this ideally
>                     be done by somebody from the Samba team rather
>                     than me? Many thanks & best regards Andreas
>
>             -- To unsubscribe from this list go to the following URL
>             and read the instructions:
>             https://lists.samba.org/mailman/options/samba
>             <https://lists.samba.org/mailman/options/samba>
>
>
>